Metaprogramming

CS251ProgrammingLanguagesFall2017,LynTurbakDepartmentofComputerScienceWellesleyCollege

TheseslidesborrowheavilyfromBenWood’sFall‘15slides.

Howtoimplementaprogramminglanguage

InterpretaBonAninterpreterwriDenintheimplementaBonlanguagereadsaprogramwriDeninthesourcelanguageandevaluatesit.

TranslaBon(a.k.a.compilaBon)Antranslator(a.k.a.compiler)wriDenintheimplementaBonlanguagereadsaprogramwriDeninthesourcelanguageandtranslatesittoanequivalentprograminthetargetlanguage.

ButnowweneedimplementaBonsof:implementaBonlanguagetargetlanguage

Metaprogramming 2

Metaprogramming:InterpretaBon

InterpreterforlanguageLonmachineM

MachineMPrograminlanguageL

Metaprogramming 3

Interpreters

Data

Output

Source Program

Interpreter =

virtual machine

Metaprogramming 4

Metaprogramming:TranslaBon

InterpreterforlanguageBonmachineM

MachineM

PrograminlanguageA AtoBtranslator

PrograminlanguageB

Metaprogramming 5

if (x == 0) { x = x + 1; } ...

cmp (1000), $0 bne L add (1000), $1 L: ...

C Source Program C Compiler

x86 Target Program

Compiler

x86 Target Program

x86 computer

Data

Output

Metaprogramming 6

ThankstoBenWoodfortheseandfollowingpictures

InterpretersvsCompilers

InterpretersNoworkaheadofLmeIncrementalmaybeinefficient

CompilersAllworkaheadofLmeSeewholeprogram(ormoreofprogram)TimeandresourcesforanalysisandopLmizaLon

Metaprogramming 7

JavaCompiler

if (x == 0) { x = x + 1; } ...

load 0 ifne L load 0 inc store 0 L: ...

Source Program Java Compiler

Target Program

(compare compiled C to compiled Java) Metaprogramming 8

Compilers...whoseoutputisinterpreted

Target Program Java

Virtual Machine

Data

Output

Source Program Java Compiler

Target Program

Doesn’tthislookfamiliar? Metaprogramming 9

Interpreters...thatusecompilers.

Target Program

Virtual Machine

Data

Output

Source Program Compiler

Metaprogramming 10

JITCompilersandOpBmizaBon

Target Program

Virtual

Machine Data

Output

Just In Time Compiler

PerformanceMonitor

Source Program

Compiler

•  HotSpotJVM•  JikesRVM•  SpiderMonkey•  v8•  Transmeta•  ...

Metaprogramming 11

VirtualMachineModel

High-LevelLanguageProgram

VirtualMachineLanguage

Bytecodecompiler

Virtualmachine(interpreter)

JITcompiler

runBme

compileBme

Ahead-of-Lmecompiler

NaLveMachineLanguage

Metaprogramming 12

TypicalCompiler

Source Program

Lexical Analyzer

Syntax Analyzer

Semantic Analyzer

Intermediate Code Generator

Code Optimizer

Code Generator

Target Program

Analysis

Synthesis

Metaprogramming 13

Howtoimplementaprogramminglanguage

InterpreterRuleP-in-Lprogram Linterpretermachine

PmachineTranslatorRule

P-in-Sprogram S-to-TtranslatormachineP-in-Tprogram

Candescribebyderivinga“proof”oftheimplementaLonusingtheseinferencerules:

Metaprogramming 14

ImplementaBonDerivaBonExample

Provehowtoimplementa"251webpagemachine"using:

•  251-web-page-in-HTMLprogram(awebpagewriDeninHTML)•  HTML-interpreter-in-Cprogram(awebbrowserwriDeninC)•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)

Nopeakingahead!

Metaprogramming 15

ImplementaBonDerivaBonExampleSoluBon

Wecanomitsomeoccurrencesof“program”and“machine”:

Metaprogramming 16

ImplementaBonDerivaBonAreTreesAndsowecanrepresentthemasnestedstructures,likenestedbulletedlists:

251webpagemachine(I)q  251-web-page-in-HTMLprogramq  HTMLinterpretermachine(I)

²  HTML-interpreter-in-x86program(T)o  HTML-interpreter-in-Cprogramo  C-to-x86compilermachine(I)

•  C-to-x86compiler-in-x86program•  X86computer

²  x86computer

q  251-web-page-in-HTMLprogramo  HTML-interpreter-in-Cprogram

•  C-to-x86compiler-in-x86program•  X86computer

o  C-to-x86compilermachine(I)²  HTML-interpreter-in-x86program(T)²  x86computer

q  HTMLinterpretermachine(I)251webpagemachine(I)

Versionthatshowsconclusionsbelowbullets.MoresimilartoderivaLonswithhorizontallines,buthardertocreateandread

Preferred“top-down”versionthatshowsconclusionsabovebullets.

Metaprogramming 17

Metaprogramming:BootstrappingPuzzles

HowcanwewriteaJava-to-x86compilerinJava?

HowcanwewriteSchemeinterpreterinScheme?

Metaprogramming 18

MetacircularityandBootstrappingManyexamples:•  LispinLisp/SchemeinScheme/RacketinRacket•  PythoninPython:PyPy•  JavainJava:JikesRVM,MaxineVM•  …•  C-to-x86compilerinC•  evalconstructinlanguageslikeLisp,JavaScript

Howcanthisbepossible?

Keyinsightstobootstrapping:•  ThefirstimplementaLonofalanguagecannotbein

itself,butmustbeinsomeotherlanguage.•  OnceyouhaveoneimplementaLonofalanguage,you

canimplementitinitself.Metaprogramming 19

Supposeyouaregiven:•  Scheme-interpreter-in-Pythonprogram•  Pythonmachine•  Scheme-interpreter-in-SchemeprogramHowdoyoucreateaSchemeinterpretermachineusingtheScheme-interpreter-in-Schemeprogram?

MetacircularityExample1:Problem

Metaprogramming 20

Supposeyouaregiven:•  Scheme-interpreter-in-Pythonprogram•  Pythonmachine•  Scheme-interpreter-in-SchemeprogramHowdoyoucreateaSchemeinterpretermachineusingtheScheme-interpreter-in-Schemeprogram?

Schemeinterpretermachine#2(I)q Scheme-interpreter-in-Schemeprogramq Scheme-interpretermachine#1(I)

² Scheme-interpreter-in-Pythonprogram² Pythonmachine

ButwhycreateSchemeinterpretermachine#2whenyoualreadyhaveScheme-interpretermachine#1?

MetacircularityExample1:SoluBon

Metaprogramming 21

Supposeyouaregiven:•  Scheme-subset-interpreter-in-Pythonprogram(implements

onlycoreSchemefeatures;nodesugaringorotherfrills)•  Pythonmachine•  Full-Scheme-interpreter-in-SchemeprogramHowdoyoucreateaFull-SchemeinterpretermachineusingtheFull-Scheme-interpreter-in-Schemeprogram?

Full-Schemeinterpretermachine(I)q Scheme-interpreter-in-Schemeprogramq Scheme-subsetinterpretermachine#1(I)

² Scheme-subset-interpreter-in-Pythonprogram² Pythonmachine

MetacircularityExample1:MoreRealisBc

Metaprogramming 22

Supposeyouaregiven:•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)•  C-to-x86-translator-in-C-subsetprogram

HowdoyoucompiletheC-to-x86-translator-in-C?

MetacircularityExample2:Problem

Metaprogramming 23

Supposeyouaregiven:•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)•  C-to-x86-translator-in-Cprogram

HowdoyoucompiletheC-to-x86-translator-in-C?

MetacircularityExample2:SoluBon

C-to-x86-translatormachine#2(I)q C-to-x86-translator-in-x86program#2(T)

² C-to-x86-translator-in-C² C-to-x86-translatormachine#1(I)

o  C-to-x86-translator-in-x86program#1o  x86computer

q x86computer

ButwhycreateC-to-x86-translator-in-x86program#2(T)whenyoualreadyhaveC-to-x86-translator-in-x86program#1?

Metaprogramming 24

Supposeyouaregiven:•  C-subset-to-x86-translator-in-x86program

(acompilerforasubsetofCwriDeninx86)•  x86interpretermachine(anx86computer)•  Full-C-to-x86-translator-in-C-subsetprogram

(acompilerforthefullClanguagewriDeninasubsetofC)

HowdoyoucreateaFull-C-to-x86-translatormachine?

MetacircularityExample2:MoreRealisBc

Full-C-to-x86-translatormachine(I)q Full-C-to-x86-translator-in-x86program(T)

² Full-C-to-x86-translator-in-C-subset² C-subset-to-x86-translatormachine(I)

o  C-subset-to-x86-translator-in-x86programo  x86computer

q x86computerMetaprogramming 25

AlonglineofCcompilersC-version_n-to-target_n-translatormachine(I)q  C-version_n-to-target_n-translatorprogramintarget_n-1(T)

²  C-version_n-to-target_n-translatorprograminC-version_n-1²  C-version_n-1-to-target_n-1translatormachine(I)

o  C-version_n-1-to-target_n_1-translatorprogramintarget_n-2(T)

Ø  C-version_2-to-target_2-translator-programintarget_1(T)

§  C-version_2-to-target_2-translatorprograminC-version_1§  C-version_1-to-target_1translatormachine(I)

•  C-version_1-to-target_1-translatorprograminassembly_0•  assembly_0computer

Ø  target_1computer

o  target_n-2computerq  target_n-1computer

…

…

o  TheversionsofCandtargetlanguagescanchangeateachstage.o  Trojanhorsesfromearliersourcefilescanremainintranslatormachinesevenif

they’renotinlatersourcefile!SeeKenThompson’sReflec4ononTrus4ngTrustMetaprogramming 26

MoreMetaprogramminginSML

•  We’vealreadyseenPostFixandIntexSML•  AsequencesofexpressionlanguagesimplementedinSMLthatlookcloserandclosertoRacket:•  Bindex:addnaming•  Valex:addmorevaluetypes,dynamictypechecking,desugaring

•  HOFL:firstclassfuncLonvalues,closurediagrams

Metaprogramming 27

Remember:language!=implementaBon•  Easytoconfuse"thewaythislanguageisusually

implemented"or"theimplementaLonIuse"with"thelanguageitself.”

•  JavaandRacketcanbecompiledtox86

•  CcanbeinterpretedinRacket

•  x86canbecompiledtoJavaScript

•  CanwecompileC/C++toJavascript?hDp://kripken.github.io/emscripten-site/

Metaprogramming 28