high-performance problem solving compiler architecture...
TRANSCRIPT
![Page 1: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/1.jpg)
Compiler Architecture forHigh-Performance Problem Solving
A Quest for High-Level ProgrammingSystems
Ken KennedyCenter for High Performance Software
Rice University
http://www.cs.rice.edu/~ken/Presentations/CompilerArchitecture.pdf
Center for High Performance Software
![Page 2: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/2.jpg)
Center for High Performance Software
Context� Explosive Growth of Information Technology
�Now represents 20 percent of economy, 35 percent of GDP growth�Essential to operation of most organizations, especially government
![Page 3: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/3.jpg)
Center for High Performance Software
Context� Explosive Growth of Information Technology
�Now represents 20 percent of economy, 35 percent of GDP growth�Essential to operation of most organizations, especially government
� Enormous Demand for Software
� Shortage of IT Professionals�Challenge: double the number of CS graduates
![Page 4: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/4.jpg)
Center for High Performance Software
Context� Explosive Growth of Information Technology
�Now represents 20 percent of economy, 35 percent of GDP growth�Essential to operation of most organizations, especially government
� Enormous Demand for Software
� Shortage of IT Professionals�Challenge: double the number of CS graduates
� Complex Computer Architectures�Deep memory hierarchies, high degrees of parallelism�Heterogeneous, geographically distributed platforms
� Changes in performance of nodes and links during execution
![Page 5: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/5.jpg)
Center for High Performance Software
Context� Explosive Growth of Information Technology
�Now represents 20 percent of economy, 35 percent of GDP growth�Essential to operation of most organizations, especially government
� Enormous Demand for Software
� Shortage of IT Professionals�Challenge: double the number of CS graduates
� Complex Computer Architectures�Deep memory hierarchies, high degrees of parallelism�Heterogeneous, geographically distributed platforms
� Changes in performance of nodes and links during execution
� Complex Applications�Many diverse components, dynamic, adaptive, unstructured
![Page 6: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/6.jpg)
Center for High Performance Software
Philosophy� Compiler Technology = Off-Line Processing
�Goals: improved performance and language usability� Making it practical to use the full power of the language
![Page 7: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/7.jpg)
Center for High Performance Software
Philosophy� Compiler Technology = Off-Line Processing
�Goals: improved performance and language usability� Making it practical to use the full power of the language
�Trade-off: preprocessing time versus execution time�Rule: performance of both compiler and application must be
acceptable to the end user
![Page 8: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/8.jpg)
Center for High Performance Software
Philosophy� Compiler Technology = Off-Line Processing
�Goals: improved performance and language usability� Making it practical to use the full power of the language
�Trade-off: preprocessing time versus execution time�Rule: performance of both compiler and application must be
acceptable to the end user
� Examples�Macro expansion
� PL/I macro facility � 10x improvement with compilation
![Page 9: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/9.jpg)
Center for High Performance Software
Philosophy� Compiler Technology = Off-Line Processing
�Goals: improved performance and language usability� Making it practical to use the full power of the language
�Trade-off: preprocessing time versus execution time�Rule: performance of both compiler and application must be
acceptable to the end user
� Examples�Macro expansion
� PL/I macro facility � 10x improvement with compilation�Database query optimization
![Page 10: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/10.jpg)
Center for High Performance Software
Philosophy� Compiler Technology = Off-Line Processing
�Goals: improved performance and language usability� Making it practical to use the full power of the language
�Trade-off: preprocessing time versus execution time�Rule: performance of both compiler and application must be
acceptable to the end user
� Examples�Macro expansion
� PL/I macro facility � 10x improvement with compilation�Database query optimization�Emulation acceleration
� TransMeta �code morphing�
![Page 11: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/11.jpg)
Center for High Performance Software
Philosophy� Compiler Technology = Off-Line Processing
�Goals: improved performance and language usability� Making it practical to use the full power of the language
�Trade-off: preprocessing time versus execution time�Rule: performance of both compiler and application must be
acceptable to the end user
� Examples�Macro expansion
� PL/I macro facility � 10x improvement with compilation�Database query optimization�Emulation acceleration
� TransMeta �code morphing��Communication planning in dynamic applications (Inspector/Executor)
![Page 12: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/12.jpg)
Center for High Performance Software
Making Languages Usable
It was our belief that if FORTRAN, during itsfirst months, were to translate any reasonable�scientific� source program into an object programonly half as fast as its hand-coded counterpart,then acceptance of our system would be in seriousdanger... I believe that had we failed to produceefficient programs, the widespread use oflanguages like FORTRAN would have been seriouslydelayed.
� John Backus
![Page 13: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/13.jpg)
Center for High Performance Software
A Java Experiment� Scientific Programming In Java
�Goal: make it possible to use the full object-oriented power forscientific applications� Many scientific implementations mimic Fortran style
![Page 14: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/14.jpg)
Center for High Performance Software
A Java Experiment� Scientific Programming In Java
�Goal: make it possible to use the full object-oriented power forscientific applications� Many scientific implementations mimic Fortran style
� OwlPack Benchmark Suite�Three versions of LinPACK in Java
� Fortran style� Lite object-oriented style� Full polymorphism
No differences for type
![Page 15: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/15.jpg)
Center for High Performance Software
A Java Experiment� Scientific Programming In Java
�Goal: make it possible to use the full object-oriented power forscientific applications� Many scientific implementations mimic Fortran style
� OwlPack Benchmark Suite�Three versions of LinPACK in Java
� Fortran style� Lite object-oriented style� Full polymorphism
No differences for type
� Experiment�Compare running times for different styles on same Java VM�Evaluate potential for compiler optimization
![Page 16: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/16.jpg)
Center for High Performance Software
Performance Results
0
5
10
15
20
25
30
35
Run Timein
Secs
dgefa dgesl dgedi
Fortran StyleLite OO StyleOO StyleOptimized OONative F90
Results Using JDK 1.2JIT on SUN Ultra 5
![Page 17: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/17.jpg)
Center for High Performance Software
Preliminary Conclusions� Definition of Application Will Become Fuzzy
�Knowledge of the computation will be revealed in stages�Examples:
� Compilation with input data,� Compiler-generated run-time preprocessing� Optimization with late binding of target platform� Compilation based on predefined component libraries
![Page 18: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/18.jpg)
Center for High Performance Software
Preliminary Conclusions� Definition of Application Will Become Fuzzy
�Knowledge of the computation will be revealed in stages�Examples:
� Compilation with input data,� Compiler-generated run-time preprocessing� Optimization with late binding of target platform� Compilation based on predefined component libraries
� Performance Will Be More Elusive�Even reliable performance will be hard to achieve�Compiler will need to be even more heroic,
� Yet programmer will continue to want control
![Page 19: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/19.jpg)
Center for High Performance Software
Preliminary Conclusions� Definition of Application Will Become Fuzzy
�Knowledge of the computation will be revealed in stages�Examples:
� Compilation with input data,� Compiler-generated run-time preprocessing� Optimization with late binding of target platform� Compilation based on predefined component libraries
� Performance Will Be More Elusive�Even reliable performance will be hard to achieve�Compiler will need to be even more heroic,
� Yet programmer will continue to want control
� Compilers Structure Will Be More Flexible�Compilation will be carried out in stages
![Page 20: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/20.jpg)
Center for High Performance Software
Compiling with Data
CompilerCompiler
ApplicationApplication
ProgramProgram
![Page 21: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/21.jpg)
Center for High Performance Software
Compiling with Data
CompilerCompiler
Reduced ApplicationReduced Application
ProgramProgram
Slowly-Changing Data
Slowly-Changing Data
![Page 22: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/22.jpg)
Center for High Performance Software
Compiling with Data
CompilerCompiler
Reduced ApplicationReduced Application
ProgramProgram
Rapidly-Changing Data
Rapidly-Changing Data
Slowly-Changing Data
Slowly-Changing Data
AnswersAnswers
![Page 23: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/23.jpg)
Center for High Performance Software
Run-Time Compilation
CompilerCompiler
ApplicationApplication
ProgramProgram
![Page 24: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/24.jpg)
Center for High Performance Software
Run-Time Compilation
CompilerCompiler
ApplicationApplication
ProgramProgram
Slowly-Changing Data
Slowly-Changing Data Pre-OptimizerPre-Optimizer
![Page 25: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/25.jpg)
Center for High Performance Software
Run-Time Compilation
CompilerCompiler
ApplicationApplication
ProgramProgram
Rapidly-Changing Data
Rapidly-Changing Data
Slowly-Changing Data
Slowly-Changing Data
AnswersAnswers
Pre-OptimizerPre-Optimizer
![Page 26: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/26.jpg)
Center for High Performance Software
Dynamic Optimization
CompilerCompiler
ApplicationApplication
ProgramProgram
![Page 27: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/27.jpg)
Center for High Performance Software
Dynamic Optimization
CompilerCompiler
ApplicationApplication
ProgramProgram
ConfigurationAnd Data
ConfigurationAnd Data
Dynamic Optimizer(Optimizing Loader)
Dynamic Optimizer(Optimizing Loader)
![Page 28: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/28.jpg)
Center for High Performance Software
Dynamic Optimization
CompilerCompiler
ApplicationApplication
ProgramProgram
Rapidly-Changing Data
Rapidly-Changing Data
ConfigurationAnd Data
ConfigurationAnd Data
AnswersAnswers
Dynamic Optimizer(Optimizing Loader)
Dynamic Optimizer(Optimizing Loader)
![Page 29: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/29.jpg)
Center for High Performance Software
Grid Compilation Architecture� Goal: reliable performance under varying load
Whole-ProgramCompiler
Libraries
DynamicOptimizer
Real-timePerformance
Monitor
PerformanceProblem
ServiceNegotiator
Scheduler
GridRuntimeSystem
SourceAppli-cation
Config-urableObject
Program
SoftwareComponents
Performance Feedback
Negotiation
GrADS Project: Berman, Chien, Cooper, Dongarra, Foster, Gannon, Johnsson, Kennedy, Kesselman, Reed, Torczon, Wolski
![Page 30: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/30.jpg)
Center for High Performance Software
Grid Compilation Architecture� Goal: reliable performance under varying load
Whole-ProgramCompiler
Libraries
DynamicOptimizer
Real-timePerformance
Monitor
PerformanceProblem
ServiceNegotiator
Scheduler
GridRuntimeSystem
SourceAppli-cation
Config-urableObject
Program
SoftwareComponents
Performance Feedback
Negotiation
GrADS Project: Berman, Chien, Cooper, Dongarra, Foster, Gannon, Johnsson, Kennedy, Kesselman, Reed, Torczon, Wolski
Execution Environment
![Page 31: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/31.jpg)
Center for High Performance Software
Programming Productivity� Challenges
�programming is hard�professional programmers are in short supply�high performance will continue to be important
![Page 32: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/32.jpg)
Center for High Performance Software
Programming Productivity� Challenges
�programming is hard�professional programmers are in short supply�high performance will continue to be important
� One Strategy: Make the End User a Programmer�professional programmers develop components�users integrate components using:
� problem-solving environments (PSEs)� scripting languages (possibly graphical)
examples: Visual Basic, Tcl/Tk, AVS, Khoros
![Page 33: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/33.jpg)
Center for High Performance Software
Programming Productivity� Challenges
�programming is hard�professional programmers are in short supply�high performance will continue to be important
� One Strategy: Make the End User a Programmer�professional programmers develop components�users integrate components using:
� problem-solving environments (PSEs)� scripting languages (possibly graphical)
examples: Visual Basic, Tcl/Tk, AVS, Khoros
� Compilation for High Performance�translate scripts and components to common intermediate language�optimize the resulting program using interprocedural methods
![Page 34: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/34.jpg)
Center for High Performance Software
Script-Based Programming
Component Library
Component Library
User LibraryUser Library
ScriptScript
![Page 35: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/35.jpg)
Center for High Performance Software
Script-Based Programming
Component Library
Component Library
User LibraryUser Library
ScriptScript
IntermediateCode
IntermediateCodeTranslatorTranslator
![Page 36: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/36.jpg)
Center for High Performance Software
Script-Based Programming
Component Library
Component Library
User LibraryUser Library
ScriptScript
IntermediateCode
IntermediateCode
GlobalOptimizerGlobal
Optimizer
TranslatorTranslator
![Page 37: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/37.jpg)
Center for High Performance Software
Code GeneratorCode
Generator
Script-Based Programming
Component Library
Component Library
User LibraryUser Library
ScriptScript
IntermediateCode
IntermediateCode
GlobalOptimizerGlobal
Optimizer
TranslatorTranslator
![Page 38: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/38.jpg)
Center for High Performance Software
Code GeneratorCode
Generator
Script-Based Programming
Component Library
Component Library
User LibraryUser Library
ScriptScript
IntermediateCode
IntermediateCode
GlobalOptimizerGlobal
Optimizer
TranslatorTranslator
Problem: long compilation times,even for short scripts!
![Page 39: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/39.jpg)
Center for High Performance Software
Telescoping Languages
L1 ClassLibrary
L1 ClassLibrary
![Page 40: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/40.jpg)
Center for High Performance Software
Telescoping Languages
L1 ClassLibrary
L1 ClassLibrary
CompilerGeneratorCompilerGenerator
L1 CompilerL1 Compiler
Could run for hours
![Page 41: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/41.jpg)
Center for High Performance Software
Telescoping Languages
L1 ClassLibrary
L1 ClassLibrary
ScriptScript
CompilerGeneratorCompilerGenerator
L1 CompilerL1 CompilerScriptTranslatorScript
Translator
OptimizedApplicationOptimizedApplication
VendorCompilerVendorCompiler
Could run for hours
understandslibrary callsas primitives
![Page 42: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/42.jpg)
Center for High Performance Software
Telescoping Languages: Advantages� Compile times can be reasonable
�More compilation time can be spent on libraries� Amortized over many uses
�Script compilations can be fast� Components reused from scripts may be included in libraries
![Page 43: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/43.jpg)
Center for High Performance Software
Telescoping Languages: Advantages� Compile times can be reasonable
�More compilation time can be spent on libraries� Amortized over many uses
�Script compilations can be fast� Components reused from scripts may be included in libraries
� High-level optimizations can be included�Based on specifications of the library designer
� Properties often cannot be determined by compilers� Properties may be hidden after low-level code generation
![Page 44: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/44.jpg)
Center for High Performance Software
Telescoping Languages: Advantages� Compile times can be reasonable
�More compilation time can be spent on libraries� Amortized over many uses
�Script compilations can be fast� Components reused from scripts may be included in libraries
� High-level optimizations can be included�Based on specifications of the library designer
� Properties often cannot be determined by compilers� Properties may be hidden after low-level code generation
� User retains substantive control over language performance�Mature code can be built into a library and incorporated into
language
![Page 45: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/45.jpg)
Center for High Performance Software
Applications� Matlab Compiler
�Automatically generated from LAPACK or ScaLAPACK� With help via annotations from the designer
� Flexible Data Distributions�Failing of HPF: inflexible distributions�Data distribution == collection of interfaces that meet specs�Compiler applies standard transformations
� Automatic Generation of POOMA�Data structure library implemented via template expansion in C++�Long compile times, missed optimizations
� Generator for Grid Computations�GrADS: automatic generation of NetSolve
![Page 46: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/46.jpg)
Center for High Performance Software
Requirements of Script Compilation
� Scripts must generate efficient programs�Comparable to those generated from standard interprocedural
methods�Avoid need to recode in standard language
![Page 47: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/47.jpg)
Center for High Performance Software
Requirements of Script Compilation
� Scripts must generate efficient programs�Comparable to those generated from standard interprocedural
methods�Avoid need to recode in standard language
� Script compile times should be proportional to length of script�Not a function of the complexity of the library�Principle of �least astonishment�
![Page 48: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/48.jpg)
Center for High Performance Software
Telescoping Languages
ScriptScript L1 CompilerL1 CompilerScriptTranslatorScript
Translator
OptimizedApplicationOptimizedApplication
VendorCompilerVendorCompiler
understandslibrary callsas primitives
![Page 49: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/49.jpg)
Center for High Performance Software
Script Compilation Algorithm
� Propagate variable property information throughout the program�Use jump functions to propagate through calls to library
![Page 50: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/50.jpg)
Center for High Performance Software
Script Compilation Algorithm
� Propagate variable property information throughout the program�Use jump functions to propagate through calls to library
� Apply high-level transformations�Driven by information about properties�Ensure that process applies to expanded code
![Page 51: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/51.jpg)
Center for High Performance Software
Script Compilation Algorithm
� Propagate variable property information throughout the program�Use jump functions to propagate through calls to library
� Apply high-level transformations�Driven by information about properties�Ensure that process applies to expanded code
� Perform low-level code specialization�At each call site, determine the best estimate to parameter
properties that is reflected by a specialized fragment in the codedatabase� Use a method similar to �unification�
�Substitute fragment from database for call� This could contain a call to a lower-level library routine.
![Page 52: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/52.jpg)
Center for High Performance Software
Telescoping Languages
L1 ClassLibrary
L1 ClassLibrary
CompilerGeneratorCompilerGenerator
L1 CompilerL1 Compiler
Could run for hours
![Page 53: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/53.jpg)
Center for High Performance Software
Library Analysis and Preparation
� Discovery of and Propagation Critical Properties
� Analysis of Transformation Specifications
� Code specialization for different sets of parameter properties
![Page 54: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/54.jpg)
Center for High Performance Software
Library Analysis and Preparation
� Discovery and Propagation of Critical Properties�Which properties of parameters affect optimization
� Examples: value, type, rank and size of matrix
![Page 55: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/55.jpg)
Center for High Performance Software
Discovery of Critical Properties
� From specifications by the library designer�If the matrix is triangular, then�
� From examining the code itself�Look at a promising optimization point�Determine conditions under which we can make significant
optimizations�See if any of these conditions can be mapped back to parameter
properties
� From sample calling programs provided by the designer call average(shift(A,-1), shift(A,+1))
� Can save on memory accesses
![Page 56: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/56.jpg)
Center for High Performance Software
Examining the Code
� Example from LAPACK
subroutine VMP(C, A, B, m, n, s) integer m,n,s; real A(n), B(n), C(m) i = 1 do j = 1, n C(i) = C(i) + A(j)*B(j) i = i + s enddoend VMP
Vectorizable if s != 0
![Page 57: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/57.jpg)
Center for High Performance Software
Library Analysis and Preparation
� Discovery and Propagation of Critical Properties�Which properties of parameters affect optimization
� Examples: value, type, rank and size of matrix�Construction of jump functions for the library calls
� With respect to critical properties
![Page 58: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/58.jpg)
Center for High Performance Software
Propagation of Properties
� Jump Functions (Transfer Functions)�Tell the effect on properties of a call to a library routine
� Whether it preserves the property or changes it�Computed during library preparation (compiler generation) phase
� Can use lots of time�Can be designed to be fast
� Tradeoff between accuracy and performance
� Advantage of Jump Functions�Avoid necessity for analysis of library source
� Prevent blowup of compilation times
![Page 59: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/59.jpg)
Center for High Performance Software
Library Analysis and Preparation
� Discovery and Propagation of Critical Properties�Which properties of parameters affect optimization
� Examples: value, type, rank and size of matrix�Construction of jump functions for the library calls
� With respect to critical properties
� Analysis of Transformation Specifications�Construction of a specification-driven translator for use in compiling
scripts
![Page 60: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/60.jpg)
Center for High Performance Software
High-level Identities
� Often library developer knows high-level identities�Difficult for the compiler to discern�Optimization should be performed on sequences of calls rather than
code remaining after expansion
� Example: Push and Pop�Designer Push(x) followed by y = Pop() becomes y = x
� Ignore possibility of overflow in Push
� Example: Trigonometric Functions�Sin and Cos used in same loop�both computed using expensive calls
to the trig library�Recognize that cos(x) and sin(x) can be computed by a single call to
sincos(x,s,c) in a little more than the time required for sin(x).
![Page 61: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/61.jpg)
Center for High Performance Software
Contextual Expansions� Out of Core Arrays
�Operatons Get(I,J) and GetRow(I,Lo,N)
� Get in a loop Do I Do J � Get(I,J) Enddo Enddo
� When can we vectorize?�Turn into GetRow�Answer: if Get is not involved in a recurrence.
� How can we know?
![Page 62: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/62.jpg)
Center for High Performance Software
Library Analysis and Preparation
� Discovery and Propagation of Critical Properties�Which properties of parameters affect optimization
� Examples: value, type, rank and size of matrix�Construction of jump functions for the library calls
� With respect to critical properties
� Analysis of Transformation Specifications�Construction of a specification-driven translator for use in compiling
scripts
� Code specialization for different sets of parameter properties�For each set, assume and optimize to produce specialized code
![Page 63: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/63.jpg)
Center for High Performance Software
Code Selection Example� Library compiler develops inlining tables
subroutine VMP(C, A, B, m, n, s) integer m,n,s; real A(n), B(n), C(m) i = 1 do j = 1, n C(i) = C(i) + A(j)*B(j) i = i + s enddoend VMP
case on s: ==0: C(1) = C(1) + sum(A(1:n)*B(1:n)) !=0: C(1:n:s) = C(1:n:s) + A(1:n)*B(1:n)default: call VMP(C,A,B,m,n,s)
Inlining Table
vector
![Page 64: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/64.jpg)
Center for High Performance Software
Flexible Compiler Architecture� Flexible Definition of Computation
�Parameters� program scheme� base library sequence (l1, l2, �, lp)� subprogram source files (s1, s2, ..., sn)� run history (r1, r2, ..., rk)� data sets (d1, d2, ..., dm)� target configuration
![Page 65: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/65.jpg)
Center for High Performance Software
Flexible Compiler Architecture� Flexible Definition of Computation
�Parameters� program scheme� base library sequence (l1, l2, �, lp)� subprogram source files (s1, s2, ..., sn)� run history (r1, r2, ..., rk)� data sets (d1, d2, ..., dm)� target configuration
� Compilation = Partial Evaluation�several compilation steps as information becomes available
![Page 66: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/66.jpg)
Center for High Performance Software
Flexible Compiler Architecture� Flexible Definition of Computation
�Parameters� program scheme� base library sequence (l1, l2, �, lp)� subprogram source files (s1, s2, ..., sn)� run history (r1, r2, ..., rk)� data sets (d1, d2, ..., dm)� target configuration
� Compilation = Partial Evaluation�several compilation steps as information becomes available
� Program Management�When to back out of previous compilation decisions due to change�When to invalidate certain inputs
� Examples: change in library or run history
![Page 67: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/67.jpg)
Center for High Performance Software
Summary� Optimization Enables Language Power
�Principle: encourage rather than discourage use of powerfulfeatures� Good programming practice should be rewarded
http://www.cs.rice.edu/~ken/Presentations/CompilerArchitecture.pdf
![Page 68: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/68.jpg)
Center for High Performance Software
Summary� Optimization Enables Language Power
�Principle: encourage rather than discourage use of powerfulfeatures� Good programming practice should be rewarded
� Target Platforms, Languages, and Apps Becoming More Complex�Platforms: Parallel, heterogeneous, deep memory hierarchies�Applications: dynamic, irregular, extensive use of domain libraries
http://www.cs.rice.edu/~ken/Presentations/CompilerArchitecture.pdf
![Page 69: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/69.jpg)
Center for High Performance Software
Summary� Optimization Enables Language Power
�Principle: encourage rather than discourage use of powerfulfeatures� Good programming practice should be rewarded
� Target Platforms, Languages, and Apps Becoming More Complex�Platforms: Parallel, heterogeneous, deep memory hierarchies�Applications: dynamic, irregular, extensive use of domain libraries
� Programming support is hard�Ideally, should support high-level domain-specific programming
� Telescoping languages
http://www.cs.rice.edu/~ken/Presentations/CompilerArchitecture.pdf
![Page 70: High-Performance Problem Solving Compiler Architecture forken/Presentations/CompilerArchitecture.pdf · Compiler Architecture for High-Performance Problem Solving A Quest for High-Level](https://reader036.vdocument.in/reader036/viewer/2022070805/5f03ea477e708231d40b65d9/html5/thumbnails/70.jpg)
Center for High Performance Software
Summary� Optimization Enables Language Power
�Principle: encourage rather than discourage use of powerfulfeatures� Good programming practice should be rewarded
� Target Platforms, Languages, and Apps Becoming More Complex�Platforms: Parallel, heterogeneous, deep memory hierarchies�Applications: dynamic, irregular, extensive use of domain libraries
� Programming support is hard�Ideally, should support high-level domain-specific programming
� Telescoping languages
� Compiler Structure Will Be Correspondingly Complex�Partial evaluation in stages with incremental information
http://www.cs.rice.edu/~ken/Presentations/CompilerArchitecture.pdf