cling – the new c++ interpreter for root 6 cern, ph-sft & fermilab v. vassilev, p. canal, a....

Cling – The New C++ Cling – The New C++ Interpreter for ROOT 6Interpreter for ROOT 6

CERN, PH-SFT & FermiLabCERN, PH-SFT & FermiLab

V. Vassilev, P. Canal, A. Naumann, P. RussoV. Vassilev, P. Canal, A. Naumann, P. Russo

vvassilev / CHEP 2012.05.21 2

Role of C++ Interpreter in ROOT

Load/Store C++ objects Runtime Dynamism

TFile::Open(“http://...”)

gDirectory->Get(“hist”)

python runReco.py

Fast Prototyping

ROOT

CINT

Reflection

TPluginManager

IO

...

PyR

OO

T

Type Info

TClass


Role of C++ Interpreter in ROOT

ROOT uses the interpreter much more than one would expect:


Cling Is Better Than CINT

Full C++ support STL + templates Path to C++11

Correctness Better type information and representations Always compile in memory Much less code to maintain


Cling's Dual Personality

An interpreter – looks like an interpreter and behaves like an interpreterCling follows the read-evaluate-print-loop (repl) concept.

More than interpreter – built on top of compiler libraries (Clang and LLVM)Contains interpreter parts and compiler parts. More of an interactive compiler or an interactive compiler interface for clang.

No need to compile Cling/ROOT with Clang/LLVM


Cling Uses Clang & LLVM

LLVM and Clang “The LLVM Project is a collection of modular and reusable compiler and

toolchain technologies...”


Cling's Codebase

* No testsuites included.Credits: generated using David A. Wheeler's 'SLOCCount'

Other ROOT – 1400K SLOC*

CINT+Reflex – 230K SLOC*

Cling – 7K SLOC*

ROOT

CINT+Reflex


Other ROOT – 1400K SLOC*

CINT+Reflex – 230K SLOC*

Cling – 7K SLOC*

externals:

LLVM + Clang – 800K SLOC*

Cling's Codebase

* No testsuites included.Credits: generated using David A. Wheeler's 'SLOCCount'

ROOT

Cling


Challenges

Incompatible concepts like compilation and interpretationMany tasks that are trivial for an interpreter become a nightmare for a compiler.

Make C++ usable at the promptIncorporate the experience we have with CINT. First step: adopt the successful usability extensions from CINT.


Cling in a NutshellCling in a Nutshell


Extending C++ Language

[cling]$ sin(12);void wrapper() { sin(12);}

[cling]$ [cling]$

int i = 12;sin(i);

void wrapper1() { int i = 12;}

void wrapper2() { sin(i);}

?We want to be able to

run statements


[cling]$ [cling]$

int i = 12; printf("%d\n",i);printf("%f\n",sin(i));

Extending C++ Language

Wrap the input Look for declarations Extract the declarations one level up, as global

declarations

void wrapper1() { int i = 12; printf("%d\n",i);}

void wrapper2() { printf("%f\n", sin(i));}

int i = 12;


Streaming Execution Results

[cling]$ [cling]$

int i = 12(int) 12sin(i)(double const) -5.365729e-01

No semicolon (;)No semicolon (;)


Error Recovery

Filled input-by-input Incorrect inputs must be discarded as a whole


Work in progress: Code Unloading

[cling]$ [cling]$ [cling]$ [cling]$ [cling]$ [cling]$ [cling]$ [cling]$ [cling]$

.L Calculator.hCalculator calc;calc.Add(3, 1)(int) 2 //WTF!?*.U Calculator.h.L Calculator.hCalculator calc;calc.Add(3, 1)(int) 4 //

// Calculator.hclass Calculator { int Add(int a, int b) { return a + b; }...};

// Calculator.hclass Calculator { int Add(int a, int b) { return a - b; }...};

* What's That Function


Late Binding

{ TFile* F = 0; if (is_day_of_month_even()) F = TFile::Open("even.root"); else F = TFile::Open("odd.root"); hist->Draw();}hist->Draw();

Defined in the root file

The root file is gone. Issue an error.

Opens a dynamic scope. It tells the compiler that cling will take over the resolution of possible unknown identifiers


Late Binding

{ TFile* F = 0; if (is_day_of_month_even()) F = TFile::Open("even.root"); else F = TFile::Open("odd.root"); gCling->EvaluateT<void>("hist->Draw()", ...);}

hist->Draw(); Tell the compiler the identifier will be resolved at runtime

Wrap it into valid C++ code

Partially recompile at runtime

Automatically transformed into valid

C++ code


Challenges

Error recoveryEven though the user has typed wrong input at the prompt cling must survive, i.e issue an error and continue to work.

Initialization of global variablesCling depends on global variables, which need to be initialized. However, the global variables continue to be added (potentially) with every input line.

Late bindingCling needs to provide a way for symbols unavailable at compile-time a second chance to be provided at runtime.

Value printerThe interactive mode obeys the repl concept and there should be way of easy print value and type of expression in a user-extensible way.

Running statementsCINT-specific C++ extension improving the user interaction with the interpreter from the terminal.


Cling In The World

Announced in July 2011 as working C++ interpreter

Cling and OpenGLhttp://www.youtube.com/watch?v=eoIuqLNvzFs

Cling and QThttp://www.youtube.com/watch?v=BrjV1ZgYbbA

MATLAB to C++ translator Regular bug reports from outside HEP


Cling In ROOTCling In ROOT


Dictionaries

Describe compiled codeIncarnation of the type information and reflection in ROOT. The only way of crossing the border between interpretation and compilation.

CINT and Reflex dictionaries: Double the size of the libraries Multiple copies of the dictionary data in the memory Incompatible reflection formats Do not cover 100% of C++ CINT

Reflection

TPluginManager

IO

...

PyR

OO

T

Type Info

TClass


CINT Dictionary Use

Dictionary.(h|cxx)

ClassDef

Call Stubs

Reflection Info

TClass

Source of ROOT's class description

Member functions injected by the

ClassDef macro

Target-independent, normalized signatures used to call compiled

functions.Teaches CINT what a

class contains

Dictionary Size (SLOC)

6%

10%

61%

23%


LLVM Just-In-Time Compiler

Cling

Clang

LLVM Bit Code

LLVM JIT

Machine Code (x86, Alpha, ...)

Compiles lazily just before the function is called

Has full knowledge about the target architecture

Much faster than interpreting the LLVM Bit Code

Compiled libs(.so)


Cling + ROOT = ROOT 6

ROOT 6 in NovemberSee plenary by Fons on Wednesday

Windows support to arrive after 6.00Work in progress in clang

“genreflex”-compatible dictionary generator after 6.00


Reduce size of dictionaries Mid term: Call Stubs & Reflection Info goes away! Long term: No dictionaries at all

Compiled TFormula

ROOT 6.x

Dictionary.(h|cxx)

ClassDef

Call Stubs

Reflection Info

TClass


Additional Features

World class performance and optimizations OpenCL C Objective C[++] ...


In ActionIn Action

Come and see at our booth!Come and see at our booth!

C N GC N G


Thank you!Thank you!

Fons's plenary on Wednesday Google Tech Talk LLVM Euro Dev Meeting 2011

http://www.llvm.org/devmtg/2011-09-16/EuroLLVM2011-ImplementingDynamicScopesInCling.pdf

LLVM Dev Meeting 2011http://www.llvm.org/devmtg/2010-11/Naumann-Cling.pdf

http://www.llvm.org/devmtg/2010-11/videos/Naumann_Cling-desktop.mp4


Backup slidesBackup slides


Pre-Compiled Headers/Modules

Carefully crafted data structures designed to improve translator's performance:

Reduce lexical, syntax and semantic analysis

Loaded “lazily” on demand

http

://c

lang

.llvm

.org

/do

cs/P

CH

Inte

rnal

s.ht

ml


Pre-Compiled Headers/Modules

Design advantages: Loading is significantly faster than re-parsing Minimize the cost of reading Read times don't depend on size Cost of generating isn't large


PCH vs PCM

PCH PCM

And thus PCM much more flexible


Expressive Diagnostics

Column numbers and caret diagnosticsCaretDiagnostics.C:4:13: warning: '.*' specified field precision is missing a matching 'int' argumentprintf("%.*d"); ~~^~

Range highlightingRangeHighlight.C:14:39: error: invalid operands to binary expression ('int' and 'A')return y + func(y ? ((SomeA.X + 40) + SomeA) / 42 + SomeA.X : SomeA.X); ~~~~~~~~~~~~ ^ ~~~~~~~

Fix-it hintsFixItHints.C:7:27: warning: use of GNU old-style field designator extensionstruct point origin = { x: 0.0, y: 0.0 }; ^~ .x =


LLVM Compiler Toolchain

LLVM Core “The LLVM Project is a collection of modular and reusable compiler and toolchain technologies...”

More than 120 active contributorsApple, ARM, Google, Qualcomm, QuIC, NVidia, AMD and more

~250 commits/week Clang

“The goal of the Clang project is to create a new C, C++, Objective C and Objective C++ front-end for the LLVM compiler.“

More than 100 active contributorsApple, ARM, AMD and more

~150 commits/week* Stats from last year until 14.10.2011


Correctness

http://root.cern.ch/drupal/content/requested-cling-features

cling – the new c++ interpreter for root 6 cern, ph-sft & fermilab v. vassilev, p. canal, a....

Documents

interpreter cling

semicolon vvassilev

maintainvvassilev chep

nutshellvvassilev chep

clangllvmvvassilev chep

statementsvvassilev

wholevvassilev chep

int addint