generic programming galore using d
TRANSCRIPT
![Page 1: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/1.jpg)
c© 2011 Andrei Alexandrescu 1 / 33
Generic Programming Galore Using D
Andrei Alexandrescu, PhD
Research Scientist
![Page 2: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/2.jpg)
37% off
c© 2011 Andrei Alexandrescu 2 / 33
![Page 3: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/3.jpg)
Generic Programming
c© 2011 Andrei Alexandrescu 3 / 33
![Page 4: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/4.jpg)
What is Generic Programming?
c© 2011 Andrei Alexandrescu 4 / 33
![Page 5: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/5.jpg)
What is Generic Programming?
c© 2011 Andrei Alexandrescu 4 / 33
• Find most general algorithm representation◦ Narrowest requirements◦ Widest guarantees◦ Big-O() encapsulation should be a crime◦ No need to regress to hand-written code
![Page 6: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/6.jpg)
What is Generic Programming?
c© 2011 Andrei Alexandrescu 4 / 33
• Find most general algorithm representation◦ Narrowest requirements◦ Widest guarantees◦ Big-O() encapsulation should be a crime◦ No need to regress to hand-written code
• Define types to implement said requirements
![Page 7: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/7.jpg)
What is Generic Programming?
c© 2011 Andrei Alexandrescu 4 / 33
• Find most general algorithm representation◦ Narrowest requirements◦ Widest guarantees◦ Big-O() encapsulation should be a crime◦ No need to regress to hand-written code
• Define types to implement said requirements• Leverage the algorithm for ultimate reuse
![Page 8: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/8.jpg)
What is Generic Programming?
c© 2011 Andrei Alexandrescu 4 / 33
• Find most general algorithm representation◦ Narrowest requirements◦ Widest guarantees◦ Big-O() encapsulation should be a crime◦ No need to regress to hand-written code
• Define types to implement said requirements• Leverage the algorithm for ultimate reuse• . . .• Profit!
![Page 9: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/9.jpg)
What is Generic Programming?
c© 2011 Andrei Alexandrescu 4 / 33
• Find most general algorithm representation◦ Narrowest requirements◦ Widest guarantees◦ Big-O() encapsulation should be a crime◦ No need to regress to hand-written code
• Define types to implement said requirements• Leverage the algorithm for ultimate reuse• . . .• Profit!
• Arguably one of the noblest endeavors of our metier
![Page 10: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/10.jpg)
Generic Programming
c© 2011 Andrei Alexandrescu 5 / 33
• “Write once, instantiate anywhere”◦ Specialized implementations welcome
• Prefers static type information and static expansion(macro style)
• Fosters strong mathematical underpinnings◦ Minimizing assumptions common theme in math
• Tenuous relationship with binary interfaces• Starts with algorithms, not interfaces or objects• “Premature encapsulation is the root of some
derangement”
![Page 11: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/11.jpg)
Warmup: the “dream min”
c© 2011 Andrei Alexandrescu 6 / 33
• Should work at efficiency comparable to hand-writtencode
• Take variadic arguments: min(a, b, ...)
◦ Avoid nonsensical calls min() and min(a)
• Work for all ordered types and conversions• Decline incompatible types, without prejudice
![Page 12: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/12.jpg)
First prototype
c© 2011 Andrei Alexandrescu 7 / 33
auto min(L, R)(L lhs, R rhs) {
return rhs < lhs ? rhs : lhs;
}
auto a = min(x, y);
• This function is as efficient as any specialized,handwritten version (true genericity)
• Deduction for argument types and result type
![Page 13: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/13.jpg)
Variadic arguments
c© 2011 Andrei Alexandrescu 8 / 33
auto min(T...)(T x) {
static if (x.length > 2)
return min(min(x[0], x[1]), x[2 .. $]);
else
return x[0] > x[1] ? x[1] : x[0];
}
...
auto m = min(a + b, 100, c);
• x is not an array• This is not classic recursion
![Page 14: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/14.jpg)
Reject nonsensical calls
c© 2011 Andrei Alexandrescu 9 / 33
auto min(T...)(T x) if (x.length > 1) {
...
}
• Rejects calls with 0 or 1 arguments• Allows other overloads to take over, e.g. min element
over a collection• More work needed
◦ Only accept types with a valid intersection◦ Only accept types comparable with ”<”
![Page 15: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/15.jpg)
Common type
c© 2011 Andrei Alexandrescu 10 / 33
• Task: Given a list of types, find the common type of all
template CommonType(T...)
{
static if (T.length == 1)
alias T[0] CommonType;
else
static if (is(typeof(1 ? T[0].init : T[1].init) U))
alias CommonType!(U, T[2 .. $]) CommonType;
else
alias void CommonType;
}
// Usage
static assert(is(CommonType!(int, short, long) == long));
![Page 16: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/16.jpg)
Using CommonType
c© 2011 Andrei Alexandrescu 11 / 33
auto min(T...)(T x)
if (x.length > 1
&& is(typeof(CommonType!T.init < CommonType!T.init)
== bool))
{
static if (x.length > 2)
return min(min(x[0], x[1]), x[2 .. $]);
else
return x[0] > x[1] ? x[1] : x[0];
}
![Page 17: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/17.jpg)
How about min over many elements?
c© 2011 Andrei Alexandrescu 12 / 33
auto min(R)(R range)
if (isInputRange!R &&
is(typeof(range.front < range.front) == bool))
{
auto result = range.front;
range.popFront();
foreach (e; range) {
if (e < result) result = e;
}
return result;
}
auto m = [ 1, 5, 2, 0, 7, 9 ].min();
• Works over anything that can be iterated
![Page 18: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/18.jpg)
How about argmin now?
c© 2011 Andrei Alexandrescu 13 / 33
auto argmin(alias fun, R)(R r)
if (isInputRange!R &&
is(typeof(fun(r.front) < fun(r.front)) == bool))
{
auto result = r.front;
auto cache = fun(result);
r.popFront();
foreach (e; r) {
auto cand = fun(e);
if (cand > cache) continue;
result = e;
cache = cand;
}
return result;
}
![Page 19: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/19.jpg)
argmin
c© 2011 Andrei Alexandrescu 14 / 33
auto s = ["abc", "a", "xz"];
auto m = s.argmin!((x) => x.length);
• Works on anything iterable and any predicate• Predicate is passed by alias• No loss of efficiency
![Page 20: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/20.jpg)
The Generative Connection
c© 2011 Andrei Alexandrescu 15 / 33
![Page 21: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/21.jpg)
Generative programming
c© 2011 Andrei Alexandrescu 16 / 33
• In brief: code that generates code• Generic programming often requires algorithm
specialization• Specification often present in a DSL
![Page 22: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/22.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 17 / 33
Force into host language’s syntax?
![Page 23: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/23.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 18 / 33
• Formatted printing?
![Page 24: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/24.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 18 / 33
• Formatted printing?
• Regular expressions?
![Page 25: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/25.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 18 / 33
• Formatted printing?
• Regular expressions?
• EBNF?
![Page 26: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/26.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 18 / 33
• Formatted printing?
• Regular expressions?
• EBNF?
• PEG?
![Page 27: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/27.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 18 / 33
• Formatted printing?
• Regular expressions?
• EBNF?
• PEG?
• SQL?
![Page 28: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/28.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 18 / 33
• Formatted printing?
• Regular expressions?
• EBNF?
• PEG?
• SQL?
• . . . Pasta for everyone!
![Page 29: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/29.jpg)
Embedded DSLs
c© 2011 Andrei Alexandrescu 19 / 33
Here: use with native grammar
Process during compilation
Generate D code accordingly
![Page 30: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/30.jpg)
Compile-Time Evaluation
c© 2011 Andrei Alexandrescu 20 / 33
• A large subset of D available for compile-time evaluation
ulong factorial(uint n) {
ulong result = 1;
foreach (i; 2 .. n) result *= i;
return result;
}
...
auto f1 = factorial(10); // run-time
static f2 = factorial(10); // compile-time
![Page 31: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/31.jpg)
Code injection with mixin
c© 2011 Andrei Alexandrescu 21 / 33
mixin("writeln(\"hello, world\");");
mixin(generateSomeCode());
• Not as glamorous as AST manipulation but darneffective
• Easy to understand and debug
• Now we have compile-time evaluation AND mixin. . .
![Page 32: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/32.jpg)
c© 2011 Andrei Alexandrescu 22 / 33
Wait a minute!
![Page 33: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/33.jpg)
Example: bitfields in library
c© 2011 Andrei Alexandrescu 23 / 33
struct A {
int a;
mixin(bitfields!(
uint, "x", 2,
int, "y", 3,
uint, "z", 2,
bool, "flag", 1));
}
A obj;
obj.x = 2;
obj.z = obj.x;
![Page 34: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/34.jpg)
Parser
c© 2011 Andrei Alexandrescu 24 / 33
import pegged.grammar; // by Philippe Sigaud
mixin(grammar("
Expr < Factor AddExpr*
AddExpr < (’+’/’-’) Factor
Factor < Primary MulExpr*
MulExpr < (’*’/’/’) Primary
Primary < Parens / Number / Variable
/ ’-’ Primary
Parens < ’(’ Expr ’)’
Number <~ [0-9]+
Variable <- Identifier
"));
![Page 35: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/35.jpg)
Usage
c© 2011 Andrei Alexandrescu 25 / 33
// Parsing at compile-time:
static parseTree1 = Expr.parse(
"1 + 2 - (3*x-5)*6");
pragma(msg, parseTree1.capture);
// Parsing at run-time:
auto parseTree2 = Expr.parse(readln());
writeln(parseTree2.capture);
![Page 36: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/36.jpg)
Scaling up
c© 2011 Andrei Alexandrescu 26 / 33
1000 lines of D grammar →3000 lines D parser
![Page 37: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/37.jpg)
c© 2011 Andrei Alexandrescu 27 / 33
Highly integrated lex+yacc
![Page 38: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/38.jpg)
c© 2011 Andrei Alexandrescu 28 / 33
What about regexen?
![Page 39: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/39.jpg)
Compile-time regular expressions
c© 2011 Andrei Alexandrescu 29 / 33
• GSoC project by Dmitry Olshansky: FReD• Fully UTF capable, no special casing for ASCII• Two modes sharing the same backend:
auto r1 = regex("^.*/([^/]+)/?$");
static r2 = ctRegex!("^.*/([^/]+)/?$");
• Run-time version uses intrinsics in a few places• Static version generates specialized automaton, then
compiles it
![Page 40: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/40.jpg)
![Page 41: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/41.jpg)
Summary
c© 2011 Andrei Alexandrescu 31 / 33
![Page 42: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/42.jpg)
Summary
c© 2011 Andrei Alexandrescu 32 / 33
• Generic/generative programming—long unattainedpromise
• D’s angle◦ Powerful base language◦ Type manipulation◦ Compile-time evaluation◦ Code generation
![Page 43: Generic Programming Galore Using D](https://reader033.vdocument.in/reader033/viewer/2022042700/557e7bbdd8b42a4d108b4f4f/html5/thumbnails/43.jpg)
Grill the Speaker!
c© 2011 Andrei Alexandrescu 33 / 33
More about ranges? • Thought the talk was
boring • Intriguing! • He lost me at mixin •
Awesome • Went over my head • Meh • I wonder
how I can implement binary search • What’s for
dinner? • Accent is bothersome • Real
programmers use C • Must. . . control. . . fist. . .
of. . . death. . .