the next generation mop, jochen theodorou, gr8conf 2013
DESCRIPTION
http://gr8conf.eu/Presentations/The-next-generation-MOPTRANSCRIPT
![Page 1: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/1.jpg)
The next generation MOP
Jochen "blackdrag" Theodorou
![Page 2: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/2.jpg)
About blackdrag● Working on Groovy Core since about 2005 ● Almost as long as that, Tech Lead of Groovy
● Currently Employed at Pivotal
● Responsible for most of the technical side of Groovy
Email: [email protected]
![Page 3: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/3.jpg)
About a new MOP
● discussions since 2005
● good-for-all solution always missing
● some ideas open to discussion
● and some cleanup duties
● we insist on you helping us
![Page 4: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/4.jpg)
Why change
![Page 5: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/5.jpg)
Why change● has many inconsistencies
● makes many optimizations impossible
● in theory powerful
● have to know implementation details
● Impossible to extend and spec resistent
● the API just plain sucks
![Page 6: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/6.jpg)
Some Basics and History
![Page 7: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/7.jpg)
The Meta Class● Each class has a meta class
● Saves all the dynamic and static properties/methods for Groovy
● Internal control flow involves exceptions
● Designed for invoking methods through the meta class
![Page 8: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/8.jpg)
The Meta Class● early Groovy meta programming:
● MOP methods invokeMethod and get/setProperty● A custom meta class (foo.metaClass = x)
● later class based automated meta class lookup
● transformed into the meta class creation handle (used by ExpandoMetaClass#enableGlobally())
![Page 9: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/9.jpg)
The Meta Class
● Basic principle: Each instance has a meta class
● More specified: Only every GroovyObject instance(later we changed that with a global map)
● Global registry specifying initial meta class on first use
![Page 10: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/10.jpg)
The Meta Class // myMetaClass some custom metaclass
// meta class in registry differentdef x1 = new X()assert x1.metaClass != myMetaClassx1.metaClass = myMetaClassassert x1.metaClass == myMetaClassdef x2 = new X()assert x2.metaClass != x1.metaClassX.metaClass = myMetaClassdef phantom = new X()def x3 = new X()assert x3.metaClass == x1.metaClassassert x3.metaClass != x2.metaClass
![Page 11: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/11.jpg)
The Meta Class
X.metaClass = x2.metaClassassert phantom.metaClass == ???
![Page 12: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/12.jpg)
Adding Methods/Properties● Standard meta class: MetaClassImpl
● does not support modifications
● New meta class for this: ExpandoMetaClass
● enabled with ExpandoMetaClass.enableGlobally()
● not always equally behaving to MetaClassImpl
![Page 13: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/13.jpg)
More MetaClasses● ProxyMetaClass (intercepting, decorating)● MixinMetaClass (mixins)● ClosureMetaClass (GeneratedClosure)● DelegatingMetaClass (base class)● OwnedMetaClass (related to mixins)● HandleMetaClass (related to mixins)
Plus your own custom meta class
![Page 14: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/14.jpg)
DSL not consistentFoo.metaClass.bar = 1 //defines propertyFoo.metaClass.bar = {1} //defines method
to use a closure as property:
foo.metaClass.bar = nullfoo.bar = {1}
● only for the instance● get metaproperty and set initial value creator
![Page 15: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/15.jpg)
Overriding Super Methodsclass A { def doIt(){two() + ' done.'}def two(){'two'}
} class B extends A {}B.metaClass.two = {'my new two!'}def b = new B() assert b.two() == 'my new two!'assert b.doIt() == 'two done.'
![Page 16: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/16.jpg)
Overriding Super MethodsTo make it work:
class A implements GroovyInterceptable { def doIt(){two() + ' done.'}def two(){'two'}
} class B extends A {}B.metaClass.two = {'my new two!'}def b = new B() assert b.two() == 'my new two!'assert b.doIt() == 'my new two! done.'
![Page 17: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/17.jpg)
Adding Super Methodsclass A { def doIt(){two() + ' done.'}def methodMissing(String name, args){'two'}
} class B extends A {}def b = new B()assert b.two() == 'two'assert b.doIt() == 'two done.'A.metaClass.two = {'my new two!'}assert b.two() == 'my new two!'assert b.doIt() == 'my new two! done.'
![Page 18: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/18.jpg)
Super Methods OverloadDoes not:
class A { def doIt(x){two(x) + ' done.'}def two(x) {'two'}
} class B extends A {}def b = new B()assert b.two('1') == 'two'assert b.doIt('1') == 'two done.'A.metaClass.two = {String s->'my new two!'}assert b.two('1') == 'my new two!'assert b.doIt('1') == 'my new two! done.'
![Page 19: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/19.jpg)
Private Multi Methodsclass A { def doIt(){two() + ' done.'}
} class B extends A {private two(){1}
}def b = new B()assert b.two() == 1assert b.doIt() == '1 done.'
![Page 20: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/20.jpg)
Speaking of privateclass A { private foo=1def b={foo}
} class B extends A {}def b = new B()assert b.b() == 1 //fails
● Information loss thorugh Closure#getProperty
![Page 21: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/21.jpg)
get/setMetaClass
● persistency framework needs to be aware
● transient works for Serialization
● what about other frameworks?
● seamless integration anyone?
![Page 22: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/22.jpg)
Propertiesclass X extends
org.xml.sax.helpers.XMLFilterImpl {def foo
}
● XMLFilterImpl has a get/setProperty
● cannot do new X().foo = bar
● cannot do println new X().foo
![Page 23: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/23.jpg)
invokeMethod
No such conflict known.... but!
● dynamic entry point from Java● as methodMissing ● with GroovyInterceptable (EMC too) as upfront
method
conflicting concepts
![Page 24: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/24.jpg)
What to make better?
![Page 25: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/25.jpg)
… besides fixing those problems
![Page 26: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/26.jpg)
Optimization efforts Lesson:
Java7 with invokedynamic is much better suited for Groovy's dynamic method calls
Reaction:
make Java7 the default (backport); rewrite DefaultGroovyMethods to use indy; throw out a lot of old code
![Page 27: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/27.jpg)
Optimization efforts Lesson:
Hotspot is not happy about invoking target methods in the meta class (mega morphic call sites)
Reaction:
The meta class only gives back something you can call and does not do the call itself.
![Page 28: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/28.jpg)
Optimization efforts Lesson:
Synchronization, Locks, volatiles usages on each method call destroy multithread performance as well as hotspot optimizations. Most applications set up mc changes on startup.
Reaction:
metaclass goes immutable; adding methods creates new meta class; lazy thread update (user level synchronization required)
![Page 29: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/29.jpg)
Hot Swapping Lesson:
Keeping internal state in the class is bad (see timestamp_xxx, timestamp, $callSiteArray)
Reaction:
Removal. CallSiteArray not needed anymore, the timestamps are kept track off by the loader, not in the class
![Page 30: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/30.jpg)
Optimization efforts Lesson:
Garbage collecting and recreating meta classes is very bad.
Reaction:
Keep the base meta class around and reuse everything possible to make new meta classes as lean as possible
![Page 31: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/31.jpg)
API Design Lesson:
Conventions are good, forcing them is bad (GroovyObject)
Reaction:
Don't implement GroovyObject by default anymore.
![Page 32: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/32.jpg)
General Design Lesson:
Too many ways of doing the same thing is no good
Reaction:
Most probably only methodMissing/propertyMissing anymore but easy way to „register“ a method to do invokeMethod.
![Page 33: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/33.jpg)
API Design Lesson:
Having multiple, not equivalent entry points is bad. (MetaClassImpl has 3 for methods, multiusage of invokeMethod, information loss through get/setProperty)
Reaction:
Clean up the API to have only one entry point (removal of MetaClass#invokeMethod)
![Page 34: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/34.jpg)
Possibilities
![Page 35: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/35.jpg)
Internal vs. ExternalInternal usage:class X { def methodMissing(String name, args) {1}}
External usage:class X {}X.metaClass.methodMissing = {1}
Combined:class X { static {this.metaClass.methodMissing = {1}}}
![Page 36: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/36.jpg)
Dynamic Invoke from JavaBefore:
GroovyObject foo = ...;String s = (String)foo.invokeMethod(“bar“, new Object[]{});
After:
Object foo = ...;String s = MopInterface.invoke(foo, “bar“);
● helper class for dynamic interactions with Groovy● similiar for properties
![Page 37: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/37.jpg)
Adding a method from JavaBefore:GroovyObject foo = ...;foo.getMetaClass().registerInstanceMethod(“foo“,
new Closure(null){public Object doCall(){1}};
After:Object foo = ...;MopInterface.addInstanceMethod(foo, “foo“,
new T() {public Object myFancyMethod(){1}});
● Doesn't have to be a Closure or MetaMethod● All declared methods used (lambdas too)
![Page 38: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/38.jpg)
Limited Meta Class ChangesUse Case:
unrolling all changes a GroovyShell or Eval.me did
Before:
● tracking meta class mutations impossible● „unmutate“ the metaclass certainly is● can only to track newly set meta classes
![Page 39: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/39.jpg)
Limited Meta Class ChangesUse Case:
I am a library developer and I don't want intrusive changes from user code to my library classes, changing the way my library is calling methods.
Before:
If the change is not effecting the library, then most probably because of a bug.
![Page 40: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/40.jpg)
Realms
![Page 41: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/41.jpg)
RealmsA realm here defines how the class in the realm
„sees“ the meta classes. Different classes can have different realms.
MyClass(y)
MyOtherClass(x)
x.foo()
using metaClass(x)
![Page 42: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/42.jpg)
Realms
MyClass(y)
MyOtherClass(x)
x.foo()
using metaClass(x) from realm(MyClass)
<Realm defined by MyClass>
![Page 43: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/43.jpg)
Realms
MyClass(y)
MyOtherClass(x)
y.bar()
using metaClass(y) from realm(MyOtherClass)
<Realm defined by MyOtherClass>
![Page 44: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/44.jpg)
Realms
INIT
DEFAULT
realm(A_n)
realm(C_n)realm(A_1)
........
realm(B_1)
realm(C_1)........
realm(D_1)
![Page 45: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/45.jpg)
RealmsINIT does not allow changing the meta class
INIT
DEFAULT
Non isolated
isolated
![Page 46: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/46.jpg)
RealmsDefault gets all the unspecified changes
INIT
DEFAULT
Non isolated
isolated
![Page 47: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/47.jpg)
RealmsIsolated realms don't get unspecific changes
INIT
DEFAULT
Non isolated
isolated
![Page 48: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/48.jpg)
RealmsNon isolated realms can override meta classes in
default without impossing themINIT
DEFAULT
Non isolated
isolated
![Page 49: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/49.jpg)
Isolated Realm@Realm(
marker=SomeMarkerClass, parentRealm=Realm.INIT)
class MyLibraryClass {def foo(){bar()}def bar(){1}
}MyLibraryClass.metaClass.bar = {2}assert new MyLibraryClass().foo() == 1
![Page 50: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/50.jpg)
Non-Isolated Realm@Realm(marker=SomeMarkerClass)class MyLibraryClass {def foo(){bar()}def bar(){1}
}MyLibraryClass.metaClass.bar = {2}assert new MyLibraryClass().foo() == 2
![Page 51: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/51.jpg)
Testing Realm
class X {private foo(){1}}class MyTest extends GroovyTestCase {
@Realm(marker=SomeMarkerClass, allowPrivate=true)
void testPrivateWorking() {def x = new X()assert x.foo() == 1
}void testPrivateNotWorking() {
def x = new X()shouldFail(MissingMethodException) {
x.foo()}
}}
![Page 52: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/52.jpg)
Limited Meta Class ChangesBefore:
Change intrusive, visible to everyone
After:
def foo(){1}assert foo() == 1realm.withSubRealm {
this.metaClass.foo = {2}assert foo() == 2
}assert foo() == 1
![Page 53: The Next Generation MOP, Jochen Theodorou, GR8Conf 2013](https://reader033.vdocument.in/reader033/viewer/2022051613/54c781744a7959c6148b462e/html5/thumbnails/53.jpg)
Q/A?