performance and extensibility with emf

Thursday, March 25, 2010 © IBM Corp., Committerati Consulting Corporation | EPL v1.0

Dave Steinberg, IBMMarcelo Paternostro, IBM

Kenn Hussey, Committerati

Getting the Most out of Your Models Performance and Extensibility with EMF

1


2

Exercise Prerequisites• USB drives are being passed around

– Separate images for 64-bit and 32-bit platforms– Copy the contents to a temporary location and pass it on

• Extract platform-appropriate eclipse-SDK-3.6M6-*.zip to some location– Do not nest deeply on Windows

• Launch Eclipse and use Help > Install New Software... to install the modeling features– Add a local site, and specify the modeling-projects/ subfolder of the

temporary location– Install all the features from the site– Accept installation of unsigned software– Restart the workbench

• Import all of the projects from emf-tutorial-workspace.zip


3

Agenda• Introduction• Performance

– Generated code– Resources– Exercise

• Extensibility– Validation delegates– Setting delegates– Invocation delegates– Exercise

• Q & A


4

As you already know...• EMF is the foundation for modeling and data

integration at Eclipse• Ecore models describe types of objects, their

attributes, operations, and relationships– Extensible import support including UML, XML Schema, and

annotated Java interfaces

• Merging generator enables mixing of generated and hand-written code

• Runtime framework provides notification, reflective API, dynamic EMF, XML persistence, validation...


5




• Q & A


6

Balancing Performance

Speed Memory


7

Balancing Performance

Speed Memory


8

Measuring Performance• Measuring performance in Java is tricky• Speed

– What are you measuring? (inlining, JIT, garbage collection, etc.)– Precision was a big problem for small operations, until Java 5

added System.nanoTime()

• Memory– What are you measuring? (deep size, shared objects, static fields,

etc.)– No way to estimate the size of a given object until Java 5 added

Instrumentation.getObjectSize()

• Comparisons are useful, single measurements not so much


9




• Q & A


public class EObjectImpl ...{ protected int eFlags; // deliver, dynamic, proxy protected BasicEList<Adapter> eAdapters; protected InternalEObject eContainer; protected int eContainerFeatureID; protected EPropertiesHolder eProperties; ...}

protected static class EPropertiesHolderImpl ...{ protected EClass eClass; protected Resource.Internal eResource; protected Object[] eSettings; protected URI eProxyURI; protected EList<EObject> eContents; protected EList<EObject> eCrossReferences; ...}

A fully fluffed up (but empty) instance has 144 bytes of overhead!

10

EObject Overhead32 bytes

EAdapterList≥ 32 bytes

ECrossReferenceEList≥ 24 bytes

EContentsEList≥ 24 bytes

32 bytes


11

Alternative EObject Implementations• DynamicEObjectImpl moves eSettings and eClass out of properties holder

• FlatEObjectImpl moves eProxyURI, eContents, and eCrossReferences out of properties holder


“EMF Ultra Slim Diet”• MinimalEObjectImpl uses a different approach:

– Dynamically sized array (or single object) for all overhead– Does not cache contents or cross references– Adapters stored as array or simply delegated to container

• Overhead of empty, fluffed up MinimalEObjectImpl.Container instance is just 24 bytes

12

public class MinimalEObjectImpl ...{ private int eFlags; private Object eStorage; ...

public static class Container extends MinimalEObjectImpl { protected InternalEObject eContainer; ... }}


Modeled Features

13

public class ProjectImpl extends EObjectImpl implements Project{ ...

protected ProjectPhase phase = PHASE_EDEFAULT; protected boolean ipClean = IP_CLEAN_EDEFAULT; protected boolean inSimultaneousRelease = IN_SIMULTANEOUS_RELEASE_EDEFAULT; ...

public ProjectPhase getPhase() { return phase; }

public boolean isIpClean() { return ipClean; }

...}

1 byte

4 bytes


Boolean Flags & Packed Enums

14

public class ProjectImpl extends EObjectImpl implements Project{ ... protected int flags = 0; protected static final int PHASE_EFLAG_OFFSET = 0; protected static final int PHASE_EFLAG = 0x3 << PHASE_EFLAG_OFFSET; private static final ProjectPhase[] PHASE_EFLAG_VALUES = ProjectPhase.values(); protected static final int IP_CLEAN_EFLAG = 1 << 2; protected static final int IN_SIMULTANEOUS_RELEASE_EFLAG = 1 << 3; ...

public ProjectPhase getPhase() { return PHASE_EFLAG_VALUES[ (flags & PHASE_EFLAG) >>> PHASE_EFLAG_OFFSET]; }

public boolean isIpClean() { return (flags & IP_CLEAN_EFLAG) != 0; } ...}


Boolean Flags & Packed Enums• Unset states also recorded in flags field • Reduces size by up to 87.5% for booleans

• Can leverage protected eFlags field in EObjectImpl (not MinimalEObjectImpl), but 8 bits must be reserved

15


Virtual Delegation

16

public class ProjectImpl extends EObjectImpl implements Project{ ... protected Object[] eVirtualValues; protected int eVirtualIndexBits0;

protected Object[] eVirtualValues() { return eVirtualValues; } ...

public ProjectPhase getPhase() { return (ProjectPhase)eVirtualGet( FoundationPackage.PROJECT__PHASE, PHASE_EDEFAULT); } ...}


Virtual Delegation• Appropriate only for very sparsely populated models

with many features – measure before adopting!

17


Reflective Method Dispatch

18

public class TopLevelProjectImpl extends ProjectImpl implements TopLevelProject{ ...

public Object eGet(int featureID, boolean resolve, boolean coreType) { switch (featureID) { case FoundationPackage.TOP_LEVEL_PROJECT__PROJECTS: return getProjects(); case FoundationPackage.TOP_LEVEL_PROJECT__PMC_MEMBERS: return getPmcMembers(); } return super.eGet(featureID, resolve, coreType); } ...}


O(1) Reflective Methods

19

public class TopLevelProjectImpl extends ProjectImpl implements TopLevelProject{ ... public Object eGet(int featureID, boolean resolve, boolean coreType) { switch (featureID) { case FoundationPackage.TOP_LEVEL_PROJECT__PHASE: return getPhase(); case FoundationPackage.TOP_LEVEL_PROJECT__IP_CLEAN: return isIpClean(); case FoundationPackage.TOP_LEVEL_PROJECT__IN_SIMULTANEOUS_RELEASE: return isInSimultaneousRelease(); case FoundationPackage.TOP_LEVEL_PROJECT__PROJECTS: return getProjects(); case FoundationPackage.TOP_LEVEL_PROJECT__PMC_MEMBERS: return getPmcMembers(); } return eDynamicGet(featureID, resolve, coreType); } ...}


O(1) Reflective Methods• Performance improvement possible only for deep inheritance

hierarchies• Trade-offs are memory impact of duplicated code (not heap)

and cross-model inheritance fragility (need to regenerate)

• Note: default reflective dispatch is not binary compatible

20


21




• Q & A


Identifying EObjects in Resources• Fragment paths

– The default, a slash-delimited path computed based on containment within the resource

– For multi-valued features, segments can be based simply on an index or on values of one or more “key” attributes

• Intrinsic IDs– Used when an EObject's EClass defines an “ID” attribute and that

attribute is set

• Extrinsic IDs (XML resources only)– Used only when an EObject is explicitly assigned an ID– Particular resource implementations may do this automatically

22


Extrinsic IDs

23

• EObject-Extrinisic ID associations are recorded in two maps (one for each direction)

• These maps are maintained by the XMLResource and should not be manipulated directly

XMLResource xmlResource = ... Project project = ... xmlResource.setID(project, "EMF-Project-123"); xmlResource.getContents().add(project); assert project == xmlResource.getEObject("EMF-Project-123"); assert "EMF-Project-123" == xmlResource.getID(project);


Extrinsic IDs

24

• Serialized via xmi:id or xsi:id attributes• UUIDs can be automatically assigned by the

resource

• Note: a single UUID consumes 88 bytes!

public class MyXMLResourceImpl extends XMLResourceImpl{ @Override protected boolean useUUIDs() { return true; }}


Intrinsic IDs• An EObject's intrinsic IDs is part of its state

– The string value of the EAttribute that is defined to be the ID of the EClass

• Finding an EObject by intrinsic ID involves walking over the entire resource

25

Resource resource = ... Project emf = (Project)resource.getEObject("EMF"); assert FoundationPackage.Literals.PROJECT__NAME.isID(); assert "EMF".equals(project.getName());


“Secret” Caches• It is possible to improve performance by using some

lesser known caches:– Intrinsic ID to EObject (in Resource)– URI to Resource (in ResourceSet)

• Both caches– can be pre-populated– are lazily populated as elements are retrieved

26


Intrinsic ID to EObject Cache

27

• Heads up:– EObjects that are removed from the resource are automatically

removed from the cache– EObjects with non-null intrinsic IDs that are added to the

resource are automatically added to the cache– The cache may become inconsistent (e.g. if an EObject's intrinsic

ID is changed after it has been added to the resource)

Resource resource = ...

Map<String, EObject> cache = new HashMap<String, EObject>(); ((ResourceImpl)resource).setIntrinsicIDToEObjectMap(cache);

Project project = ... project.setName("EMF"); assert FoundationPackage.Literals.PROJECT__NAME.isID();

resource.getContents().add(project);


URI to Resource Cache

28

• Obtaining the resource for a URI can be expensive, especially if it involves URI normalization

• Heads up:– Resources that are removed from the resource set are automatically

removed from the cache– The cache may become inconsistent (e.g. if a resource's URI is

changed)– URIs in the cache (key values) don't include fragments

ResourceSet resourceSet = ...

Map<URI, Resource> cache = new HashMap<URI, Resource>(); ((ResourceSetImpl)resourceSet).setURIResourceMap(cache);


Intrinsic ID Performance

29


Large Resource Performance

30


Cross-Resource Containment• Cross-resource containment allows an object

hierarchy to be persisted across multiple resources– eObject.eResource() may be different from

eObject.eContainer().eResource()

• Side benefit: facilitates finer-grained partial loading

31


Cross-Resource Containment

32

DynamicModel

EClass TopLevelProjectEClass = ...EClass projectEClass = ...

EReference projects = EcoreFactory.eINSTANCE.createEReference();projects.setName("projects");projects.setUpperBound(ETypedElement.UNBOUNDED_MULTIPLICITY);projects.setEType(projectEClass);projects.setContainment(true);projects.setResolveProxies(true);topLevelProjectEClass.getEStructuralFeatures().add(projects);

GeneratedModel


Binary Resources• EMF also supports a high-performance proprietary

binary serialization format– Numeric IDs for EObjects, metadata, URIs– String lengths specified, no delimiters– Signature to recognize corruption– Versioned to allow for future format changes

33


Binary Resources

34


Binary Resource

35

Resource.Factory binaryResourceFactory = new Resource.Factory() { public Resource createResource(URI uri) { return new BinaryResourceImpl(uri); } };

ResourceSet resourceSet = new ResourceSetImpl();resourceSet.getResourceFactoryRegistry(). getExtensionToFactoryMap().put("dat", binaryResourceFactory);Resource resource = resourceSet.createResource( URI.createFileURI("projects.dat"));


Save and Load Options• Resource implementations offer a set of options that

can be used to tweak load and save behavior• Options passed to the resource’s save and load

methods as entries in a map– Key: the constant that represents the option– Value: appropriately typed for the option

36


Save and Load Options

37

• Options are usually declared in– resource interface (e.g. XMLResource)– resource implementation class (e.g. BinaryResourceImpl)

• Get used to opening the EMF source code or Javadoc to read about them

XMLResource xmlResource = ...

Map<String, Object> options = new HashMap<String, Object>(); options.put(Resource.OPTION_ZIP, true); options.put(XMLResource.OPTION_USE_PARSER_POOL, new XMLParserPoolImpl()); options.put(XMLResource.OPTION_USE_CACHED_LOOKUP_TABLE, new ArrayList<Object>()); xmlResource.save(options);


Save and Load Options

38


A Nod to Scalability: CDO• Connected Data Objects (CDO): an EMF component

offering central persistence for models and instances– Multiple clients can have views on and transactions against a

single remote repository– Clients may modify the shared object graph concurrently– Other clients are immediately notified about modifications

• Supports CDO-targeted generated models, dynamic models, and “legacy” models

• The way forward for truly scalable EMF– Pluggable DB storage (Derby, MySQL, HSQLDB, etc.) – Full demand loading and unloading of objects, with partial

collection loading and adaptable pre-fetching

39


40




• Q & A


Exercise• Have Fun!

– The goal for the next few minutes is to let you experiment with EMF

– Use the knowledge you've learned during the tutorial so far:• Try saving and loading resources using

different options• See the difference between different types of

resources (XMI and binary for example)• Re-generate the model using different

genmodel settings

41


Sample Model

42


Cheat Sheet: Test Harness• In project org.eclipsecon.foundation.test:

– ModelFactory• Creates instances of the Foundation model• The number of instances can be controlled by constructor

arguments– MemoryTester

• Outputs the size of the instances of the model• Requires the “-javaagent” JVM argument

– ResourceTester• Base class for classes that time saving and loading resources

43


Cheat Sheet: Resource Options

44

• org.eclipse.emf.ecore.resource.Resource

– OPTION_ZIP– OPTION_SAVE_ONLY_IF_CHANGED

• OPTION_SAVE_ONLY_IF_CHANGED_MEMORY_BUFFER• OPTION_SAVE_ONLY_IF_CHANGED_FILE_BUFFER

• org.eclipse.emf.ecore.resource.impl.BinaryResourceImpl

– OPTION_BUFFER_CAPACITY


Cheat Sheet: XML Resource

45

• Load– OPTION_DEFER_IDREF_RESOLUTION

– OPTION_USE_PARSER_POOL– OPTION_USE_XML_NAME_TO_FEATURE_MAP

– OPTION_USE_DEPRECATED_METHODS

• Save– OPTION_FLUSH_THRESHOLD

– OPTION_CONFIGURATION_CACHE– OPTION_FORMATTED

– OPTION_USE_FILE_BUFFER– OPTION_USE_CACHED_LOOKUP_TABLE


PerformanceExercise 1

46


47




• Q & A


Extensibility• Many enhancements have been made to EMF over

the past few years to make the framework more extensible

• As of the M4 milestone of the Helios release, EMF has been made even more extensible with the addition of support for validation, feature setting, and operation invocation delegation

• We'll consider each of these new mechanisms in turn and look at how they can be implemented and used for our sample model

48


49




• Q & A


Validation Delegation• The EMF Validation Framework previously supported

evaluation of invariants and constraints, defined as methods and implemented in Java, via a validator that is invoked at important moments by an application or at a user’s discretion

• In order to delegate evaluation of invariants and constraints to an external engine, the framework was enhanced by introducing the concept of a validation delegate

50


Implementing Validation Delegates• Define a validation delegate by implementing the ValidationDelegate interface

• Register the validation delegate, either programmatically or via an extension

51


Using Validation Delegates• Reference the validation delegate via an annotation

on an Ecore package

• Define the behaviors of invariants and/or constraints via expressions in annotations on Ecore operations and/or classifiers

• Evaluate invariants and/or constraints, either statically via generated code or dynamically via EMF reflection

52


53




• Q & A


Feature Setting Delegation• EMF previously supported computation of features,

defined as methods and implemented in Java, via an API that is invoked at important moments by an application

• In order to delegate computation of a feature’s value to an external engine, the framework was enhanced by exposing the previously existing concept of a setting delegate

54


Implementing Setting Delegates• Define a setting delegate by implementing the SettingDelegate interface

• Define a setting delegate factory by implementing the Factory interface

• Register the setting delegate factory, either programmatically or via an extension

55


Using Setting Delegates• Reference the setting delegate factory via an

annotation on an Ecore package

• Define the values of features via expressions in annotations on Ecore structural features

• Compute the values of features, either statically via generated code or dynamically via EMF reflection

56


57




• Q & A


Operation Invocation Delegation• EMF previously supported execution of operations,

defined as methods and implemented in Java, via an API that is invoked at important moments by an application

• In order to delegate execution of an operation’s behavior to an external engine, the framework has been enhanced by introducing the concept of an invocation delegate

58


Implementing Invocation Delegates• Define an invocation delegate by implementing the InvocationDelegate interface

• Define an invocation delegate factory by implementing the Factory interface

• Register the invocation delegate factory, either programmatically or via an extension

59


Using Invocation Delegates• Reference the invocation delegate factory via an

annotation on an Ecore package

• Define the values of operations via expressions in annotations on Ecore operations

• Execute the behaviors of operations, either statically via generated code or dynamically via EMF reflection

60


61




• Q & A


ExtensibilityExercise 2

62


63




• Q & A


Questions?

64


• Online help– http://help.eclipse.org/ganymede/index.jsp?nav=/14

• Website– http://www.eclipse.org/emf

• Downloads• Wiki• FAQ• Newsgroup• Documentation

• Book– Eclipse Modeling Framework

• Second Edition– http://safari.awprofessional.com/9780321331885

Resources

65

http://help.eclipse.org/ganymede/index.jsp?nav=/14

http://help.eclipse.org/ganymede/index.jsp?nav=/14

http://www.eclipse.org/emf

http://www.eclipse.org/emf

http://safari.awprofessional.com/9780321331885

http://safari.awprofessional.com/9780321331885


Legal NoticesCopyright © IBM Corp. and Committerati Consulting Corporation, 2010. All rights reserved. This presentation and the source code in it are made available under the EPL, v1.0.Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.Eclipse and the Eclipse logo are trademarks of Eclipse Foundation, Inc.IBM and the IBM logo are trademarks or registered trademarks of IBM Corporation, in the United States, other countries or both.UML and XMI are registered trademarks of the Object Management Group.HTTP, Metadata, and XML are trademarks of the World Wide Web Consortium; marks of W3C are registered and held by its host institutions MIT, ERCIM, and Keio.Other company, product, or service names may be trademarks or service marks of others.

THE INFORMATION DISCUSSED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, AND IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, SUCH INFORMATION. ANY INFORMATION CONCERNING IBM'S PRODUCT PLANS OR STRATEGY IS SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

66

performance and extensibility with emf

Technology