java core | java 8 and osgi modularisation | tim ellison & neil bartlett
DESCRIPTION
The talk will cover a bit of background first to set things up: what is a module, why do we need a module system, summary of Java's existing support for modularity. Then it will move on to give a comparison of OSGi's and Jigsaw's dependency models. Pros and cons of each model in different environments will be discussed. Finally, opportunities and challenges for interoperability: from the perspective of both application developers (who may need to integrate modules from both kinds) and from library module developers (who may need to target both module systems)TRANSCRIPT
![Page 1: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/1.jpg)
Java 8 and OSGi modularization
Tim Ellison, IBM UK LabsNeil Bartlett, Paremus Ltd
![Page 2: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/2.jpg)
2
What is a module?Why do we need a module system?
• Modules facilitate software engineering & enhanced runtime capabilities
– Decomposition of complex systems
– Software assemblies of pre-built modules
– Provisioning / deployment, including module repositories
– Versioning and configuration management
• The module system is used to manage the modules, including installing, activating, resolving module requirements and conflicts.
• Interacting with the module system provides a higher level abstraction than the language provides directly.
![Page 3: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/3.jpg)
3
Modules in Java
• The module as a unit of encapsulation...
• The Java language already has two kinds of module: classes and packages.
– classes encapsulate fields and methods
– packages encapsulate classes and resources
• Problem: neither of these is a deployable unit (too granular, too tightly coupled).
• Java's unit of deployment is the JAR file.
– No encapsulation, therefore not a module!
• Problem: need an entity that is a deployable unit of encapsulation.
![Page 4: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/4.jpg)
4
Desirable characteristics of a module
• A module should be highly coherent.
• A module should be loosely coupled to other modules.
• A module should be explicit about what it provides to other modules.
• A module should be explicit about what it depends on from other modules.
![Page 5: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/5.jpg)
5
Why modules for the Java runtime (JRE)?
• Presently the JRE is a monolithic entity– in general the runtime must assume your application will use anything and everything
– class loaders provide runtime type definition and isolation
– download time and start-up time are directly impacted by the number of types available in the runtime
• Application start-up time includes a linear search through the class path to find system and application code
– e.g. Oracle 1.7 Windows bootclass path contains nearly 20k classes
– resources.jar rt.jar jsse.jar jce.jar charsets.jar
– rt.jar index alone is ~1Mb
• To the JRE, your application's jars’ indexes are disjoint & unsorted– there are JRE implementation 'tricks' like look aside tables and shared classes that can
help
– class loading is a BIG time hog to amortize over the length of the run
![Page 6: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/6.jpg)
6
Why modules for the Java runtime (JRE)? - cont.
• Dependency management– avoid “JAR hell” → trying satisfy competing requirements by simple path ordering
– type name space comprising (unversioned) package name is often insufficient
– e.g. my app depends upon foo-v2.jar and bar-v2.jarbut foo-v2.jar depends upon bar-v1.jar
-classpath foo-v2.jar; bar-v2.jar; bar-v1.jar –> my app “wins”
-classpath foo-v2.jar; bar-v1.jar; bar-v2.jar –> foo “wins”
• Module level visibility– public, protected, private and package-private level visibility means some
implementation types (com.sun.) need to be marked public to be called by java. APIs.
– convention asks people not to depend upon them...
–
• Version control– ability to define the compatibility of a module based on numbering scheme
– different module versions for different target devices / resource goals
![Page 7: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/7.jpg)
7
Introducing Project Jigsaw
• Project Jigsaw is an effort at OpenJDK
– defining a simple module system for Java
– modularizing the JRE itself
– originally intended as an implementation detail for the JRE / applications, but being adopted as a Java standard (JSR)
• Working within constraints of backwards compatibility
– existing Java APIs “cannot” be changed
– Some packages contain wildly different functionalitye.g. java.util contains collection hierarchy and Locale infrastructure
• Introduces language changes to define modulessrc/com/example/myclass.javasrc/com/example/myclassimpl.javasrc/module-info.java module com.example @ 1.0 {
requires jdk.base;requires foo @ 2.0;requires bar @ 2.0;
}
![Page 8: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/8.jpg)
8
What about OSGi?
• Existing, mature, de facto modularity solution for Java applications.
• Rich modularity model based on code-level (package) dependencies, module-level dependencies, and services.
• First practical solution for effective component reuse
– well supported by multiple implementations, tooling, etc.
• Millions use applications built with OSGi
– most application servers are based on OSGi
• No language changes.
– defines info in META-INF/MANIFEST.MF
Bundle-ManifestVersion: 2Bundle-SymbolicName: com.ex.mybundleBundle-Version: 1.0.0Import-Package: com.example.barExport-Package: com.example.foo
![Page 9: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/9.jpg)
9
Module dependency resolution
• The module system must resolve bundle dependencies, possibly in the face of multiple valid alternatives
– e.g. my app depends upon (foo-v2.jar and (bar-v1.jar or bar-v2.jar))and foo-v2.jar depends upon bar-v1.jar
– if my app touches bar first, the module system may chose bar-v2.jar
– if my app touches foo first, the module system will chose bar-v1.jar
• OSGi's dynamic dependency resolution always finds the best fit for currently invoked bundles at runtime
– Modules are apparent at runtime
– Runtime activation lifecycle: installed, resolved, starting, active, stopping, uninstalled
• Jigsaw resolves module dependencies during build, installation, or runtime
– Gives opportunity for load time optimizations such as pre-verify, indexing, etc.
![Page 10: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/10.jpg)
10
OSGi Dependency Model
• OSGi best practice is to depend on APIs.
• APIs are defined in terms of packages.
• Read almost any JSR. It defines the package(s) that the JSR specifies, e.g.:– JSR 112 → javax.resource.spi, cci– JSR 173 → javax.xml.stream– JSR 315 → javax.servlet (3.0)
![Page 11: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/11.jpg)
11
OSGi: As Simple As Possible...
• The primary unit of sharing is the package.
• Bundle A exports a package– Export-Package: api
• Bundle B imports a package– Import-Package: api
![Page 12: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/12.jpg)
12
But No Simpler!
• Module-to-module dependencies also supported– Require-Bundle: A
• Most OSGi devs consider Require-Bundle to be deprecated.
![Page 13: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/13.jpg)
13
Refactoring with Import-Package
![Page 14: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/14.jpg)
14
Refactoring with Require-Bundle
![Page 15: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/15.jpg)
15
Import-Package vs Require-Bundle
• Import-Package directly reflects reality
• Can be derived straight from the code!
• Require-Bundle is a parallel dependency system => manually maintained, hard to statically verify.
![Page 16: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/16.jpg)
16
Don't Split the Packages!
• In OSGi, the package is king.
• Import-Package will resolve to exactly one export of that package, and this is a good thing!!
• Require-Bundle can be used when packages are split for legacy/stupid reasons.
![Page 17: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/17.jpg)
17
More On Split Packages
• Java packages were never meant to be split across deployment units.
• Even plain Java provides a way to enforce this: Sealed Packages.
• You wouldn't split a method across classes... or a class across packages...
![Page 18: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/18.jpg)
18
Handling platform-specific code
• The JRE contains many classes and native code that are specific to an OS / CPU architecture
• “Universal modules”
– Contain code for all platforms
– Requires download of unused code, and new revisions that are not relevant
• “Platform-specific sub-modules”
– Put platfom-specific code in a separate module
– Many of the differences are in the native code only, so requires a native module
• “Native package”
– Hooks into the OS packaging system, so that dependencies can be handled by the OS (e.g. .deb file)
![Page 19: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/19.jpg)
19
Handling platform-specific code – OSGi style
• OSGi allows you to deliver platform-specific native code in a bundle JAR file.
Bundle-NativeCode:
libfoo.so; osname=Linux; processor=x86,
foo.dll; osname=Windows7; processor=x86
• The OSGi implementation of ClassLoader#findLibrary(“foo”) returns the path to the correct version.
• Flexibility to structure as a universal module, or sub-module.
http://www.osgi.org/Specifications/Reference
![Page 20: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/20.jpg)
20
Modularising the JRE
• JRE libraries evolved organically (haphazardly!) over 12+ years.
• Many cyclic dependencies• Many weird/unexpected dependencies• In particular java.util is an incoherent
mess (this always happens to general util packages!)
• => Split packages are unavoidable
![Page 21: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/21.jpg)
21
Can We Use OSGi?
• OSGi does support split packages, through Require-Bundle
• Works, though considered deprecated.• But that's not the whole story.
![Page 22: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/22.jpg)
22
Single Boot Classloader Assumption
• Many parts of the JRE assume a single boot classloader.
• Private package (aka “default”) access assumes whole packages are in single classloader.
• Therefore need to split some packages across modules but not across classloaders.
=> Need to break 1-to-1 module/classloader mapping.
![Page 23: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/23.jpg)
23
But OSGi Supports Classloader Sharing!
• True, through “fragments”.
• Fragments share the CL of their “host”.
• Dependency is in the “wrong” direction, i.e. fragment depends on host.
• Can OSGi be modified to support host-to-fragment dependencies?
• Almost certainly!
![Page 24: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/24.jpg)
24
Refactoring Verboten
• Normal and best solution: move classes to the correct, i.e. most logical package.
• In the JRE is this impossible!• 9 million developers (Sun estimate, 2009)
and billions of apps depend on this library.
![Page 25: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/25.jpg)
25
Irrelevant Stuff
• Let's not worry about:– Format of metadata– Location of metadata– Format of deployable modules
• Differences here are surmountable• E.g. OSGi already supports WAR file
deployment.
![Page 26: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/26.jpg)
26
Less Important Stuff
• Let's even not worry about versioning right now.
• 3 numeric segments vs 20...• Qualifier ordering, snapshots...• Again, we can adapt.
![Page 27: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/27.jpg)
27
Jigsaw Dependency Model
• Modules require other modules by name (and optionally, version)
module B @ 1.0 { require A @ [2.0,3.0);}
![Page 28: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/28.jpg)
28
Jigsaw Dependency Model
• Requirements can be “local”.
• Target module is loaded in same classloader as present module.
module B @ 1.0 { require local A @ [2.0,3.0);}
![Page 29: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/29.jpg)
29
Jigsaw Dependency Model
• Modules list their exports: at package and type level.
• May include re-exported contents of required modules
module B @ 1.0 { require A @ [2.0,3.0); export org.foo.ClassFoo; export org.bar.*}
![Page 30: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/30.jpg)
30
Jigsaw Dependency Model
• Modules can restrict which other modules are allowed to require them.
• Compare: “friend classes” in C++.• Note that the permit clause is un-versioned.
module A @ 2.0 { permit B;}
![Page 31: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/31.jpg)
31
Jigsaw Dependency Model
• Modules can logically “provide” other module names.
• Compare: “virtual packages” in Debian.• Supports substitution, but unclear how it will
support refactoring (splitting/aggregating).
module com.ibm.stax @ 1.0 { provide jdk.stax @ 2.0;}
![Page 32: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/32.jpg)
32
Jigsaw Versions
• Jigsaw modules will be versioned
• Module requirements use exact version or a range
• Versions have no semantics besides ordering
![Page 33: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/33.jpg)
33
Comparison
• Whole-module dependencies are the biggest difference.
• Jigsaw will suffer all the same problems seen in OSGi with Require-Bundle.
• E.g. Eclipse Core Runtime refactoring hell.
• While Jigsaw will achieve JDK modularity it will increase developer maintenance burden.
![Page 34: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/34.jpg)
34
Both Are Here to Stay!
• Try to get the best of both worlds!
• OSGi is established and will continue to be used by millions
• Jigsaw is underway and a key component in Java 8 plans (Summer 2013)
• It need not be a zero-sum game
![Page 35: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/35.jpg)
35
Project Penrose: Jigsaw / OSGi interop (proposed)
• Level 0 : toleration
– ensure that OSGi frameworks continue to run unmodified on a Jigsaw enabled runtime
– creating modules / bundles that have both Jigsaw & OSGi metadata on the same JAR
• Level 1 : interop of module info metadata
– teach OSGi to read Jigsaw module info
• mapping Jigsaw metadata into OSGi format for the framework to understand, e.g. requires ⇒ Require-Bundle:
– resolve Jigsaw modules using the OSGi resolver
![Page 36: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/36.jpg)
36
Project Penrose: Jigsaw / OSGi interop (proposed)
• Level 2 : OSGi implementation exploit of Jigsaw modularity
– Enhance OSGi to use Jigsaw publication repositories etc.
• Level 3+ : Full interop
– A blend of OSGi and Jigsaw cross delegation on module phases
OSGi supportIt must be demonstrated by prototype to be feasible to modify an OSGi micro-kernel such that OSGi bundles running in that kernel can depend upon Javamodules. The kernel must be able to load Java modules directly and resolvethem using its own resolver, except for core system modules. Core systemmodules can only be loaded using the module system’s reification API. http://openjdk.java.net/projects/jigsaw/doc/draft-java-module-system-requirements-12
![Page 37: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/37.jpg)
37
Project Penrose: results to date
• Level 0 : toleration
– ensure that OSGi frameworks continue to run unmodified on a Jigsaw enabled runtime
– creating modules / bundles that have both Jigsaw & OSGi metadata on the same JAR
• Achievement #1: pass the OSGi tests on a Jigsaw enabled runtime
– Equinox 3.7 – the OSGi reference implementation
– OSGi 4.3 Compliance Tests – with minor patch for test case error
– Windows XP sp3, junit-3.8.2, java6u26, java7-ea-b145
• Achievement #2: run a Java application as either OSGi or Jigsaw modules
– Took Java 2D demo
– Broke it into multiple functional units, and run the demo as OSGi bundles OR Jigsaw modules
![Page 38: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/38.jpg)
38
Project Penrose: next steps
• Level 1 : interop of module info metadata
– teach OSGi to read Jigsaw module info
• mapping Jigsaw metadata into OSGi format for the framework to understand, e.g. requires ⇒ Require-Bundle:
– resolve Jigsaw modules using the OSGi resolver
• Goal #0: move the work into a new OpenJDK project
• Goal #1: map Jigsaw module metadata to OSGi equivalents
– discussions of how OSGi shall interpret Jigsaw directives
• Goal #2: modify OSGi to use Jigsaw reification APIs
– load a third-party module from the Jigsaw repository
– resolve it using the OSGi resolver
![Page 39: Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett](https://reader030.vdocument.in/reader030/viewer/2022020207/5555ccc9d8b42a711f8b4869/html5/thumbnails/39.jpg)
39
Advice
• Stop using internal APIs and implementation classes
– You will be broken come Java 8
• If you need a modularity story that works before summer 2013 then the answer is OSGi
– OSGi will continue to work beyond Java 7 too so it is the safest option
• OSGi has greater expressiveness
– While you may not need it now, do you want to change system when you do need it?
• Think about how to interoperate with Jigsaw
– Be modest in your API usage and we can likely help with a tailored JRE