Modularity in Java 9: Stacking up with Project Jigsaw, Penrose, and OSGi

This article provides an overview of proposals, specifications, and platforms aimed at making Java technology more modular in Java 9. I'll discuss factors contributing to the need for a more modular Java architecture, briefly describe and compare the solutions that have been proposed, and introduce the three modularity updates planned for Java 9, including their potential impact on Java development.

Why do we need Java modularity?

Modularity is a general concept. In software, it applies to writing and implementing a program or computing system as a number of unique modules, rather than as a single, monolithic design. A standardized interface is then used to enable the modules to communicate. Partitioning an environment of software constructs into distinct modules helps us minimize coupling, optimize application development, and reduce system complexity.

Modularity enables programmers to do functionality testing in isolation and engage in parallel development efforts during a given sprint or project. This increases efficiency throughout the entire software development lifecycle.

Some characterizing attributes of a genuine module are:

• An autonomous unit of deployment (loose coupling)
• A consistent and unique identity (module ID and version)
• Easily identified and discovered requirements and dependencies (standard compile-time and deployment facilities and meta-information)
• An open and understandable interface (communication contract)
• Hidden implementation details (encapsulation)

Systems that are built to efficiently process modules should do the following:

• Support modularity and dependency-discovery at compile-time
• Execute modules in a runtime environment that supports easy deployment and redeployment without system downtime
• Implement an execution lifecycle that is clear and robust
• Provide facilities for easy registry and discovery of modules

Object-oriented, component-oriented, and service-oriented solutions have all attempted to enable pure modularity. Each solution has its own set of quirks that prevent it from achieving modular perfection, however. Let's briefly consider.

Java classes and objects as modular constructs

Doesn't the object-oriented nature of Java satisfy the requirements of modularity? After all, object-oriented programming with Java stresses and sometimes enforces uniqueness, data encapsulation, and loose coupling. While these points are a good start, notice the modularity requirements that aren't met by Java's object-oriented framework: identity at the object level is unreliable; interfaces are not versioned: and classes are not unique at the deployment level. Loose coupling is a best practice, but certainly not enforced.

Reusing classes in Java is difficult when third-party dependencies are so easily misused. Compile-time tools such as Maven seek to address this shortcoming. After-the-fact language conventions and constructs such as dependency injection and inversion-of-control help developers in our attempts to control the runtime environment, and sometimes they succeed, especially if used with strict discipline. Unfortunately, this situation leaves the chore of creating a modular environment up to proprietary framework conventions and configurations.

Java also adds package namespaces and scope visibility to the mix as a means for creating modular compile-time and deployment-time mechanisms. But these language features are easily sidestepped, as I'll explain.

Packages as a modular solution

Packages attempt to add a level of abstraction to the Java programming landscape. They provide facilities for unique coding namespaces and configuration contexts. Sadly, though, package conventions are easily circumvented, frequently leading to an environment of dangerous compile-time couplings.

The state of modularity in Java at present (aside from OSGi, which I will discuss shortly) is most often accomplished using package namespaces, JavaBeans conventions, and proprietary framework configurations like those found in Spring.

Aren't JAR files modular enough?

JAR files and the deployment environment in which they operate greatly improve on the many legacy deployment conventions otherwise available. But JAR files have no intrinsic uniqueness, apart from a rarely used version number, which is hidden in a .jar manifest. The JAR file and the optional manifest are not used as modularity conventions within the Java runtime environment. So the package names of classes in the file and their participation in a classpath are the only parts of the JAR structure that lend modularity to the runtime environment.

In short, JARs are a good attempt at modularization, but they don't fulfill all the requirements for a truly modular environment. Frameworks and platforms like Spring and OSGi use patterns and enhancements to the JAR specification to provide environments for building very capable and modular systems. Over time, however, even these tools will succumb to a very unfortunate side-effect of the JAR specification JAR hell!

Classpath/JAR hell

When the Java runtime environment allows for arbitrarily complex JAR loading mechanisms, developers know they are in classpath hell or JAR hell. A number of configurations can lead to this condition.

First, consider a situation where a Java application developer provides an updated version of the application and has packaged it in a JAR file with the exact same name as the old version. The Java runtime environment provides no validation facilities for determining the correct JAR file. The runtime environment will simply load classes from the JAR file that it finds first or that satisfies one of many classpath rules. This leads to unexpected behavior at best.

Another instance of JAR hell arises where two or more applications or processes depend on different versions of a third-party library. Using standard class-loading facilities, only one version of the third-party library will be available at runtime, leading to errors in at least one application or process.

A full-featured and efficient Java module system should facilitate separation of code into distinct, easily understood, and loosely coupled modules. Dependencies should be clearly specified and strictly enforced. Facilities should be available that allow modules to be upgraded without having a negative effect on other modules. A modular runtime environment should enable configurations that are specific to a particular domain or vertical market, thus reducing the startup time and system footprint of the environment.

Modularity solutions for Java

Along with the modularity features mentioned so far, recent efforts add a few more. The following features are intended to optimize performance and enable extending the runtime environment:

• Segmented source code: Source code separated into distinct, cached segments, each of which contains a specific type of compiled code. The goals of which include skipping non-method code during garbage sweeps, incremental builds, and better memory management.
• Build-time enforcements: Language constructs to enforce namespaces, versioning, dependencies, and others.
• Deployment facilities: Support for deploying scaled runtime environments according to specific needs, such as those of a mobile device environment.

A number of modularity specifications and frameworks have sought to facilitate these features, and a few have recently risen to the top in proposals for Java 9. An overview of Java modularity proposals is below.

JSR (Java Specification Request) 277

Currently inactive is Java Specification Request (JSR) 277, the Java Module System; introduced by Sun in June of 2005. This specification covered most of the same areas as OSGi. Like OSGi, JSR 277 also defines discovery, loading, and consistency of modules, with sparse support for runtime modifications and/or integrity checking.

Drawbacks to JSR 277 include:

• No dynamic loading and unloading of modules/bundles
• No runtime checks for class-space uniqueness

OSGi (Open Service Gateway Initiative)

Introduced by the OSGI Alliance in November of 1998, the OSGI platform is the most widely used modularity answer to the formal standard question for Java. Currently atrelease 6, the OSGi specification is widely accepted and used, especially of late.

In essence, OSGi is a modular system and a service platform for the Java programming language that implements a complete and dynamic component model in the form of modules, services, deployable bundles, and so on.

The primary layers of the OSGI architecture are as follows:

• Execution environment: The Java environment (for example, Java EE or Java SE) under which a bundle will run.
• Module: Where the OSGi framework processes the modular aspects of a bundle. Bundle metadata is processed here.
• Life-cycle: Initializing, starting, and stopping of bundles happens here.
• Service registry: Where bundles list their services for other bundles to discover.

One of the biggest drawbacks to OSGi is its lack of a formal mechanism for native package installation.

JSR 291

JSR 291 is a dynamic component framework for Java SE that is based on OSGi, is currently in the final stage of development. This effort focuses on taking OSGi into mainstream Java, such as was done for the Java mobile environment by JSR 232.

JSR 294

JSR 294 defines a system of meta-modules and delegates the actual embodiment of pluggable modules (versions, dependencies, restrictions, etc.) to external providers. This specification introduces language extensions, such as "superpackages" and hierarchically-related modules, to facilitate modularity. Strict encapsulation and distinct compilation units are also part of the spec's focus. JSR 294 is currently dormant.

Project Jigsaw

Project Jigsaw is the most likely candidate for modularity in Java 9. Jigsaw seeks to use language constructs and environment configurations to define a scalable module system for Java SE. Primary goals of Jigsaw include:

• Making it very easy to scale the Java SE runtime and the JDK down to small devices.
• Improving the security of Java SE and the JDK by forbidding access to internal JDK APIs and by enforcing and improving the SecurityManager.checkPackageAccessmethod.
• Improving application performance via optimizations of existing code and facilitating look-ahead program optimization techniques.
• Simplifying application development within Java SE by enabling libraries and applications to be constructed from developer-contributed modules and from a modular JDK
• Requiring and enforcing a finite set of version constraints

JEP (Java Enhancement Proposal) 200

Java Enhancement Proposal 200 created in July of 2014, seeks to define a modular structure for the JDK. JEP 200 builds on the Jigsaw framework to facilitate segmenting the JDK, according to Java 8 Compact Profiles, into sets of modules that can be combined at compile time, build time, and deploy time. These combinations of modules can then be deployed as scaled-down runtime environments that are composed of Jigsaw-compliant modules.

JEP 201

JEP 201 seeks to build on Jigsaw to reorganize the JDK source code into modules. These modules can then be compiled as distinct units by an enhanced build system that enforces module boundaries. JEP 201 proposes a source-code restructuring scheme throughout the JDK that emphasizes module boundaries at the top level of source code trees.

Penrose

Penrose would manage interoperability between Jigsaw and OSGi. Specifically, it would facilitate the ability to modify OSGi micro-kernels in order for bundles running in the modified kernel to utilize Jigsaw modules. It relies on using JSON to describe modules.

Plans for Java 9

Java 9 is a unique major release for Java. What makes it unique is its introduction of modular components and segments throughout the entire JDK. The primary features supporting modularization are:

• Modular source code: In Java 9, the JRE and JDK will be reorganized into interoperable modules. This will enable the creation of scalable runtimes that can be executed on small devices.
• Segmented code cache: While not strictly a modular facility, the new segmented code cache of Java 9 will follow the spirit of modularization and enjoy some of the same benefits. The new code cache will make intelligent decisions to compile frequently accessed code segments to native code and store them for optimized lookup and future execution. The heap will also be segmented into 3 distinct units: non-method code that will be stored permanently in the cache; code that has a potentially long lifecycle (known as "non-profiled code"); and code that is transient (known as "profiled code").
• Build-time enforcements: The build system will be enhanced, via JEP 201, to compile and enforce module boundaries.
• Deployment facilities: Tools will be provided within the Jigsaw project that will support module boundaries, constraints, and dependencies at deployment time.

Java 9 early access release

While the exact release date of Java 9 remains a mystery, you can download an early access release at Java.net.

In conclusion

This article has been an overview of modularity within the Java platform, including prospects for modularity in Java 9. I explained how long-standing issues like classpath hell contribute to the need for a more modular Java architecture and discussed some of the most recent new modularity features proposed for Java. I then described and contextualized each of the Java modularity proposals or platforms, including OSGi and Project Jigsaw.

The need for a more modular Java architecture is clear. Current attempts have fallen short, although OSGi comes very close. For the Java 9 release Project Jigsaw and OSGi will be the main players in the modular space for Java, with Penrose possibly providing the glue between them.