Java modules

Java modules

• 1.
  Java Modules (why yourstuff stopped working) @bbdsoftware @rorypreddy
• 2.
  Meet Rory • Developerin R&D team at BBD • Organizer for the Java and AWS User groups • Not related to Tyrion Lannister • Don’t do kids parties @rorypreddy
• 5.
  Agenda • Introduction • Modules101 • Jlink • Demo • Migrating to Java 9+
• 7.
  common/hadoop-common-3.0.0-SNAPSHOT.jar:common/hadoop-nfs-3.0.0-SNAPSHOT.jar:common/lib/activation-1.1.jar:common/lib/apacheds-i 18n-2.0.0-M15.jar:common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:common/lib/api-asn1-api-1.0.0-M20.jar:common/lib/api-util-1.0 .0-M20.jar:common/lib/asm-3.2.jar:common/lib/avro-1.7.4.jar:common/lib/commons-beanutils-1.7.0.jar:common/lib/commons-beanutils- core-1.8.0.jar:common/lib/commons-cli-1.2.jar:common/lib/commons-codec-1.4.jar:common/lib/commons-collections-3.2.1.jar:common/l ib/commons-compress-1.4.1.jar:common/lib/commons-configuration-1.6.jar:common/lib/commons-digester-1.8.jar:common/lib/commons-ht tpclient-3.1.jar:common/lib/commons-io-2.4.jar:common/lib/commons-lang-2.6.jar:common/lib/commons-logging-1.1.3.jar:common/lib/c ommons-math3-3.1.1.jar:common/lib/commons-net-3.1.jar:common/lib/curator-client-2.7.1.jar:common/lib/curator-framework-2.7.1.jar :common/lib/curator-recipes-2.7.1.jar:common/lib/gson-2.2.4.jar:common/lib/guava-11.0.2.jar:common/lib/hadoop-annotations-3.0.0- SNAPSHOT.jar:common/lib/hadoop-auth-3.0.0-SNAPSHOT.jar:common/lib/hamcrest-core-1.3.jar:common/lib/htrace-core4-4.0.1-incubating .jar:common/lib/httpclient-4.2.5.jar:common/lib/httpcore-4.2.5.jar:common/lib/jackson-core-asl-1.9.13.jar:common/lib/jackson-jax rs-1.9.13.jar:common/lib/jackson-mapper-asl-1.9.13.jar:common/lib/jackson-xc-1.9.13.jar:common/lib/java-xmlbuilder-0.4.jar:commo n/lib/jaxb-api-2.2.2.jar:common/lib/jaxb-impl-2.2.3-1.jar:common/lib/jcip-annotations-1.0.jar:common/lib/jersey-core-1.9.jar:com mon/lib/jersey-json-1.9.jar:common/lib/jersey-server-1.9.jar:common/lib/jets3t-0.9.0.jar:common/lib/jettison-1.1.jar:common/lib/ jetty-6.1.26.jar:common/lib/jetty-util-6.1.26.jar:common/lib/jsch-0.1.51.jar:common/lib/json-smart-1.1.1.jar:common/lib/jsp-api- 2.1.jar:common/lib/jsr305-3.0.0.jar:common/lib/junit-4.11.jar:common/lib/log4j-1.2.17.jar:common/lib/mockito-all-1.8.5.jar:commo n/lib/netty-3.6.2.Final.jar:common/lib/nimbus-jose-jwt-3.9.jar:common/lib/paranamer-2.3.jar:common/lib/protobuf-java-2.5.0.jar:c ommon/lib/servlet-api-2.5.jar:common/lib/slf4j-api-1.7.10.jar:common/lib/slf4j-log4j12-1.7.10.jar:common/lib/snappy-java-1.0.4.1 .jar:common/lib/stax-api-1.0-2.jar:common/lib/xmlenc-0.52.jar:common/lib/xz-1.0.jar:common/lib/zookeeper-3.4.6.jar:hdfs/hadoop-h dfs-3.0.0-SNAPSHOT.jar:hdfs/hadoop-hdfs-nfs-3.0.0-SNAPSHOT.jar:hdfs/lib/commons-daemon-1.0.13.jar:hdfs/lib/hadoop-hdfs-client-3. 0.0-SNAPSHOT.jar:hdfs/lib/hpack-0.11.0.jar:hdfs/lib/leveldbjni-all-1.8.jar:hdfs/lib/netty-all-4.1.0.Beta5.jar:hdfs/lib/okhttp-2. 4.0.jar:hdfs/lib/okio-1.4.0.jar:hdfs/lib/xercesImpl-2.9.1.jar:mapreduce/hadoop-mapreduce-client-app-3.0.0-SNAPSHOT.jar:mapreduce /hadoop-mapreduce-client-common-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-mapreduce-client-core-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-ma preduce-client-hs-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-mapreduce-client-hs-plugins-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-mapreduce- client-jobclient-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-mapreduce-client-nativetask-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-mapreduce-c lient-shuffle-3.0.0-SNAPSHOT.jar:mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-api-3.0.0-SNAPSHOT.jar: yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.0.0-SNAP SHOT.jar:yarn/hadoop-yarn-client-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-common-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-registry-3.0.0-S NAPSHOT.jar:yarn/hadoop-yarn-server-applicationhistoryservice-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-server-common-3.0.0-SNAPSHOT.j ar:yarn/hadoop-yarn-server-nodemanager-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-server-resourcemanager-3.0.0-SNAPSHOT.jar:yarn/hadoop -yarn-server-sharedcachemanager-3.0.0-SNAPSHOT.jar:yarn/hadoop-yarn-server-web-proxy-3.0.0-SNAPSHOT.jar:yarn/lib/aopalliance-1.0 .jar:yarn/lib/commons-math-2.2.jar:yarn/lib/curator-test-2.7.1.jar:yarn/lib/fst-2.24.jar:yarn/lib/guice-3.0.jar:yarn/lib/guice-s ervlet-3.0.jar:yarn/lib/javassist-3.18.1-GA.jar:yarn/lib/javax.inject-1.jar:yarn/lib/jersey-client-1.9.jar:yarn/lib/jersey-guice -1.9.jar:yarn/lib/objenesis-2.1.jar Classpathhell
• 8.
  Runtime Compile time
• 10.
  JigSaw 2011 JSR-294 Improved Modularity Support 2006 Java 7: No JigSaw Java8: No JigSaw JSR-376 Java Platform Module System JSR-277 Java Module System 2005 JSR-291 'OSGi 4.1' 2014
• 11.
  Java Platform ModuleSystem
• 12.
  Why Modules? • Reliableconfiguration + • Strong encapsulation • Leads to: • Scalable Java platform • Greater platform integrity • Improved performance -Compile + Runtime -Cleaner, more logical designs -95 Modules -sun.misc.Unsafe -Lazy loading
• 13.
  Modules 101 “Programming isbreaking of one big impossible task into several very small possible tasks”. -Jazzwant
• 14.
  Define a Module -Its name - The packages it makes available publicly - The modules it depends on module name exports requires module-info.java
• 15.
  Some Rules • Modulesare eagerly loaded • Dependency graph is built and checked on startup • Application won’t start if modules are missing • Module rules enforced when jar on module-path • Rules not enforced when same jar on class-path • Module names must be unique • Names should follow reverse domain name pattern • No support for versioning • Only one version of a module may be loaded • Tools (Maven) maybe used to load different versions of a module
• 18.
  Example – com.coffee com.coffee | ---com ---coffee |Coffee.class | CoffeeMaker.class | ---types Arabica.class Robusta.class
• 19.
  Example – com.coffee com.coffee |module-info.class | ---com ---coffee | Coffee.class | CoffeeMaker.class | ---types Arabica.class Robusta.class module com.coffee { requires java.logging; exports com.coffee; }
• 21.
  Example – com.app com.app | ---com ---app CoffeeApp.class TeaApp.class
• 22.
  Example – com.app com.app |module-info.class | ---com ---app CoffeeApp.class TeaApp.class module com.app { requires com.coffee; requires com.tea; requires java.sql; requires java.logging; }
• 23.
  Readability And Accessibility Writecode that is easy to delete, not easy to extend. - Tef, Programming is Terrible
• 24.
  Readability - DirectReadability com.app com.coffee com.tea java.logging java.base java.sql java.xml
• 25.
  Readability - ImpliedReadability module java.sql { requires transitive java.logging; requires transitive java.xml; exports java.sql; exports javax.sql; exports javax.transaction.xa; } String url = ...; Properties props = ...; Driver d = DriverManager.getDriver(url); Connection c = d.connect(url, props); d.getParentLogger().info("Connection acquired"); com.app com.coffee com.tea java.logging java.base java.sql java.xml
• 26.
  Accessibility
• 27.
  • Other modulescan only see and use code if: 1. public 2. package is exported by target module 3. package is readable from this module • Combination referred to as "accessibility" Accessibility
• 28.
  package com.app; import com.coffee.Coffee; importcom.coffee.CoffeeMaker; import com.coffee.types.Arabica; import … public class CoffeeApp {… The type com.coffee.types.Arabica is not accessible module com.app { requires com.coffee; requires com.tea; requires com.coffee.types; } com.coffee.types cannot be resolved to a module Accessibility
• 29.
  Unnamed And AutomaticModules “Programs must be written for people to read, and only incidentally for machines to execute”. -Harold Abelson
• 30.
  Unnamed Module -Used for -Java8 and older versions -Small or temporary applications - Added via the class path - The unnamed module reads code in any other module - The unnamed module exports all of its packages. >jdeps minecraft1.12.jar Warning: split package: javax.annotationjrt:/java.xml.ws.annotation server.jar com.google.gson-> java.sql <unnamed> -> com.google.common.base ….
• 31.
  Automatic Modules • JARfrom the class path that has been put in the module path. • Don’t wait for third party libraries to become modularized. • Becomes a named module with • Automatic-Module-Name in Manifest.mf • Otherwise, a name is derived from the JAR filename • Reads every other named module. • Exports all of its packages. • It guarantees direct and implied readability to all other modules.
• 32.
  Summary of Readability ModuleType Origin Export Packages Read Modules Named Platform Provided by platform Explicitly NamedApplication All JARS containing module-info on the module path Explicitly • Platform • Application • Automatic • Unnamed Automatic All JARs without module-info but on module path All • Platform • Application • Automatic • Unnamed Unnamed Classpath All • Platform • Automatic • Application
• 33.
  Cloud-native is anapproachto building and running applicationsthat exploits the advantages of the cloud computing delivery model. Source: https://pivotal.io/cloud-native jlink
• 34.
  jlink jlink java.sql java.xml java.base java.logging com.app Modular run-time image …confbin jmods
• 35.
  jlink $ jlink --module-path$JDKMODS:$MYMODS --add-modules com.app –-output myimage $ myimage/bin/java –-list-modules com.app com.coffee@1.0 com.tea@1.0 java.base@10 java.sql@10 java.logging@10 27 MBs vs 226 MBs
• 36.
  Demo http://github.com/roryp/spring-maven-modules
• 37.
  Migrating to Java9
• 38.
  Migrating to Java9+ module java.base module java.sql module java.logging module java.xml myapp.jar mylib.jar lib1.jar otherapp.jar lib2.jar Bottom Up Library Migration Top Down App MigrationYour app Libraries and Third party apps JDK
• 39.
  module java.base module java.sql module java.logging module java.xml module myapp module mylib lib1.jar otherapp.jar lib2.jar TopDown App MigrationYour app Libraries and Third party apps JDK AUTOMATIC MODULE Migrating to Java 9+ (Migrating your application and using “not migrated” libraries that are still residing on the classpath)
• 40.
  Bottom Up Library Migration module java.base module java.sql module java.logging module java.xml myapp.jarmylib.jar module lib1 module otherapp module lib2 Your app Libraries and Third party apps JDK UNNAMED MODULE Migrating to Java 9+ (Migrating the libraries first and using migrated modularized libraries while your app is still on the class path)
• 41.
  HOW TO MODULARIZE •Introduce a module-info.java file for each module • A module-info.java file can be: • Created manually by the developers • Automatically generated using the JDeps tool with the option --generate-module-info
• 43.
  MODULARIZE - Issues 1)Encapsulated JDK internal APIs 2) Not resolved modules 3) Cyclic dependencies 4) Split packages
• 44.
  ENCAPSULATED JDK INTERNALAPI • Most of the internal JDK APIs are inaccessible in Java 9 • Trying to access them causes a compilation error • Internal APIs are in the sun.* package are part of the jdk.unsupported module . Not java.base • The module jdk.unsupported provides: • sun.misc.Unsafe, • sun.reflect.Reflection, • …
• 45.
  batik-codec.jar -> JDKremoved internal API JDK Internal API Suggested Replacement ---------------- --------------------- com.sun.image.codec.jpeg.JPEGCodec Use javax.imageio @since 1.4 com.sun.image.codec.jpeg.JPEGDecodeParam Use javax.imageio @since 1.4 ENCAPSULATED JDK INTERNAL APIS • JDeps can report internal dependencies $ jdeps --jdk-internals 'lib/batik/batik-codec.jar'
• 46.
  Solve the problemof JDK internal APIs: • Replace each of your JDK internal APIs calls with supported APIs. OR • Break the encapsulation with --add-exports --add-exports java.base/sun.net=ALL-UNNAMED
• 47.
  NOT RESOLVED PLATFORMMODULES • From jdk-9+118 running code on the class path , these types will not be visible by default: • java.activation • java.annotations.common • java.corba • java.transaction • java.xml.bind • java.xml.ws
• 48.
  NOT RESOLVED PLATFORMMODULES • Make that module available for compilationwith --add-modules: >Javac --add-modules java.xml.bind • And again for execution: >java --add-modules java.xml.bind
• 49.
  CYCLIC DEPENDENCIES • Forbiddenat compile-time (between modules) • JPMS does not allow cycles in the “requires” clauses • JPMS allows cycles in the “reads” relation of run-time modules module A { requires B; } module B { requires A; } • One solution is to merge the JARs into a single module
• 50.
  SPLIT PACKAGES • Modulesthat contain packages having the same name must not interfere with each other • Solution for JAR files: 1. Create a single JAR file out of the two JAR files 2. Rename one of the package • Solution for modules: 1. Create a single module out of two or more modules 2. Create a third module 3. Remove the package dependencies
• 51.
  Looking Ahead
• 53.
  Do not adda module-info.java module declaration until: 1. All of your runtime dependencies have been modularised • (either as a full module or with a MANIFEST.MF entry) 2. All those modularised dependencies have been released to Maven Central 3. Your library depends on the updated versions If you can't meet these 3 criteria • Add a MANIFEST.MF entry following the agreed module naming conventions Module Hell If everyone does this we stand a reasonable chance of avoiding Module Hell
• 54.
  Questions? • http://github.com/roryp/javamodules • http://github.com/roryp/spring-maven-modules@bbdsoftware @rorypreddy