managing dependencies with gradle
TRANSCRIPT
HeadlineGradle / Dependencies Managing Dependencies with Gradle
Ads Engineering
Problem 1● Managing dependencies in multi-module projects
TOP LEVELbuild.gradlesettings.gradle
Module Abuild.gradle:compile “netflix.platform:1.2”
Module Bbuild.gradle:compile “netflix.platform:?”
Problem 1 -- cont’d● Managing dependencies in multi-module projects
TOP LEVELbuild.gradlesettings.gradle
Module Abuild.gradle:compile “netflix.platform:1.2”
Module Bbuild.gradle:compile “netflix.platform:1.2”
Problem 1 -- cont’d● To update platform version to 2.3:
TOP LEVELbuild.gradlesettings.gradle
Module Abuild.gradle:compile “netflix.platform:2.3”
Module Bbuild.gradle:compile “netflix.platform:2.3”
● Result: 2 edits …. or N edits!
Problem 1 -- Solution● Use gradle.properties
TOP LEVELbuild.gradlesettings.gradlegradle.properties
Problem 1 -- Solution -- cont’d● What is it?
○ Normal Java properties file -- used to store module versions (amongst others):
gradle.properties…platformVersion=1.2libraryXVersion=latest.release...
● The values from the properties file can be referenced in build.gradle file:
build.gradle…compile “netflix:platform: $platformVersion”compile “some:library: $libraryXVersion”...
● Use the groovy string “ (double quotes not single)
Problem 1 -- Solution -- cont’d● The versions can be referenced in all build.gradle files!
TOP LEVELbuild.gradlesettings.gradlegradle.properties
Module Abuild.gradle:compile “netflix.platform:$platformVersion”
Module Bbuild.gradle:compile “netflix.platform:$platformVersion”
…platformVersion=1.2libraryXVersion=latest.release...
● One centralized place to change version numbers.
Problem 2● Nebula promises repeatable immutable builds● But!
// build.gradle snippet
...
compile “netflix:platform:latest.release”
…
Problem 2 -- cont’dDay 1
platform.versions
● 1.1● 1.2
latest.release -> 1.2
Build includes platform-1.2
Problem 2 -- cont’dDay 1
platform.versions
● 1.1● 1.2
latest.release -> 1.2
Day 2
platform.versions
● 1.1● 1.2● 1.3
latest.release -> 1.3
Build includes platform-1.3
Problem 2 -- cont’dDay 1
platform.versions
● 1.1● 1.2
latest.release -> 1.2
Day 2
platform.versions
● 1.1● 1.2● 1.3
latest.release -> 1.3
Day 3
platform.versions
● 1.1● 1.2● 1.3● 2.0 (breaks binary compatibility)
latest.release -> 2.0
Build includes platform-2.0 (and fails!)
Problem 2 -- One Solution● Pin version down
// build.gradle snippet
...
compile “netflix:platform:1.2”
…
Problem 2 -- One Solution -- cont’d● Pin version down
// build.gradle snippet
...
compile “netflix:platform:1.2”
…
Problem: Have to manually update versions now every time there is a new release. (Tedious and error-prone.)
Problem 2 -- nebula-dependency-lock Gradle Plugin● Part of the Nebula gradle plugins family: https://github.com/nebula-
plugins/gradle-dependency-lock-plugin
A plugin to allow people using dynamic dependency versions to lock them to specific versions.
● We can still use “latest.release” as the version number, but decide when the version gets incremented, regardless of the versions of the components available in the repository
● How?● Creates a (JSON) “lock” file in the project directory with the current version
numbers. ● Lock file does NOT get updated during the normal build process -- so
versions are “locked” until the lock file is updated● Provides Gradle tasks to update the lock file● Committing the “lock” file into SCM (git/stash/etc) means building from the
commit (hash) at any time will use the same versions always
nebula-dependency-lock Gradle Plugin -- Cont’d
● Usage: simply reference the plugin in the build.gradle:
apply plugin: 'nebula.dependency-lock'
● Create a lock file:
gradle generateLock saveLock
Or (for multi-module projects)
gradle generateGlobalLock saveGlobalLock
(in root project)
nebula-dependency-lock Gradle Plugin -- Cont’d
● To update dependency graph (i.e. when new library gets added to dependencies) -- but not update the versions!
gradle updateLock saveLock
Or
gradle updateGlobalLock saveGlobalLock
● In fact generateLock/updateLock and generateGlobalLock/updateGlobalLock are equivalent so they can be interchanged
○ Same command can be used in both cases
nebula-dependency-lock Gradle Plugin -- Cont’d
● To update versions
gradle updateLock saveLock --refresh-dependencies
Or
gradle updateGlobalLock saveGlobalLock --refresh-dependencies
nebula-dependency-lock Gradle Plugin -- Cont’d
● More goodness: plugging in nebula gradle-scm-plugins● What is it
○ Suite of Nebula plugins for interfacing with SCM (git/stash/etc)○ On Github: https://github.com/nebula-plugins/gradle-scm-plugin
● Specialized plugins for each SCM○ gradle-git-scm-plugin is the plugin for Stash/Git○ On Github: https://github.com/nebula-plugins/gradle-git-scm-plugin
● Creates tasks for committing from build.gradle into Stash/Git
nebula-dependency-lock Gradle Plugin -- Cont’d
● When used in the same project with nebula-dependency-lock, a commitLock task is created:
○ Commits the dependency “lock” file into SCM○ For git/stash it does a commit + push (sync local/remote repos)
● Following updates the lock file and pushes it to the remote repository:
gradle updateLock saveLock commitLock --refresh-dependencies
Or
gradle updateGlobalLock saveGlobalLock commitLock --refresh-dependencies
(Note the name of task is commitLock for both types of projects!)
nebula-dependency-lock Gradle Plugin -- Cont’d
Automatic nightly checked dependencies version upgrade:
● Everyone commits into master (assume we commit just code -- not update dependencies too)
● Nightly, Jenkins job to:a. gradle updateLock saveLockb. gradle build test integrationTestc. gradle commitLock
● Every morning the lock file will contain the latest versions which don’t break the project!
■ Or if one of the new versions causes issues then you get notified by Jenkins!
nebula-dependency-lock Gradle Plugin -- Cont’d
● Multi-module or separate modules?
Problem 3
Module A Module B
libX:1.0
libY:2.0
libZ:3.0
Module A: latest
libX:1.0
libY:2.0
TOP LEVEL
Module A
libX:1.0
libY:2.0
Module B
libZ:3.0
Module A
Problem 3 -- cont’d
Module A Module B
libX:1.0
libY:2.0
libZ:3.0
Module A: latest
libX:1.0
libY:2.0
TOP LEVEL
Module A
libX:1.0
libY:2.0
Module B
libZ:3.0
Module AOwn RepoOwn Jenkins Job
Own RepoOwn Jenkins Job
One RepoOne Jenkins Job
Problem 3 -- cont’d
Module A Module B
libX:1.0
libY:latest
libZ:3.0
Module A: latest
libX:1.0
libY:2.0
● Dependencies Update -- separate modules
libY:● 2.0● 2.1
libY:2.1
Artifactory
Module A: 1.1
Problem 3 -- cont’d
Module A Module B
libX:1.0
libY:latest
libZ:3.0
Module A: 1.1
libX:1.0
libY:2.1
● Dependencies Update -- separate modules
libY:● 2.0● 2.1
libY:2.1
Artifactory
Module A: 1.1
Problem 3 -- cont’d
Module A Module B
libX:1.0
libY:latest
libZ:3.0
Module A: 1.1
libX:1.0
libY:2.1
● Dependencies Update -- separate modules
libY:● 2.0● 2.1
libY:2.1
Artifactory
Module A: 1.1
CONFLICT!(Only visible when Module B gets compiled)
Problem 3 -- cont’dSolutions for conflict (separate modules):
● Go back to Module A and pin libY to version to 2.0○ Requires changes in A + rebuild A
● Change Module B and force pin libY to version 2.0○ Simply pin to 2.0 won’t work because Module A drags a new version (2.1)
○ Now Module A and B use different versions of libY (so any project using both of them will have to force pin libY)
● Change Module B to exclude libY when pulling Module A○ Will use whatever version Module B has for libY○ Again, Module A and B use different versions
Problem 3 -- cont’d
Module B
libZ:3.0
Module A: latest
libX:1.0
libY:latest
● Dependencies Update -- multi-modules
libY:● 2.0● 2.1
Artifactory
libY:2.1
Problem 3 -- cont’d
Module B
libZ:3.0
Module A: latest
libX:1.0
libY:latest
● Dependencies Update -- multi-modules
libY:● 2.0● 2.1
Artifactory
libY:2.1
CONFLICT!(Visible right away)
Problem 3 -- cont’dSolutions for conflict (multi-module):
● Pin libY to version to 2.0○ Requires one single change (in gradle.properties)
● Use dependency locking○ The nightly build “catches” the incompatibility with 2.1 and doesn’t upgrade dependencies
Questions
?