tdc2016poa | trilha cloud computing - source-to-image - how to transform any application code into...

Red Hat Forum Brasil 2016

Source to Image - builds reproduzíveis

Filippe Spolti

Software Engineer

@filippespolti


Developer Workflow

IDE

Git Repo/Local source

Builder Image

Build

Docker Registry

Invoke Build Commit App Image

Push

RunRunning Docker

Container


Project Goals

• Ecosystem• Any existing Docker image should be easily adaptable to S2I• Create an ecosystem of reusable builder images

• Ease of Use• End users don’t need to learn/write Dockerfiles• Enable existing source build tools, e.g. maven, rake, etc

• Reproducibility• Minimize inputs into the build process (just source)

• Speed• Perform build operations as a single layer/single commit

• Security• Run the assemble process as a non-privileged user


Demo


S2I vs Docker build

• S2I can enforce non-root permissions• S2I encourages fast builds by separating dependencies into parent

layers• Use case is targeted at anything that needs to build from source but

the layering approach can help create the desired ecosystem in a lot of scenarios.

• Reduces number of layers committed during build and in final image• Typical use case is to build parent layers with dockerfiles and final layer

with S2IRHEL7 + JBoss + My App DependenciesRHEL7 + JBoss Dockerbuild

RHEL7 + JBoss + My App Dependencies + My WAR

S2I


Anatomy of an S2I Builder Image● Base language runtime+build tools

● Assemble script

○ Converts source code into a runnable application

○ Downloads dependencies

○ Injects configuration

● Run script

○ Responsible for starting the application in the final image

○ S2I will set this as the RUN command in the final image

● Save-artifacts script

○ Optional: extracts reusable dependencies from previous application image


S2I Build Steps1. Create a container based on the build image and pass it a tar file that contains:

a. The application source in src

b. The build artifacts in artifacts (incremental builds)

2. Set the environment variables from .s2i/environment (optional)

3. Start the container and run its assemble script

4. Wait for the container to finish

5. Commit the container, setting the CMD for the output image to be the run script and tagging

the image with the name provided.


Sample Assemble Script

#!/bin/sh

echo "---> Installing application source"cp -Rf /tmp/src/. ./

# Change the npm registry mirror if providedif [ -n "$NPM_MIRROR" ]; then

npm config set registry $NPM_MIRRORfi

echo "---> Building your Node application from source"npm install -d


Sample Run Script

#!/bin/sh

echo "Launching via npm..."exec npm run -d $NPM_RUN


Source can be more than “source”● Framework configuration

○ Php.ini, EAP/Wildfly standalone.xml, etc

○ Baked in config can be better than env variable config

● Database images

○ Configuration for the DB

○ Schema initialization scripts

○ NOT: actual DB content (do not bake into the image)

● Jenkins

○ Plugins (actual binaries or list of plugins to install)

○ General Jenkins configuration

○ Jenkins job definitions


S2I Incremental BuildsWhen s2i detects:

● A builder image is compatible with incremental building (save-artifacts exists)

● A previous image exists, with the same name as the output name for this build

● Incremental build is enabled

An incremental build is performed:

1. Creates a new Docker container from the prior build image

2. Run save-artifacts in this container (streams out a tar of the artifacts to stdout)

3. Build the new output image:

i. The artifacts from the previous build will be in the artifacts directory of the tar passed to the buildii. The build image's assemble script is responsible for detecting and using the build artifacts


Demo


S2I Layered BuildsWhen s2i detects:

● Builder image does not contain ‘sh’ or ‘tar’ binaries

An intermediate layered build is performed:

1. Generate a Dockerfile FROM the builder image

2. ADD the local source working directory

3. Docker build an intermediate image

4. S2I build using the resulting image (contains input files already)

Caution: will invoke ONBUILD commands from the builder image


S2I Extended Builds (Tech Preview)

Source

Builder Image

Build

Docker Registry

Start Build CommitArtifact Image

Push

R

un

Running Docker

Container

BuildExtract

Runtime Image

CommitApplication

Image


Options● Assemble/Run/Save-artifacts script overriding

○ Via source repo or http url

● Pull policies

○ Never/Always/IfNotPresent

● Environment variables

○ Via environment file in source repo or command line args

○ Used during assemble and included in final image

● Incremental build enabled/disabled

● Inject files

○ Make local files available during assemble

○ File contents will not be included in the final application image


Future● Fit and finish on Extended Builds

● Ability to override part of assemble/run script

○ Hook points (pre-assemble/post-assemble, pre-run, etc)

● Remove git support?

○ Make s2i purely a local filesystem tool, use your own repo tools

● Abstract the Docker layer

○ Support other container platforms


Resources● Source-To-Image Repository - https://github.com/openshift/source-to-image

● Community images - https://github.com/openshift-s2i/

● SCL/OpenShift images - https://github.com/sclorg/s2i-xxxx-container

○ nodejs, perl, php, python, ruby

● Jenkins image - https://github.com/openshift/jenkins

https://github.com/openshift/source-to-image

https://github.com/openshift-s2i/

https://github.com/sclorg/s2i-xxxx-container

https://github.com/openshift/jenkins

tdc2016poa | trilha cloud computing - source-to-image - how to transform any application code into...

Education