Deploying on OpenShift

This guide covers generating and deploying OpenShift resources based on sane default and user supplied configuration.

Prerequisites

To complete this guide, you need:

  • roughly 5 minutes

  • an IDE

  • JDK 11+ installed with JAVA_HOME configured appropriately

  • Apache Maven 3.8.1+

  • access to an OpenShift cluster (Minishift is a viable option)

  • OpenShift CLI (Optional, only required for manual deployment)

Bootstrapping the project

First, we need a new project that contains the OpenShift extension. This can be done using the following command:

mvn io.quarkus.platform:quarkus-maven-plugin:2.5.1.Final:create \
    -DprojectGroupId=org.acme \
    -DprojectArtifactId=openshift-quickstart \
    -DclassName="org.acme.rest.GreetingResource" \
    -Dpath="/greeting" \
    -Dextensions="resteasy,openshift"

cd openshift-quickstart

Quarkus offers the ability to automatically generate OpenShift resources based on sane defaults and user supplied configuration. The OpenShift extension is actually a wrapper extension that brings together the kubernetes and container-image-s2i extensions with sensible defaults so that it’s easier for the user to get started with Quarkus on OpenShift.

When we added the OpenShift extension to the command line invocation above, the following dependency was added to the pom.xml

    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-openshift</artifactId>
    </dependency>

Log Into the OpenShift Cluster

Before we build and deploy our application we need to log into an OpenShift cluster. You can log in via the OpenShift CLI:

Log In - OpenShift CLI Example
oc login -u myUsername (1)
1 You’ll be prompted for the required information such as server URL, password, etc.

Alternatively, you may log in using the API token:

Log In - OpenShift CLI With API Token Example
oc login --token=myToken --server=myServerUrl
You can request the token via the Copy Login Command link in the OpenShift web console.

Finally, you don’t need to use the OpenShift CLI at all. Instead, set the quarkus.kubernetes-client.master-url config property and authenticate with the quarkus.kubernetes-client.token, or quarkus.kubernetes-client.username and quarkus.kubernetes-client.password respectively.

Log In - Configuration Properties Example
./mvnw clean package -Dquarkus.kubernetes-client.master-url=myServerUrl -Dquarkus.kubernetes-client.token=myToken

Build and Deployment

You can trigger a build and deployment in a single step or build the container image first and then configure the OpenShift application manually if you need more control over the deployment configuration.

To trigger a build and deployment in a single step:

./mvnw clean package -Dquarkus.kubernetes.deploy=true

This command will build your application locally, then trigger a container image build and finally apply the generated OpenShift resources automatically. The generated resources use OpenShift’s DeploymentConfig that is configured to automatically trigger a redeployment when a change in the ImageStream is noticed. In other words, any container image build after the initial deployment will automatically trigger redeployment, without the need to delete, update or re-apply the generated resources.

To confirm the above command has created an image stream, a service resource and has deployed the application (has a pod running), apply these commands:

oc get is (1)
oc get pods (2)
oc get svc (3)
1 Lists the image streams created.
2 Get the list of pods.
3 Get the list of Kubernetes services.

By default, the service is not exposed to the outside world. To expose the created service to a route and test it:

oc expose svc/greeting (1)
oc get routes (2)
curl http://<route>/greeting (3)
1 Expose the service.
2 Get the list of exposed routes.
3 Access your application.

Configure the OpenShift Application Manually

If you need more control over the deployment configuration you can build the container image first and then configure the OpenShift application manually.

To trigger a container image build:

./mvnw clean package -Dquarkus.container-image.build=true

The build that will be performed is an s2i binary build. The input of the build is the jar that has been built locally and the output of the build is an ImageStream that is configured to automatically trigger a deployment.

During the build you may find the Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed exception due to self-signed certificate. To solve this, just add the following line to your application.properties:

quarkus.kubernetes-client.trust-certs=true

For more information, see deploying to kubernetes.

Once the build is done we can create a new application from the relevant ImageStream.

oc get is (1)
oc new-app --name=greeting <project>/openshift-quickstart:1.0.0-SNAPSHOT (2)
oc get svc
oc expose svc/greeting (3)
oc get routes (4)
curl http://<route>/greeting (5)
1 Lists the image streams created. The image stream of our application should be tagged as <project>/openshift-quickstart:1.0.0-SNAPSHOT.
2 Create a new application from the image source.
3 Expose the service to the outside world.
4 Get the list of exposed routes.
5 Access your application.

After this setup the next time the container image is built a deployment to OpenShift is triggered automatically. In other words, you don’t need to repeat the above steps.

Non-S2I Builds

Out of the box the OpenShift extension is configured to use container-image-s2i. However, it’s still possible to use other container image extensions like:

When a non-s2i container image extension is used, an ImageStream is created that is pointing to an external dockerImageRepository. The image is built and pushed to the registry and the ImageStream populates the tags that are available in the dockerImageRepository.

To select which extension will be used for building the image:

quarkus.container-image.builder=docker

or

quarkus.container-image.builder=jib

Customizing

All available customization options are available in the OpenShift configuration options.

Some examples are provided in the sections below:

Exposing Routes

To expose a Route for the Quarkus application:

quarkus.openshift.route.expose=true

You don’t necessarily need to add this property in the application.properties. You can pass it as a command line argument:

./mvnw clean package -Dquarkus.openshift.route.expose=true

The same applies to all properties listed below.

Labels

To add a label in the generated resources:

quarkus.openshift.labels.foo=bar

Annotations

To add an annotation in the generated resources:

quarkus.openshift.annotations.foo=bar

Environment variables

OpenShift provides multiple ways of defining environment variables:

  • key/value pairs

  • import all values from a Secret or ConfigMap

  • interpolate a single value identified by a given field in a Secret or ConfigMap

  • interpolate a value from a field within the same resource

Environment variables from key/value pairs

To add a key/value pair as an environment variable in the generated resources:

quarkus.openshift.env.vars.my-env-var=foobar

The command above will add MY_ENV_VAR=foobar as an environment variable. Please note that the key my-env-var will be converted to uppercase and dashes will be replaced by underscores resulting in MY_ENV_VAR.

Environment variables from Secret

To add all key/value pairs of Secret as environment variables just apply the following configuration, separating each Secret to be used as source by a comma (,):

quarkus.openshift.env.secrets=my-secret,my-other-secret

which would generate the following in the container definition:

envFrom:
  - secretRef:
      name: my-secret
      optional: false
  - secretRef:
      name: my-other-secret
      optional: false

The following extracts a value identified by the keyName field from the my-secret Secret into a foo environment variable:

quarkus.openshift.env.mapping.foo.from-secret=my-secret
quarkus.openshift.env.mapping.foo.with-key=keyName

This would generate the following in the env section of your container:

- env:
  - name: FOO
    valueFrom:
      secretKeyRef:
        key: keyName
        name: my-secret
        optional: false

Environment variables from ConfigMap

To add all key/value pairs from ConfigMap as environment variables just apply the following configuration, separating each ConfigMap to be used as source by a comma (,):

quarkus.openshift.env.configmaps=my-config-map,another-config-map

which would generate the following in the container definition:

envFrom:
  - configMapRef:
      name: my-config-map
      optional: false
  - configMapRef:
      name: another-config-map
      optional: false

The following extracts a value identified by the keyName field from the my-config-map ConfigMap into a foo environment variable:

quarkus.openshift.env.mapping.foo.from-configmap=my-configmap
quarkus.openshift.env.mapping.foo.with-key=keyName

This would generate the following in the env section of your container:

- env:
  - name: FOO
    valueFrom:
      configMapRefKey:
        key: keyName
        name: my-configmap
        optional: false

Environment variables from fields

It’s also possible to use the value from another field to add a new environment variable by specifying the path of the field to be used as a source, as follows:

quarkus.openshift.env.fields.foo=metadata.name

Using Deployment instead of DeploymentConfig

Out of the box the extension will generate a DeploymentConfig resource. Often users, prefer to use Deployment as the main deployment resource, but still make use of Openshift specific resources like Route, BuildConfig etc. This feature is enabled by setting quarkus.openshift.deployment-kind to Deployment.

quarkus.openshift.deployment-kind=Deployment

Since Deployment is a Kubernetes resource and not Openshift specific, it can’t possibly leverage ImageStream resources, as is the case with DeploymentConfig. This means that the image references need to include the container image registry that hosts the image. When the image is built, using Openshift builds (s2i binary and docker strategy) the Openshift internral image registry image-registry.openshift-image-registry.svc:5000 will be used, unless an other registry has been explicitly specified by the user. Please note, that in the internal registry the project/namespace name is added as part of the image repository: image-registry.openshift-image-registry.svc:5000/<project name>/<name>:<tag>, so users will need to make sure that the target project/namespace name is aligned with the quarkus.container-image.group.

quarkus.container-image.group=<project/namespace name>

Validation

A conflict between two definitions, e.g. mistakenly assigning both a value and specifying that a variable is derived from a field, will result in an error being thrown at build time so that you get the opportunity to fix the issue before you deploy your application to your cluster where it might be more difficult to diagnose the source of the issue.

Similarly, two redundant definitions, e.g. defining an injection from the same secret twice, will not cause an issue but will indeed report a warning to let you know that you might not have intended to duplicate that definition.

Backwards compatibility

Previous versions of the OpenShift extension supported a different syntax to add environment variables. The older syntax is still supported but is deprecated, and it’s advised that you migrate to the new syntax.

Table 1. Old vs. new syntax

Old

New

Plain variable

quarkus.openshift.env-vars.my-env-var.value=foobar

quarkus.openshift.env.vars.my-env-var=foobar

From field

quarkus.openshift.env-vars.my-env-var.field=foobar

quarkus.openshift.env.fields.my-env-var=foobar

All from ConfigMap

quarkus.openshift.env-vars.xxx.configmap=foobar

quarkus.openshift.env.configmaps=foobar

All from Secret

quarkus.openshift.env-vars.xxx.secret=foobar

quarkus.openshift.env.secrets=foobar

From one Secret field

quarkus.openshift.env-vars.foo.secret=foobar

quarkus.openshift.env.mapping.foo.from-secret=foobar

quarkus.openshift.env-vars.foo.value=field

quarkus.openshift.env.mapping.foo.with-key=field

From one ConfigMap field

quarkus.openshift.env-vars.foo.configmap=foobar

quarkus.openshift.env.mapping.foo.from-configmap=foobar

quarkus.openshift.env-vars.foo.value=field

quarkus.openshift.env.mapping.foo.with-key=field

If you redefine the same variable using the new syntax while keeping the old syntax, ONLY the new version will be kept, and a warning will be issued to alert you of the problem. For example, if you define both quarkus.openshift.env-vars.my-env-var.value=foobar and quarkus.openshift.env.vars.my-env-var=newValue, the extension will only generate an environment variable MY_ENV_VAR=newValue and issue a warning.

Mounting volumes

The OpenShift extension allows the user to configure both volumes and mounts for the application.

Any volume can be mounted with a simple configuration:

quarkus.openshift.mounts.my-volume.path=/where/to/mount

This will add a mount to my pod for volume my-volume to path /where/to/mount

The volumes themselves can be configured as shown in the sections below:

Secret volumes

quarkus.openshift.secret-volumes.my-volume.secret-name=my-secret

ConfigMap volumes

quarkus.openshift.config-map-volumes.my-volume.config-map-name=my-secret

Persistent Volume Claims

quarkus.openshift.pvc-volumes.my-pvc.claim-name=my-pvc

Knative - OpenShift Serverless

OpenShift also provides the ability to use Knative via the OpenShift Serverless functionality.

The first order of business is to instruct Quarkus to generate Knative resources by setting:

quarkus.kubernetes.deployment-target=knative

In order to leverage OpenShift S2I to build the container image on the cluster and use the resulting container image for the Knative application, we need to set a couple of configuration properties:

# set the Kubernetes namespace which will be used to run the application
quarkus.container-image.group=geoand
# set the container image registry - this is the standard URL used to refer to the internal OpenShift registry
quarkus.container-image.registry=image-registry.openshift-image-registry.svc:5000

The application can then be deployed to OpenShift Serverless by enabling the standard quarkus.kubernetes.deploy=true property.

Configuration Reference

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

The OpenShift flavor / version to use. Older versions of OpenShift have minor differrences in the labels and fields they support. This option allows users to have their manifests automatically aligned to the OpenShift 'flavor' they use.

v3, v4

v4

The kind of the deployment resource to use. Supported values are 'Deployment' and 'DeploymentConfig' defaulting to the later.

deployment, deployment-config

deployment-config

The name of the group this component belongs too

string

The name of the application. This value will be used for naming Kubernetes resources like: - Deployment - Service and so on …​

string

${quarkus.container-image.name}

The version of the application.

string

${quarkus.container-image.tag}

The namespace the generated resources should belong to. If not value is set, then the 'namespace' field will not be added to the 'metadata' section of the generated manifests. This in turn means that when the manifests are applied to a cluster, the namespace will be resolved from the current Kubernetes context (see https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#context for more details).

string

Whether or not to add the build timestamp to the Kubernetes annotations This is a very useful way to have manifests of successive builds of the same application differ - thus ensuring that Kubernetes will apply the updated resources

boolean

true

Working directory

string

list of string

list of string

The service account

string

The host under which the application is going to be exposed

string

The number of desired pods

int

1

The type of service that will be generated for the application

cluster-ip, node-port, load-balancer, external-name

cluster-ip

The nodePort to set when serviceType is set to nodePort

int

always, if-not-present, never

always

The image pull secret

list of string

The http path to use for the probe For this to work, the container port also needs to be set Assuming the container port has been set (as per above comment), if execAction or tcpSocketAction are not set, an http probe will be used automatically even if no path is set (which will result in the root path being used)

string

The command to use for the probe.

string

The tcp socket to use for the probe (the format is host:port).

string

The amount of time to wait before starting to probe.

Duration

0S

The period in which the action should be called.

Duration

30S

The amount of time to wait for each action.

Duration

10S

int

1

int

3

The http path to use for the probe For this to work, the container port also needs to be set Assuming the container port has been set (as per above comment), if execAction or tcpSocketAction are not set, an http probe will be used automatically even if no path is set (which will result in the root path being used)

string

The command to use for the probe.

string

The tcp socket to use for the probe (the format is host:port).

string

The amount of time to wait before starting to probe.

Duration

0S

The period in which the action should be called.

Duration

30S

The amount of time to wait for each action.

Duration

10S

int

1

int

3

When true (the default), emit a set of annotations to identify services that should be scraped by prometheus for metrics. In configurations that use the Prometheus operator with ServiceMonitor, annotations may not be necessary.

boolean

true

Define the annotation prefix used for scrape values, this value will be used as the base for other annotation name defaults. Altering the base for generated annotations can make it easier to define re-labeling rules and avoid unexpected knock-on effects. The default value is prometheus.io See Prometheus example: https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus-kubernetes.yml

string

prometheus.io

Define the annotation used to indicate services that should be scraped. By default, /scrape will be appended to the defined prefix.

string

Define the annotation used to indicate the path to scrape. By default, /path will be appended to the defined prefix.

string

Define the annotation used to indicate the port to scrape. By default, /port will be appended to the defined prefix.

string

Define the annotation used to indicate the scheme to use for scraping By default, /scheme will be appended to the defined prefix.

string

string

string

string

string

If true, an Openshift Route will be created

boolean

false

If true, the service will be exposed

boolean

false

The host under which the application is going to be exposed

string

If true, the 'app.kubernetes.io/version' label will be part of the selectors of Service and DeploymentConfig

boolean

true

The optional list of Secret names to load environment variables from.

list of string

The optional list of ConfigMap names to load environment variables from.

list of string

If set, the secret will mounted to the application container and its contents will be used for application configuration.

string

If set, the config amp will mounted to the application container and its contents will be used for application configuration.

string

Custom labels to add to all resources

Map<String,String>

Custom annotations to add to all resources

Map<String,String>

The port number. Refers to the container port.

int

int

The application path (refers to web application path).

string

/

tcp, udp

tcp

int

The name of the volumeName to mount.

string

string

Path within the volumeName from which the container’s volumeName should be mounted.

string

boolean

false

string

required

Default mode. When specifying an octal number, leading zero must be present.

string

0600

boolean

false

string

required

Default mode. When specifying an octal number, leading zero must be present.

string

0600

boolean

false

string

required

The directory of the repository to mount.

string

string

string

required

Default mode. When specifying an octal number, leading zero must be present.

string

0600

boolean

false

string

required

int

string

ext4

boolean

false

string

required

string

required

Wether the volumeName is read only or not.

boolean

false

string

required

The URI of the vhd blob object OR the resourceID of an Azure managed data disk if Kind is Managed

string

required

managed, shared

managed

read-write, read-only, none

read-write

string

ext4

Wether the volumeName is read only or not.

boolean

false

string

string

list of string

list of string

string

The host under which the application is going to be exposed.

string

int

int

The application path (refers to web application path).

string

/

tcp, udp

tcp

int

always, if-not-present, never

always

list of string

The http path to use for the probe For this to work, the container port also needs to be set Assuming the container port has been set (as per above comment), if execAction or tcpSocketAction are not set, an http probe will be used automatically even if no path is set (which will result in the root path being used)

string

string

The tcp socket to use for the probe (the format is host:port).

string

The amount of time to wait before starting to probe.

Duration

0S

The period in which the action should be called.

Duration

30S

Duration

10S

int

1

int

3

The http path to use for the probe For this to work, the container port also needs to be set Assuming the container port has been set (as per above comment), if execAction or tcpSocketAction are not set, an http probe will be used automatically even if no path is set (which will result in the root path being used)

string

string

The tcp socket to use for the probe (the format is host:port).

string

The amount of time to wait before starting to probe.

Duration

0S

The period in which the action should be called.

Duration

30S

Duration

10S

int

1

int

3

string

string

Path within the volumeName from which the container’s volumeName should be mounted.

string

boolean

false

string

string

string

string

The optional list of Secret names to load environment variables from.

list of string

The optional list of ConfigMap names to load environment variables from.

list of string

The map associating environment variable names to their associated field references they take their value from.

Map<String,String>

The map associating environment name to its associated value.

Map<String,String>

The optional name of the Secret from which a value is to be extracted. Mutually exclusive with from-configmap.

string

The optional name of the ConfigMap from which a value is to be extracted. Mutually exclusive with from-secret.

string

The key identifying the field from which the value is extracted.

string

required

string

string

list of string

list of string

string

The host under which the application is going to be exposed.

string

The port number. Refers to the container port.

int

int

The application path (refers to web application path).

string

/

tcp, udp

tcp

int

always, if-not-present, never

always

list of string

The http path to use for the probe For this to work, the container port also needs to be set Assuming the container port has been set (as per above comment), if execAction or tcpSocketAction are not set, an http probe will be used automatically even if no path is set (which will result in the root path being used)

string

string

The tcp socket to use for the probe (the format is host:port).

string

The amount of time to wait before starting to probe.

Duration

0S

The period in which the action should be called.

Duration

30S

The amount of time to wait for each action.

Duration

10S

int

1

int

3

The http path to use for the probe For this to work, the container port also needs to be set Assuming the container port has been set (as per above comment), if execAction or tcpSocketAction are not set, an http probe will be used automatically even if no path is set (which will result in the root path being used)

string

string

The tcp socket to use for the probe (the format is host:port).

string

The amount of time to wait before starting to probe.

Duration

0S

The period in which the action should be called.

Duration

30S

The amount of time to wait for each action.

Duration

10S

int

1

int

3

The name of the volumeName to mount.

string

string

Path within the volumeName from which the container’s volumeName should be mounted.

string

boolean

false

string

string

string

string

The optional list of Secret names to load environment variables from.

list of string

The optional list of ConfigMap names to load environment variables from.

list of string

The map associating environment variable names to their associated field references they take their value from.

Map<String,String>

The map associating environment name to its associated value.

Map<String,String>

The optional name of the Secret from which a value is to be extracted. Mutually exclusive with from-configmap.

string

The optional name of the ConfigMap from which a value is to be extracted. Mutually exclusive with from-secret.

string

The key identifying the field from which the value is extracted.

string

required

string

The hostnames to resolve to the ip

list of string

Custom annotations to add to exposition (route or ingress) resources

Map<String,String>

The map associating environment variable names to their associated field references they take their value from.

Map<String,String>

The map associating environment name to its associated value.

Map<String,String>

The optional name of the Secret from which a value is to be extracted. Mutually exclusive with from-configmap.

string

The optional name of the ConfigMap from which a value is to be extracted. Mutually exclusive with from-secret.

string

The key identifying the field from which the value is extracted.

string

required

About the Duration format

The format for durations uses the standard java.time.Duration format. You can learn more about it in the Duration#parse() javadoc.

You can also provide duration values starting with a number. In this case, if the value consists only of a number, the converter treats the value as seconds. Otherwise, PT is implicitly prepended to the value to obtain a standard java.time.Duration format.