Using OpenTracing

Prerequisites
Architecture
Solution
Creating the Maven project
- Examine the JAX-RS resource
- Create the configuration
Run the application
Tracing additional methods
Additional instrumentation
Jaeger Configuration Reference

This guide explains how your Quarkus application can utilize OpenTracing to provide distributed tracing for interactive web applications.

OpenTelemetry is the recommended approach to tracing and telemetry for Quarkus.

When Quarkus will upgrade to Eclipse MicroProfile 6, the SmallRye OpenTracing support will be discontinued.

This technology is considered deprecated.

Being deprecated means that this extension is likely to be replaced or removed in a future version of Quarkus.

For a full list of possible statuses, check our FAQ entry.

Prerequisites

To complete this guide, you need:

Roughly 15 minutes
An IDE
JDK 11+ installed with JAVA_HOME configured appropriately
Apache Maven 3.8.6
A working container runtime (Docker or Podman)
Optionally the Quarkus CLI if you want to use it
Optionally Mandrel or GraalVM installed and configured appropriately if you want to build a native executable (or Docker if you use a native container build)

Architecture

In this guide, we create a straightforward REST application to demonstrate distributed tracing.

Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can skip right to the completed example.

Clone the Git repository: git clone https://github.com/quarkusio/quarkus-quickstarts.git, or download an archive.

The solution is located in the opentracing-quickstart directory.

Creating the Maven project

First, we need a new project. Create a new project with the following command:

CLI

quarkus create app org.acme:opentracing-quickstart \
    --extension='resteasy-reactive,quarkus-smallrye-opentracing' \
    --no-code
cd opentracing-quickstart

To create a Gradle project, add the --gradle or --gradle-kotlin-dsl option.

For more information about how to install the Quarkus CLI and use it, please refer to the Quarkus CLI guide.

Maven

mvn io.quarkus.platform:quarkus-maven-plugin:2.16.0.Final:create \
    -DprojectGroupId=org.acme \
    -DprojectArtifactId=opentracing-quickstart \
    -Dextensions='resteasy-reactive,quarkus-smallrye-opentracing' \
    -DnoCode
cd opentracing-quickstart

To create a Gradle project, add the -DbuildTool=gradle or -DbuildTool=gradle-kotlin-dsl option.

This command generates the Maven project and imports the smallrye-opentracing extension, which includes the OpenTracing support and the default Jaeger tracer.

If you already have your Quarkus project configured, you can add the smallrye-opentracing extension to your project by running the following command in your project base directory:

CLI

quarkus extension add 'smallrye-opentracing'

Maven

./mvnw quarkus:add-extension -Dextensions='smallrye-opentracing'

Gradle

./gradlew addExtension --extensions='smallrye-opentracing'

This will add the following to your build file:

pom.xml

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-smallrye-opentracing</artifactId>
</dependency>

build.gradle

implementation("io.quarkus:quarkus-smallrye-opentracing")

Examine the JAX-RS resource

Create the src/main/java/org/acme/opentracing/TracedResource.java file with the following content:

package org.acme.opentracing;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import org.jboss.logging.Logger;

@Path("/hello")
public class TracedResource {

    private static final Logger LOG = Logger.getLogger(TracedResource.class);

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        LOG.info("hello"); (1)
        return "hello";
    }
}

1	The log event carries OpenTracing information as well. In order to print OpenTracing information to the console output, the console log handler with the required OpenTracing event’s keys needs to be defined in the `application.properties` file.

Notice that there is no tracing specific code included in the application. By default, requests sent to this endpoint will be traced without any code changes being required. It is also possible to enhance the tracing information. This can be achieved by SmallRye OpenTracing an implementation of MicroProfile OpenTracing.

Create the configuration

There are two ways to configure the Jaeger tracer within the application.

The first approach is by providing the properties within the src/main/resources/application.properties file:

quarkus.jaeger.service-name=myservice (1)
quarkus.jaeger.sampler-type=const (2)
quarkus.jaeger.sampler-param=1 (3)
quarkus.log.console.format=%d{HH:mm:ss} %-5p traceId=%X{traceId}, parentId=%X{parentId}, spanId=%X{spanId}, sampled=%X{sampled} [%c{2.}] (%t) %s%e%n (4)

1	If the `quarkus.jaeger.service-name` property (or `JAEGER_SERVICE_NAME` environment variable) is not provided then a "no-op" tracer will be configured, resulting in no tracing data being reported to the backend.
2	Set up a sampler that uses a constant sampling strategy.
3	Sample all requests. Set sampler-param to somewhere between 0 and 1, e.g. 0.50, if you do not wish to sample all requests.
4	Add trace IDs into log message.

The second approach is to supply the properties as environment variables. These can be specified as jvm.args as shown in the following section.

Run the application

The first step is to start the tracing system to collect and display the captured traces:

docker run -p 5775:5775/udp -p 6831:6831/udp -p 6832:6832/udp -p 5778:5778 -p 16686:16686 -p 14268:14268 jaegertracing/all-in-one:latest

Now we are ready to run our application. If using application.properties to configure the tracer:

CLI

quarkus dev

Maven

./mvnw quarkus:dev

Gradle

./gradlew --console=plain quarkusDev

or if configuring the tracer via environment variables:

CLI

quarkus dev -Djvm.args="-DJAEGER_SERVICE_NAME=myservice -DJAEGER_SAMPLER_TYPE=const -DJAEGER_SAMPLER_PARAM=1"

Maven

./mvnw quarkus:dev -Djvm.args="-DJAEGER_SERVICE_NAME=myservice -DJAEGER_SAMPLER_TYPE=const -DJAEGER_SAMPLER_PARAM=1"

Gradle

./gradlew --console=plain quarkusDev -Djvm.args="-DJAEGER_SERVICE_NAME=myservice -DJAEGER_SAMPLER_TYPE=const -DJAEGER_SAMPLER_PARAM=1"

Once both the application and tracing system are started, you can make a request to the provided endpoint:

$ curl http://localhost:8080/hello
hello

When the first request has been submitted, the Jaeger tracer within the app will be initialized:

2019-10-16 09:35:23,464 INFO  [io.jae.Configuration] (executor-thread-1) Initialized tracer=JaegerTracer(version=Java-0.34.0, serviceName=myservice, reporter=RemoteReporter(sender=UdpSender(), closeEnqueueTimeout=1000), sampler=ConstSampler(decision=true, tags={sampler.type=const, sampler.param=true}), tags={hostname=localhost.localdomain, jaeger.version=Java-0.34.0, ip=127.0.0.1}, zipkinSharedRpcSpan=false, expandExceptionLogs=false, useTraceId128Bit=false)
13:20:11 INFO  traceId=1336b2b0a76a96a3, parentId=0, spanId=1336b2b0a76a96a3, sampled=true [or.ac.qu.TracedResource] (executor-thread-63) hello

Then visit the Jaeger UI to see the tracing information.

Hit CTRL+C to stop the application.

Tracing additional methods

REST endpoints are automatically traced. If you need to trace additional methods, you can add the org.eclipse.microprofile.opentracing.Traced annotation to CDI bean classes or their non-private methods.

This can be useful to trace incoming requests from non-REST calls (like request coming from a message) or to create spans inside a trace.

Here is an example of a FrancophoneService which methods are traced.

import javax.enterprise.context.ApplicationScoped;

import org.eclipse.microprofile.opentracing.Traced;

@Traced
@ApplicationScoped
public class FrancophoneService {

    public String bonjour() {
        return "bonjour";
    }
}

The best way to add OpenTracing capability to reactive messaging based applications is by adding the Traced annotation to all incoming methods.

Additional instrumentation

The OpenTracing API Contributions project offers additional instrumentation that can be used to add tracing to a large variety of technologies/components.

The instrumentation documented in this section has been tested with Quarkus and works in both standard and native mode.

JDBC

The JDBC instrumentation will add a span for each JDBC queries done by your application, to enable it, add the following dependency to your build file:

pom.xml

<dependency>
    <groupId>io.opentracing.contrib</groupId>
    <artifactId>opentracing-jdbc</artifactId>
</dependency>

build.gradle

implementation("io.opentracing.contrib:opentracing-jdbc")

Then, you need to enable it in the configuration:

quarkus.datasource.jdbc.tracing=true

quarkus.datasource.jdbc.tracing is a build time configuration property: it makes sure all the tracing infrastructure is included in your application.

This is especially important when building a native executable as we need to make sure the OpenTracing JDBC driver has been registered for reflection, together with the underlying JDBC driver.

The Agroal extension will take care of adjusting the JDBC URL with the tracing prefix when tracing is enabled, so you do not have to adjust the JDBC URL yourself.

By default, when quarkus.datasource.jdbc.tracing is true, tracing is enabled at runtime but you can explicitly disable it by setting the following property:

quarkus.datasource.jdbc.tracing.enabled=false

This way, you can have your Quarkus application ready for tracing and toggle JDBC tracing at runtime.

Kafka

The Kafka instrumentation will add a span for each message sent to or received from a Kafka topic. To enable it, add the following dependency to your build file:

pom.xml

<dependency>
    <groupId>io.opentracing.contrib</groupId>
    <artifactId>opentracing-kafka-client</artifactId>
</dependency>

build.gradle

implementation("io.opentracing.contrib:opentracing-kafka-client")

It contains OpenTracing interceptors that must be registered on Kafka producers and consumers.

If you followed the Kafka guide, the interceptors can be added on the generated-price and the prices channels as follows:

# Configure the Kafka sink (we write to it)
mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=prices
mp.messaging.outgoing.generated-price.value.serializer=org.apache.kafka.common.serialization.IntegerSerializer
mp.messaging.outgoing.generated-price.interceptor.classes=io.opentracing.contrib.kafka.TracingProducerInterceptor

# Configure the Kafka source (we read from it)
mp.messaging.incoming.prices.connector=smallrye-kafka
mp.messaging.incoming.prices.value.deserializer=org.apache.kafka.common.serialization.IntegerDeserializer
mp.messaging.incoming.prices.interceptor.classes=io.opentracing.contrib.kafka.TracingConsumerInterceptor

interceptor.classes accept a list of classes separated by a comma.

MongoDB client

The Mongo Driver instrumentation will add a span for each command executed by your application. To enable it, add the following dependency to your build file:

pom.xml

<dependency>
    <groupId>io.opentracing.contrib</groupId>
    <artifactId>opentracing-mongo-common</artifactId>
</dependency>

build.gradle

implementation("io.opentracing.contrib:opentracing-mongo-common")

It contains the OpenTracing CommandListener that will be registered on the configuration of the mongo client. Following the MongoDB guide, the command listener will be registered defining the config property as follows:

# Enable tracing commands in mongodb client
quarkus.mongodb.tracing.enabled=true

Zipkin compatibility mode

To enable it, add the following dependency to your build file:

pom.xml

<dependency>
    <groupId>io.jaegertracing</groupId>
    <artifactId>jaeger-zipkin</artifactId>
</dependency>

build.gradle

implementation("io.jaegertracing:jaeger-zipkin")

It contains the dependencies to convert the request to zipkin format. The zipkin compatibility mode will be activated after defining the config property as follows:

# Enable zipkin compatibility mode
quarkus.jaeger.zipkin.compatibility-mode=true

Jaeger Configuration Reference

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property	Type	Default
`quarkus.jaeger.enabled` Defines if the Jaeger extension is enabled. Environment variable: `QUARKUS_JAEGER_ENABLED`	boolean	`true`
`quarkus.jaeger.metrics.enabled` Whether metrics are published in case a metrics extension is present. Environment variable: `QUARKUS_JAEGER_METRICS_ENABLED`	boolean	`false`
`quarkus.jaeger.endpoint` The traces endpoint, in case the client should connect directly to the Collector, like http://jaeger-collector:14268/api/traces Environment variable: `QUARKUS_JAEGER_ENDPOINT`	URI
`quarkus.jaeger.auth-token` Authentication Token to send as "Bearer" to the endpoint Environment variable: `QUARKUS_JAEGER_AUTH_TOKEN`	string
`quarkus.jaeger.user` Username to send as part of "Basic" authentication to the endpoint Environment variable: `QUARKUS_JAEGER_USER`	string
`quarkus.jaeger.password` Password to send as part of "Basic" authentication to the endpoint Environment variable: `QUARKUS_JAEGER_PASSWORD`	string
`quarkus.jaeger.agent-host-port` The hostname and port for communicating with agent via UDP Environment variable: `QUARKUS_JAEGER_AGENT_HOST_PORT`	host:port
`quarkus.jaeger.reporter-log-spans` Whether the reporter should also log the spans Environment variable: `QUARKUS_JAEGER_REPORTER_LOG_SPANS`	boolean
`quarkus.jaeger.reporter-max-queue-size` The reporter’s maximum queue size Environment variable: `QUARKUS_JAEGER_REPORTER_MAX_QUEUE_SIZE`	int
`quarkus.jaeger.reporter-flush-interval` The reporter’s flush interval Environment variable: `QUARKUS_JAEGER_REPORTER_FLUSH_INTERVAL`	Duration
`quarkus.jaeger.sampler-type` The sampler type (const, probabilistic, ratelimiting or remote) Environment variable: `QUARKUS_JAEGER_SAMPLER_TYPE`	string
`quarkus.jaeger.sampler-param` The sampler parameter (number) Environment variable: `QUARKUS_JAEGER_SAMPLER_PARAM`	BigDecimal
`quarkus.jaeger.sampler-manager-host-port` The host name and port when using the remote controlled sampler Environment variable: `QUARKUS_JAEGER_SAMPLER_MANAGER_HOST_PORT`	host:port
`quarkus.jaeger.service-name` The service name Environment variable: `QUARKUS_JAEGER_SERVICE_NAME`	string
`quarkus.jaeger.tags` A comma separated list of name = value tracer level tags, which get added to all reported spans. The value can also refer to an environment variable using the format ${envVarName:default}, where the :default is optional, and identifies a value to be used if the environment variable cannot be found Environment variable: `QUARKUS_JAEGER_TAGS`	string
`quarkus.jaeger.propagation` Comma separated list of formats to use for propagating the trace context. Defaults to the standard Jaeger format. Valid values are jaeger and b3 Environment variable: `QUARKUS_JAEGER_PROPAGATION`	string
`quarkus.jaeger.sender-factory` The sender factory class name Environment variable: `QUARKUS_JAEGER_SENDER_FACTORY`	string
`quarkus.jaeger.log-trace-context` Whether the trace context should be logged. Environment variable: `QUARKUS_JAEGER_LOG_TRACE_CONTEXT`	boolean	`true`
`quarkus.jaeger.disable-tracer-registration` Whether the registration of tracer as the global tracer should be disabled. This setting should only be turned on in tests that need to install a mock tracer. Environment variable: `QUARKUS_JAEGER_DISABLE_TRACER_REGISTRATION`	boolean	`false`
`quarkus.jaeger.zipkin.compatibility-mode` Whether jaeger should run in zipkin compatibility mode Environment variable: `QUARKUS_JAEGER_ZIPKIN_COMPATIBILITY_MODE`	boolean	`false`

Configuration property

Type

Default

quarkus.jaeger.enabled

Defines if the Jaeger extension is enabled.

Environment variable: QUARKUS_JAEGER_ENABLED

boolean

true

quarkus.jaeger.metrics.enabled

Whether metrics are published in case a metrics extension is present.

Environment variable: QUARKUS_JAEGER_METRICS_ENABLED

boolean

false

quarkus.jaeger.endpoint

The traces endpoint, in case the client should connect directly to the Collector, like http://jaeger-collector:14268/api/traces

Environment variable: QUARKUS_JAEGER_ENDPOINT

URI

quarkus.jaeger.auth-token

Authentication Token to send as "Bearer" to the endpoint

Environment variable: QUARKUS_JAEGER_AUTH_TOKEN

string

quarkus.jaeger.user

Username to send as part of "Basic" authentication to the endpoint

Environment variable: QUARKUS_JAEGER_USER

string

quarkus.jaeger.password

Password to send as part of "Basic" authentication to the endpoint

Environment variable: QUARKUS_JAEGER_PASSWORD

string

quarkus.jaeger.agent-host-port

The hostname and port for communicating with agent via UDP

Environment variable: QUARKUS_JAEGER_AGENT_HOST_PORT

host:port

quarkus.jaeger.reporter-log-spans

Whether the reporter should also log the spans

Environment variable: QUARKUS_JAEGER_REPORTER_LOG_SPANS

boolean

quarkus.jaeger.reporter-max-queue-size

The reporter’s maximum queue size

Environment variable: QUARKUS_JAEGER_REPORTER_MAX_QUEUE_SIZE

int

quarkus.jaeger.reporter-flush-interval

The reporter’s flush interval

Environment variable: QUARKUS_JAEGER_REPORTER_FLUSH_INTERVAL

Duration

quarkus.jaeger.sampler-type

The sampler type (const, probabilistic, ratelimiting or remote)

Environment variable: QUARKUS_JAEGER_SAMPLER_TYPE

string

quarkus.jaeger.sampler-param

The sampler parameter (number)

Environment variable: QUARKUS_JAEGER_SAMPLER_PARAM

BigDecimal

quarkus.jaeger.sampler-manager-host-port

The host name and port when using the remote controlled sampler

Environment variable: QUARKUS_JAEGER_SAMPLER_MANAGER_HOST_PORT

host:port

quarkus.jaeger.service-name

The service name

Environment variable: QUARKUS_JAEGER_SERVICE_NAME

string

quarkus.jaeger.tags

A comma separated list of name = value tracer level tags, which get added to all reported spans. The value can also refer to an environment variable using the format ${envVarName:default}, where the :default is optional, and identifies a value to be used if the environment variable cannot be found

Environment variable: QUARKUS_JAEGER_TAGS

string

quarkus.jaeger.propagation

Comma separated list of formats to use for propagating the trace context. Defaults to the standard Jaeger format. Valid values are jaeger and b3

Environment variable: QUARKUS_JAEGER_PROPAGATION

string

quarkus.jaeger.sender-factory

The sender factory class name

Environment variable: QUARKUS_JAEGER_SENDER_FACTORY

string

quarkus.jaeger.log-trace-context

Whether the trace context should be logged.

Environment variable: QUARKUS_JAEGER_LOG_TRACE_CONTEXT

boolean

true

quarkus.jaeger.disable-tracer-registration

Whether the registration of tracer as the global tracer should be disabled. This setting should only be turned on in tests that need to install a mock tracer.

Environment variable: QUARKUS_JAEGER_DISABLE_TRACER_REGISTRATION

boolean

false

quarkus.jaeger.zipkin.compatibility-mode

Whether jaeger should run in zipkin compatibility mode

Environment variable: QUARKUS_JAEGER_ZIPKIN_COMPATIBILITY_MODE

boolean

false

About the Duration format

The format for durations uses the standard java.time.Duration format. You can learn more about it in the Duration#parse() javadoc.

You can also provide duration values starting with a number. In this case, if the value consists only of a number, the converter treats the value as seconds. Otherwise, PT is implicitly prepended to the value to obtain a standard java.time.Duration format.