Dev Services for Kafka

If any Kafka-related extension is present (e.g. quarkus-smallrye-reactive-messaging-kafka), Dev Services for Kafka automatically starts a Kafka broker in dev mode and when running tests. So, you don’t have to start a broker manually. The application is configured automatically.

Because starting a Kafka broker can be long, Dev Services for Kafka uses Redpanda, a Kafka compatible broker which starts in ~1 second.

Enabling / Disabling Dev Services for Kafka

Dev Services for Kafka is automatically enabled unless:

  • quarkus.kafka.devservices.enabled is set to false

  • the kafka.bootstrap.servers is configured

  • all the Reactive Messaging Kafka channels have the bootstrap.servers attribute set

Dev Services for Kafka relies on Docker to start the broker. If your environment does not support Docker, you will need to start the broker manually, or connect to an already running broker. You can configure the broker address using kafka.bootstrap.servers.

Shared broker

Most of the time you need to share the broker between applications. Dev Services for Kafka implements a service discovery mechanism for your multiple Quarkus applications running in dev mode to share a single broker.

Dev Services for Kafka starts the container with the quarkus-dev-service-kafka label which is used to identify the container.

If you need multiple (shared) brokers, you can configure the quarkus.kafka.devservices.service-name attribute and indicate the broker name. It looks for a container with the same value, or starts a new one if none can be found. The default service name is kafka.

Sharing is enabled by default in dev mode, but disabled in test mode. You can disable the sharing with quarkus.kafka.devservices.shared=false.

Setting the port

By default, Dev Services for Kafka picks a random port and configures the application. You can set the port by configuring the quarkus.kafka.devservices.port property.

Note that the Kafka advertised address is automatically configured with the chosen port.

Configuring the image

Dev Services for Kafka uses: vectorized/redpanda images. You can select any version from https://hub.docker.com/r/vectorized/redpanda:

quarkus.kafka.devservices.image-name=vectorized/redpanda:latest
Dev Services for Kafka only support Redpanda.

Configuring Kafka topics

You can configure the Dev Services for Kafka to create topics once the broker is started. Topics are created with given number of partitions and 1 replica.

The following example creates a topic named test with 3 partitions, and a second topic named messages with 2 partitions.

quarkus.kafka.devservices.topic-partitions.test=3
quarkus.kafka.devservices.topic-partitions.messages=2

If a topic already exists with the given name, the creation is skipped, without trying to re-partition the existing topic to a different number of partitions.

You can configure timeout for Kafka admin client calls used in topic creation using quarkus.kafka.devservices.topic-partitions-timeout, it defaults to 2 seconds.