By Version

Edit this Page

Continuous Testing

Learn how to use continuous testing in your Quarkus Application.

1. Prerequisites

To complete this guide, you need:

Roughly 15 minutes
An IDE
JDK 17+ installed with JAVA_HOME configured appropriately
Apache Maven 3.9.9
Optionally the Quarkus CLI if you want to use it
Optionally Mandrel or GraalVM installed and configured appropriately if you want to build a native executable (or Docker if you use a native container build)
The completed greeter application from the Getting Started Guide

2. Introduction

Quarkus supports continuous testing, where tests run immediately after code changes have been saved. This allows you to get instant feedback on your code changes. Quarkus detects which tests cover which code, and uses this information to only run the relevant tests when code is changed.

3. Solution

Start the Getting Started application (or any other application) using:

CLI

quarkus dev

Maven

./mvnw quarkus:dev

Gradle

./gradlew --console=plain quarkusDev

Quarkus will start in development mode as normal, but down the bottom of the screen you should see the following:

--
Tests paused, press [r] to resume, [h] for more options>

Press r and the tests will start running. You should see the status change down the bottom of the screen as they are running, and it should finish with:

--
Tests all passed, 2 tests were run, 0 were skipped. Tests took 1470ms.
Press [r] to re-run, [v] to view full results, [p] to pause, [h] for more options>

If you want continuous testing to start automatically you can set quarkus.test.continuous-testing=enabled in application.properties. If you don’t want it at all you can change this to disabled.

Now you can start making changes to your application. Go into the GreetingResource and change the hello endpoint to return "hello world", and save the file. Quarkus should immediately re-run the test, and you should get output similar to the following:

2021-05-11 14:21:34,338 ERROR [io.qua.test] (Test runner thread) Test GreetingResourceTest#testHelloEndpoint() failed
: java.lang.AssertionError: 1 expectation failed.
Response body doesn't match expectation.
Expected: is "hello"
  Actual: hello world

	at io.restassured.internal.ValidatableResponseImpl.body(ValidatableResponseImpl.groovy)
	at org.acme.getting.started.GreetingResourceTest.testHelloEndpoint(GreetingResourceTest.java:21)


--
Test run failed, 2 tests were run, 1 failed, 0 were skipped. Tests took 295ms
Press [r] to re-run, [v] to view full results, [p] to pause, [h] for more options>

Change it back and the tests will run again.

4. Controlling Continuous Testing

There are various hotkeys you can use to control continuous testing. Pressing h will display the following list of commands:

The following commands are available:
[r] - Re-run all tests
[f] - Re-run failed tests
[b] - Toggle 'broken only' mode, where only failing tests are run (disabled)
[v] - Print failures from the last test run
[p] - Pause tests
[o] - Toggle test output (disabled)
[i] - Toggle instrumentation based reload (disabled)
[l] - Toggle live reload (enabled)
[s] - Force restart
[h] - Display this help
[q] - Quit

These are explained below:

[r] - Re-run all tests: This will re-run every test
[f] - Re-run failed tests: This will re-run every failing test
[b] - Toggle 'broken only' mode, where only failing tests are run: Broken only mode will only run tests that have previously failed, even if other tests would normally be affected by a code change. This can be useful if you are modifying code that is used by lots of tests, but you only want to focus on debugging the failing one.
[v] - Print failures from the last test run: Prints the failures to the console again, this can be useful if there has been lots of console output since the last run.
[p] - Pause tests: Temporarily stops running tests. This can be useful if you are making lots of changes, and don’t want feedback until they are all done.
[o] - Toggle test output: By default test output is filtered and not displayed on the console, so that test output and dev mode output is not interleaved. Enabling test output will print output to the console when tests are run. Even when output is disabled the filtered output is saved and can be viewed in the Dev UI.
[i] - Toggle instrumentation based reload: This is not directly related to testing, but allows you to toggle instrumentation based reload. This will allow live reload to avoid a restart if a change does not affect the structure of a class, which gives a faster reload and allows you to keep state.
[l] - Toggle live reload: This is not directly related to testing, but allows you to turn live reload on and off.
[s] - Force restart: This will force a scan for changed files, and will perform a live reload with and changes. Note that even if there are no changes the application will still restart. This will still work even if live reload is disabled.

5. Continuous Testing Without Dev Mode

It is possible to run continuous testing without starting dev mode. This can be useful if dev mode will interfere with your tests (e.g. running wiremock on the same port), or if you only want to develop using tests. To start continuous testing mode run mvn quarkus:test.

The Dev UI is not available when running in continuous testing mode, as this is provided by dev mode.

6. Multi-module Projects

Note that continuous testing supports multi-module projects, so tests in modules other than the application can still be run when files are changed. The modules that are run can be controlled using config as listed below.

This is enabled by default, and can be disabled via quarkus.test.only-test-application-module=true.

7. Configuring Continuous Testing

Continuous testing supports multiple configuration options that can be used to limit the tests that are run, and to control the output. The configuration properties are shown below:

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property	Type	Default
`quarkus.test.continuous-testing` If continuous testing is enabled. The default value is 'paused', which will allow you to start testing from the console or the Dev UI, but will not run tests on startup. If this is set to 'enabled' then testing will start as soon as the application has started. If this is 'disabled' then continuous testing is not enabled, and can’t be enabled without restarting the application. Environment variable: `QUARKUS_TEST_CONTINUOUS_TESTING` Show more	`paused`, `enabled`, `disabled`	`paused`
`quarkus.test.display-test-output` If output from the running tests should be displayed in the console. Environment variable: `QUARKUS_TEST_DISPLAY_TEST_OUTPUT` Show more	boolean	`false`
`quarkus.test.include-tags` Tags that should be included for continuous testing. This supports JUnit Tag Expressions. Environment variable: `QUARKUS_TEST_INCLUDE_TAGS` Show more	list of string
`quarkus.test.exclude-tags` Tags that should be excluded by default with continuous testing. This is ignored if include-tags has been set. Defaults to 'slow'. This supports JUnit Tag Expressions. Environment variable: `QUARKUS_TEST_EXCLUDE_TAGS` Show more	list of string	`slow`
`quarkus.test.include-pattern` Tests that should be included for continuous testing. This is a regular expression and is matched against the test class name (not the file name). Environment variable: `QUARKUS_TEST_INCLUDE_PATTERN` Show more	string
`quarkus.test.exclude-pattern` Tests that should be excluded with continuous testing. This is a regular expression and is matched against the test class name (not the file name). This is ignored if include-pattern has been set. Environment variable: `QUARKUS_TEST_EXCLUDE_PATTERN` Show more	string	`.\.IT[^.]+\|.IT\|.*ITCase`
`quarkus.test.include-engines` Test engine ids that should be included for continuous testing. Environment variable: `QUARKUS_TEST_INCLUDE_ENGINES` Show more	list of string
`quarkus.test.exclude-engines` Test engine ids that should be excluded by default with continuous testing. This is ignored if include-engines has been set. Environment variable: `QUARKUS_TEST_EXCLUDE_ENGINES` Show more	list of string
`quarkus.test.flat-class-path` Changes tests to use the 'flat' ClassPath used in Quarkus 1.x versions. This means all Quarkus and test classes are loaded in the same ClassLoader, however it means you cannot use continuous testing. Note that if you find this necessary for your application then you may also have problems running in development mode, which cannot use a flat class path. Environment variable: `QUARKUS_TEST_FLAT_CLASS_PATH` Show more	boolean	`false`
`quarkus.test.native-image-profile` The profile to use when testing the native image Environment variable: `QUARKUS_TEST_NATIVE_IMAGE_PROFILE` Show more	string	`prod`
`quarkus.test.integration-test-profile` The profile to use when testing using `@QuarkusIntegrationTest` Environment variable: `QUARKUS_TEST_INTEGRATION_TEST_PROFILE` Show more	string	`prod`
`quarkus.test.profile` A comma separated list of profiles (dev, test, prod or custom profiles) to use when testing using @QuarkusTest Environment variable: `QUARKUS_TEST_PROFILE` Show more	list of string	`test`
`quarkus.test.profile.tags` The tags this profile is associated with. When the `quarkus.test.profile.tags` System property is set (its value is a comma separated list of strings) then Quarkus will only execute tests that are annotated with a `@TestProfile` that has at least one of the supplied (via the aforementioned system property) tags. Environment variable: `QUARKUS_TEST_PROFILE_TAGS` Show more	list of string
`quarkus.test.container.network` Controls the container network to be used when @QuarkusIntegration needs to launch the application in a container. This setting only applies if Quarkus does not need to use a shared network - which is the case if DevServices are used when running the test. Environment variable: `QUARKUS_TEST_CONTAINER_NETWORK` Show more	string
`quarkus.test.container.additional-exposed-ports."host-port"` Set additional ports to be exposed when @QuarkusIntegration needs to launch the application in a container. Environment variable: `QUARKUS_TEST_CONTAINER_ADDITIONAL_EXPOSED_PORTS__HOST_PORT_` Show more	Map<String,String>
`quarkus.test.container.labels."label-name"` A set of labels to add to the launched container Environment variable: `QUARKUS_TEST_CONTAINER_LABELS__LABEL_NAME_` Show more	Map<String,String>
`quarkus.test.container.volume-mounts."host-path"` A set of volume mounts to add to the launched container Environment variable: `QUARKUS_TEST_CONTAINER_VOLUME_MOUNTS__HOST_PATH_` Show more	Map<String,String>
`quarkus.test.arg-line` Additional launch parameters to be used when Quarkus launches the produced artifact for `@QuarkusIntegrationTest` When the artifact is a `jar`, this string is passed right after the `java` command. When the artifact is a `container`, this string is passed right after the `docker run` command. When the artifact is a `native binary`, this string is passed right after the native binary name. Environment variable: `QUARKUS_TEST_ARG_LINE` Show more	list of string
`quarkus.test.env."environment-variable-name"` Additional environment variables to be set in the process that `@QuarkusIntegrationTest` launches. Environment variable: `QUARKUS_TEST_ENV__ENVIRONMENT_VARIABLE_NAME_` Show more	Map<String,String>
`quarkus.test.wait-time` Used in `@QuarkusIntegrationTest` to determine how long the test will wait for the application to launch Environment variable: `QUARKUS_TEST_WAIT_TIME` Show more	Duration	`PT1M`
`quarkus.test.hang-detection-timeout` Configures the hang detection in @QuarkusTest. If no activity happens (i.e. no test callbacks are called) over this period then QuarkusTest will dump all threads stack traces, to help diagnose a potential hang. Note that the initial timeout (before Quarkus has started) will only apply if provided by a system property, as it is not possible to read all config sources until Quarkus has booted. Environment variable: `QUARKUS_TEST_HANG_DETECTION_TIMEOUT` Show more	Duration	`10M`
`quarkus.test.type` The type of test to run, this can be either: quarkus-test: Only runs `@QuarkusTest` annotated test classes unit: Only runs classes that are not annotated with `@QuarkusTest` all: Runs both, running the unit tests first Environment variable: `QUARKUS_TEST_TYPE` Show more	`unit`, `quarkus-test`, `all`	`all`
`quarkus.test.only-test-application-module` If this is true then only the tests from the main application module will be run (i.e. the module that is currently running mvn quarkus:dev). If this is false then tests from all dependency modules will be run as well. Environment variable: `QUARKUS_TEST_ONLY_TEST_APPLICATION_MODULE` Show more	boolean	`false`
`quarkus.test.include-module-pattern` Modules that should be included for continuous testing. This is a regular expression and is matched against the module groupId:artifactId. Environment variable: `QUARKUS_TEST_INCLUDE_MODULE_PATTERN` Show more	string
`quarkus.test.exclude-module-pattern` Modules that should be excluded for continuous testing. This is a regular expression and is matched against the module groupId:artifactId. This is ignored if include-module-pattern has been set. Environment variable: `QUARKUS_TEST_EXCLUDE_MODULE_PATTERN` Show more	string
`quarkus.test.enable-callbacks-for-integration-tests` If the test callbacks should be invoked for the integration tests (tests annotated with `@QuarkusIntegrationTest`). Environment variable: `QUARKUS_TEST_ENABLE_CALLBACKS_FOR_INTEGRATION_TESTS` Show more	boolean	`false`

Configuration property

Type

Default

quarkus.test.continuous-testing

If continuous testing is enabled. The default value is 'paused', which will allow you to start testing from the console or the Dev UI, but will not run tests on startup. If this is set to 'enabled' then testing will start as soon as the application has started. If this is 'disabled' then continuous testing is not enabled, and can’t be enabled without restarting the application.

Environment variable: QUARKUS_TEST_CONTINUOUS_TESTING

paused, enabled, disabled

paused

quarkus.test.display-test-output

If output from the running tests should be displayed in the console.

Environment variable: QUARKUS_TEST_DISPLAY_TEST_OUTPUT

boolean

false

quarkus.test.include-tags

Tags that should be included for continuous testing. This supports JUnit Tag Expressions.

Environment variable: QUARKUS_TEST_INCLUDE_TAGS

list of string

quarkus.test.exclude-tags

Tags that should be excluded by default with continuous testing. This is ignored if include-tags has been set. Defaults to 'slow'. This supports JUnit Tag Expressions.

Environment variable: QUARKUS_TEST_EXCLUDE_TAGS

list of string

slow

quarkus.test.include-pattern

Tests that should be included for continuous testing. This is a regular expression and is matched against the test class name (not the file name).

Environment variable: QUARKUS_TEST_INCLUDE_PATTERN

string

quarkus.test.exclude-pattern

Tests that should be excluded with continuous testing. This is a regular expression and is matched against the test class name (not the file name). This is ignored if include-pattern has been set.

Environment variable: QUARKUS_TEST_EXCLUDE_PATTERN

string

.*\.IT[^.]+|.*IT|.*ITCase

quarkus.test.include-engines

Test engine ids that should be included for continuous testing.

Environment variable: QUARKUS_TEST_INCLUDE_ENGINES

list of string

quarkus.test.exclude-engines

Test engine ids that should be excluded by default with continuous testing. This is ignored if include-engines has been set.

Environment variable: QUARKUS_TEST_EXCLUDE_ENGINES

list of string

quarkus.test.flat-class-path

Changes tests to use the 'flat' ClassPath used in Quarkus 1.x versions. This means all Quarkus and test classes are loaded in the same ClassLoader, however it means you cannot use continuous testing. Note that if you find this necessary for your application then you may also have problems running in development mode, which cannot use a flat class path.

Environment variable: QUARKUS_TEST_FLAT_CLASS_PATH

boolean

false

quarkus.test.native-image-profile

The profile to use when testing the native image

Environment variable: QUARKUS_TEST_NATIVE_IMAGE_PROFILE

string

prod

quarkus.test.integration-test-profile

The profile to use when testing using @QuarkusIntegrationTest

Environment variable: QUARKUS_TEST_INTEGRATION_TEST_PROFILE

string

prod

quarkus.test.profile

A comma separated list of profiles (dev, test, prod or custom profiles) to use when testing using @QuarkusTest

Environment variable: QUARKUS_TEST_PROFILE

list of string

test

quarkus.test.profile.tags

The tags this profile is associated with. When the quarkus.test.profile.tags System property is set (its value is a comma separated list of strings) then Quarkus will only execute tests that are annotated with a @TestProfile that has at least one of the supplied (via the aforementioned system property) tags.

Environment variable: QUARKUS_TEST_PROFILE_TAGS

list of string

quarkus.test.container.network

Controls the container network to be used when @QuarkusIntegration needs to launch the application in a container. This setting only applies if Quarkus does not need to use a shared network - which is the case if DevServices are used when running the test.

Environment variable: QUARKUS_TEST_CONTAINER_NETWORK

string

quarkus.test.container.additional-exposed-ports."host-port"

Set additional ports to be exposed when @QuarkusIntegration needs to launch the application in a container.

Environment variable: QUARKUS_TEST_CONTAINER_ADDITIONAL_EXPOSED_PORTS__HOST_PORT_

Map<String,String>

quarkus.test.container.labels."label-name"

A set of labels to add to the launched container

Environment variable: QUARKUS_TEST_CONTAINER_LABELS__LABEL_NAME_

Map<String,String>

quarkus.test.container.volume-mounts."host-path"

A set of volume mounts to add to the launched container

Environment variable: QUARKUS_TEST_CONTAINER_VOLUME_MOUNTS__HOST_PATH_

Map<String,String>

quarkus.test.arg-line

Additional launch parameters to be used when Quarkus launches the produced artifact for @QuarkusIntegrationTest When the artifact is a jar, this string is passed right after the java command. When the artifact is a container, this string is passed right after the docker run command. When the artifact is a native binary, this string is passed right after the native binary name.

Environment variable: QUARKUS_TEST_ARG_LINE

list of string

quarkus.test.env."environment-variable-name"

Additional environment variables to be set in the process that @QuarkusIntegrationTest launches.

Environment variable: QUARKUS_TEST_ENV__ENVIRONMENT_VARIABLE_NAME_

Map<String,String>

quarkus.test.wait-time

Used in @QuarkusIntegrationTest to determine how long the test will wait for the application to launch

Environment variable: QUARKUS_TEST_WAIT_TIME

Duration

PT1M

quarkus.test.hang-detection-timeout

Configures the hang detection in @QuarkusTest. If no activity happens (i.e. no test callbacks are called) over this period then QuarkusTest will dump all threads stack traces, to help diagnose a potential hang. Note that the initial timeout (before Quarkus has started) will only apply if provided by a system property, as it is not possible to read all config sources until Quarkus has booted.

Environment variable: QUARKUS_TEST_HANG_DETECTION_TIMEOUT

Duration

10M

quarkus.test.type

The type of test to run, this can be either: quarkus-test: Only runs @QuarkusTest annotated test classes unit: Only runs classes that are not annotated with @QuarkusTest all: Runs both, running the unit tests first

Environment variable: QUARKUS_TEST_TYPE

unit, quarkus-test, all

all

quarkus.test.only-test-application-module

If this is true then only the tests from the main application module will be run (i.e. the module that is currently running mvn quarkus:dev). If this is false then tests from all dependency modules will be run as well.

Environment variable: QUARKUS_TEST_ONLY_TEST_APPLICATION_MODULE

boolean

false

quarkus.test.include-module-pattern

Modules that should be included for continuous testing. This is a regular expression and is matched against the module groupId:artifactId.

Environment variable: QUARKUS_TEST_INCLUDE_MODULE_PATTERN

string

quarkus.test.exclude-module-pattern

Modules that should be excluded for continuous testing. This is a regular expression and is matched against the module groupId:artifactId. This is ignored if include-module-pattern has been set.

Environment variable: QUARKUS_TEST_EXCLUDE_MODULE_PATTERN

string

quarkus.test.enable-callbacks-for-integration-tests

If the test callbacks should be invoked for the integration tests (tests annotated with @QuarkusIntegrationTest).

Environment variable: QUARKUS_TEST_ENABLE_CALLBACKS_FOR_INTEGRATION_TESTS

boolean

false

About the Duration format

To write duration values, use the standard java.time.Duration format. See the Duration#parse() Java API documentation for more information.

You can also use a simplified format, starting with a number:

If the value is only a number, it represents time in seconds.
If the value is a number followed by ms, it represents time in milliseconds.

In other cases, the simplified format is translated to the java.time.Duration format for parsing:

If the value is a number followed by h, m, or s, it is prefixed with PT.
If the value is a number followed by d, it is prefixed with P.

1. Prerequisites
2. Introduction
3. Solution
4. Controlling Continuous Testing
5. Continuous Testing Without Dev Mode
6. Multi-module Projects
7. Configuring Continuous Testing

Continuous Testing

1. Prerequisites

2. Introduction

3. Solution

4. Controlling Continuous Testing

5. Continuous Testing Without Dev Mode

6. Multi-module Projects

7. Configuring Continuous Testing

Related content

On the same topics