Skip to content

Step 05 - Human-in-the-Loop Pattern

Human-in-the-Loop Pattern

In the previous step, you built a Supervisor Pattern that orchestrates multiple agents to handle car returns, including disposition decisions for damaged vehicles.

However, that system made autonomous decisions about vehicle disposition without human oversight. What if you need human approval before executing high-stakes decisions, especially for valuable assets?

In this step, you’ll learn about the Human-in-the-Loop (HITL) pattern - a critical approach where AI agents pause execution to request human approval before proceeding with significant actions.

New Requirement from Miles of Smiles Management: Human Approval for High-Value Dispositions

The Miles of Smiles management team has now realized they may have been a bit too eager to give AI full decision powers: the system has been making autonomous disposition decisions on high-value vehicles without human oversight with some costly outcomes.

They want to implement a human approval gate with these requirements:

  1. Any vehicle worth more than $15,000 requires human approval before disposition
  2. Track approval status and reasoning for compliance and audit trail

This ensures that expensive vehicles aren’t scrapped or sold without proper human review.

What You’ll Learn

In this step, you will:

  • Understand the Human-in-the-Loop (HITL) pattern and when to use it
  • Implement a two-phase approval workflow (proposal → review → execution)
  • Create a DispositionProposalAgent that generates proposals
  • Build a HumanApprovalAgent using LangChain4j’s @HumanInTheLoop annotation
  • Modify the FleetSupervisorAgent to route high-value vehicles through approval
  • Add approval tracking to the data model
  • See how HITL provides safety and control in autonomous systems

Understanding Human-in-the-Loop

What is Human-in-the-Loop?

Human-in-the-Loop (HITL) is a pattern where:

  • AI agents perform analysis and create recommendations
  • Execution pauses to request human approval
  • Humans review proposals and make final decisions
  • System proceeds only after approval

The Two-Phase Workflow

sequenceDiagram
    participant System as Agentic System
    participant Proposal as Proposal Agent
    participant Human as Human Reviewer
    participant Execution as Execution Agent

    System->>Proposal: Analyze situation
    Proposal->>Proposal: Create recommendation
    Proposal-->>System: Proposal ready

    System->>Human: Request approval
    Note over Human: Human reviews<br/>proposal details
    Human-->>System: APPROVED/REJECTED

    alt Approved
        System->>Execution: Execute proposal
        Execution-->>System: Action completed
    else Rejected
        System->>System: Fallback action
        Note over System: Route to alternative<br/>processing path
    end
Hold "Alt" / "Option" to enable pan & zoom

The Complete HITL Architecture

graph TB
    Start(["Car Return"]) --> A["CarProcessingWorkflow<br/>Sequential"]

    A --> B["Step 1: FeedbackAnalysisWorkflow<br/>Parallel Mapper"]
    B --> B1["FeedbackTask.cleaning()"]
    B --> B2["FeedbackTask.maintenance()"]
    B --> B3["FeedbackTask.disposition()"]
    B1 --> BA["FeedbackAnalysisAgent"]
    B2 --> BA
    B3 --> BA
    BA --> BEnd["FeedbackAnalysisResults"]

    BEnd --> C["Step 2: FleetSupervisorAgent<br/>Autonomous Orchestration"]
    C --> C1{"Disposition<br/>Required?"}

    C1 -->|Yes| PA["PricingAgent<br/>Estimate Value"]
    PA --> VCheck{"Value > $15k?"}

    VCheck -->|"Yes - HIGH VALUE"| Proposal["DispositionProposalAgent<br/>Create Proposal"]
    Proposal --> Approval["HumanApprovalAgent<br/>@HumanInTheLoop"]
    Approval --> ApprovalCheck{"Approved?"}
    ApprovalCheck -->|Yes| Execute["Execute Disposition"]
    ApprovalCheck -->|No| Fallback["Route to Maintenance/Cleaning"]

    VCheck -->|"No - LOW VALUE"| Direct["DispositionAgent<br/>Direct Decision"]
    Direct --> Execute

    C1 -->|No| Other["MaintenanceAgent or CleaningAgent"]

    Execute --> CEnd["Supervisor Decision"]
    Fallback --> CEnd
    Other --> CEnd

    CEnd --> D["Step 3: CarConditionFeedbackAgent<br/>Final Summary"]
    D --> End(["Updated Car with Approval Status"])

    style A fill:#90EE90,stroke:#333,stroke-width:2,color:#000
    style B fill:#87CEEB,stroke:#333,stroke-width:2,color:#000
    style C fill:#FFB6C1,stroke:#333,stroke-width:2,color:#000
    style D fill:#90EE90,stroke:#333,stroke-width:2,color:#000
    style Proposal fill:#FFD700,stroke:#333,stroke-width:2,color:#000
    style Approval fill:#FF6B6B,stroke:#333,stroke-width:2,color:#fff
    style VCheck fill:#FFA07A,stroke:#333,stroke-width:2,color:#000
    style ApprovalCheck fill:#FFA07A,stroke:#333,stroke-width:2,color:#000
    style Start fill:#E8E8E8,stroke:#333,stroke-width:2,color:#000
    style End fill:#E8E8E8,stroke:#333,stroke-width:2,color:#000
Hold "Alt" / "Option" to enable pan & zoom

Implementing the Human-in-the-Loop Pattern

Let’s build the HITL system step by step.

Create the DispositionProposalAgent

This agent creates disposition proposals that will be reviewed by humans.

Create src/main/java/com/carmanagement/agentic/agents/DispositionProposalAgent.java:

DispositionProposalAgent.java
package com.carmanagement.agentic.agents;

import dev.langchain4j.agentic.Agent;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;

/**
 * Agent that creates disposition proposals for vehicles requiring disposition.
 * This agent analyzes the vehicle and creates a proposal that will be reviewed
 * by the HumanApprovalAgent if the vehicle value exceeds the threshold.
 */
public interface DispositionProposalAgent {

    @SystemMessage("""
        You are a car disposition specialist for a car rental company.
        Your job is to create a disposition proposal based on the car's value, condition, age, and damage.

        Disposition Options:
        - SCRAP: Car is beyond economical repair or has severe safety concerns
        - SELL: Car has value but is aging out of the fleet or has moderate damage
        - DONATE: Car has minimal value but could serve a charitable purpose
        - KEEP: Car is worth keeping in the fleet

        Decision Criteria:
        - If estimated repair cost > 50% of car value: Consider SCRAP or SELL
        - If car is over 5 years old with significant damage: SCRAP
        - If car is 3-5 years old in fair condition: SELL
        - If car has low value (<$5,000) but functional: DONATE
        - If car is valuable and damage is minor: KEEP

        Your response must include:
        1. Proposed Action with unique marker: __SCRAP__ or __SELL__ or __DONATE__ or __KEEP__
        2. Reasoning: Clear explanation of your recommendation

        Format your response as:
        Proposed Action: __[SCRAP/SELL/DONATE/KEEP]__
        Reasoning: [Your detailed explanation]

        CRITICAL: Use double underscores around the action (e.g., __KEEP__ not KEEP)
        """)
    @UserMessage("""
        Create a disposition proposal for this vehicle:
        - Make: {carMake}
        - Model: {carModel}
        - Year: {carYear}
        - Car Number: {carNumber}
        - Current Condition: {carCondition}
        - Estimated Value: {carValue}
        - Damage/Feedback: {feedback}

        Provide your disposition proposal with clear reasoning.
        """)
    @Agent(outputKey = "dispositionProposal", description = "Creates disposition proposals for vehicles requiring disposition")
    String createDispositionProposal(
            String carMake,
            String carModel,
            Integer carYear,
            Integer carNumber,
            String carCondition,
            String carValue,
            String feedback);
}

Why two disposition agents?

You might wonder why we have both DispositionProposalAgent and DispositionAgent. They serve different purposes: DispositionProposalAgent creates recommendations for human review on high-value vehicles (>$15K), while DispositionAgent makes autonomous decisions on lower-value vehicles.

Create the HumanApprovalAgent

This agent implements Human-in-the-Loop using LangChain4j’s @HumanInTheLoop annotation. Instead of relying on a separate tool, the agent method itself pauses workflow execution until a human makes a decision through the UI.

Create src/main/java/com/carmanagement/agentic/agents/HumanApprovalAgent.java:

HumanApprovalAgent.java
package com.carmanagement.agentic.agents;

import com.carmanagement.model.ApprovalProposal;
import com.carmanagement.service.ApprovalService;
import dev.langchain4j.agentic.Agent;
import dev.langchain4j.agentic.declarative.HumanInTheLoop;
import io.quarkus.arc.Arc;
import io.quarkus.logging.Log;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public interface HumanApprovalAgent {

    @Agent(outputKey = "approvalDecision", description = "Coordinates human approval for high-value vehicle dispositions using the requestHumanApproval tool")
    @HumanInTheLoop(outputKey = "approvalDecision", description = "Coordinates human approval for high-value vehicle dispositions using the requestHumanApproval tool")
    static String reviewDispositionProposal(
            String carMake,
            String carModel,
            Integer carYear,
            Integer carNumber,
            String carValue,
            String dispositionProposal,
            String dispositionReason,
            String carCondition,
            String feedback
    ) {

        Log.infof("🛑 HITL Tool: Creating approval proposal for car %d - %s %s %s",
                carNumber, carYear, carMake, carModel);
        Log.info("⏸️  WORKFLOW PAUSED - Waiting for human approval decision via UI");

        ApprovalService approvalService = Arc.container().instance(ApprovalService.class).get();

        try {
            // Create proposal and get CompletableFuture that completes when human decides
            CompletableFuture<ApprovalProposal> approvalFuture =
                    approvalService.createProposalAndWaitForDecision(
                            carNumber, carMake, carModel, carYear, carValue,
                            dispositionProposal, dispositionReason, carCondition, feedback
                    );

            // BLOCK HERE until human makes decision (with 5 minute timeout)
            ApprovalProposal result = approvalFuture.get(5, TimeUnit.MINUTES);

            Log.infof("▶️  WORKFLOW RESUMED - Human decision received: %s", result.decision);

            // Format response for the agent
            return String.format("""
                Human Decision: %s
                Reason: %s
                Approved By: %s
                Decision Time: %s
                """,
                    result.decision,
                    result.approvalReason != null ? result.approvalReason : "No reason provided",
                    result.approvedBy != null ? result.approvedBy : "Unknown",
                    result.decidedAt != null ? result.decidedAt.toString() : "Unknown"
            );

        } catch (TimeoutException e) {
            Log.error("⏱️  TIMEOUT: No human decision received within 5 minutes, defaulting to REJECTED");
            return """
                Human Decision: REJECTED
                Reason: Timeout - No human decision received within 5 minutes. Defaulting to rejection for safety.
                Approved By: System (Timeout)
                """;
        } catch (Exception e) {
            Log.errorf(e, "❌ ERROR: Failed to get human approval for car %d", carNumber);
            return String.format("""
                Human Decision: REJECTED
                Reason: Error occurred while waiting for human approval: %s
                Approved By: System (Error)
                """, e.getMessage());
        }
    }
}

Key Points:

  • The @HumanInTheLoop annotation from LangChain4j marks this agent method as requiring human interaction before completing
  • It calls ApprovalService.createProposalAndWaitForDecision() which returns a CompletableFuture
  • The Workflow execution pauses by calling .get(5, TimeUnit.MINUTES) on the future
  • A human sees the pending approval in the UI and clicks Approve/Reject
  • The future then completes and the workflow resumes with the human’s decision
  • It returns a message containing APPROVED/REJECTED and the reasoning
  • The result is stored in the AgenticScope with key approvalDecision

The ApprovalService

The ApprovalService manages the CompletableFuture instances that pause and resume workflow execution. This is the bridge between the HumanApprovalAgent and the REST endpoints that the UI calls.

Create src/main/java/com/carmanagement/service/ApprovalService.java:

ApprovalService.java
package com.carmanagement.service;

import com.carmanagement.model.ApprovalProposal;
import com.carmanagement.model.ApprovalProposal.ApprovalStatus;
import io.quarkus.logging.Log;
import io.quarkus.vertx.ConsumeEvent;
import io.smallrye.mutiny.Uni;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import jakarta.persistence.EntityManager;
import jakarta.transaction.Transactional;
import static jakarta.transaction.Transactional.TxType;

import java.time.LocalDateTime;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

/**
 * Service for managing approval proposals and the Human-in-the-Loop workflow.
 * This service handles the async nature of human approvals - creating proposals,
 * waiting for human decisions, and continuing workflow execution.
 */
@ApplicationScoped
public class ApprovalService {

    @Inject
    EntityManager entityManager;

    /**
     * Map to store CompletableFutures waiting for approval decisions.
     * Key: carNumber, Value: CompletableFuture that completes when decision is made
     */
    private final Map<Integer, CompletableFuture<ApprovalProposal>> pendingApprovals = new ConcurrentHashMap<>();

    /**
     * Executor for async proposal creation to ensure transaction commits before blocking
     */
    private final ExecutorService executor = Executors.newCachedThreadPool();

    /**
     * Create a new approval proposal and return a CompletableFuture that will complete
     * when a human makes a decision.
     * 
     * @param carNumber The car number
     * @param carMake Car make
     * @param carModel Car model
     * @param carYear Car year
     * @param carValue Estimated car value
     * @param proposedDisposition Proposed action (SCRAP, SELL, DONATE, KEEP)
     * @param dispositionReason Reasoning for the proposal
     * @param carCondition Current car condition
     * @param rentalFeedback Rental feedback
     * @return CompletableFuture that completes when human approves/rejects
     */
    public CompletableFuture<ApprovalProposal> createProposalAndWaitForDecision(
            Integer carNumber,
            String carMake,
            String carModel,
            Integer carYear,
            String carValue,
            String proposedDisposition,
            String dispositionReason,
            String carCondition,
            String rentalFeedback) {

        // Check if there's already a pending proposal for this car
        ApprovalProposal existing = ApprovalProposal.findPendingByCarNumber(carNumber);
        if (existing != null) {
            Log.warnf("Proposal already exists for car %d, returning existing future", carNumber);
            return pendingApprovals.computeIfAbsent(carNumber, k -> new CompletableFuture<>());
        }

        // Create CompletableFuture first
        CompletableFuture<ApprovalProposal> future = new CompletableFuture<>();
        pendingApprovals.put(carNumber, future);

        // Create proposal in separate thread with its own transaction
        // This ensures the transaction commits BEFORE we return the future
        executor.submit(() -> {
            try {
                createProposalInNewTransaction(carNumber, carMake, carModel, carYear, carValue,
                        proposedDisposition, dispositionReason, carCondition, rentalFeedback);
                Log.info("✅ Proposal creation transaction committed - now visible to queries");
            } catch (Exception e) {
                Log.errorf(e, "Failed to create proposal for car %d", carNumber);
                future.completeExceptionally(e);
                pendingApprovals.remove(carNumber);
            }
        });

        return future;
    }

    @Transactional(TxType.REQUIRES_NEW)
    void createProposalInNewTransaction(
            Integer carNumber,
            String carMake,
            String carModel,
            Integer carYear,
            String carValue,
            String proposedDisposition,
            String dispositionReason,
            String carCondition,
            String rentalFeedback) {

        // Create new proposal
        ApprovalProposal proposal = new ApprovalProposal();
        proposal.carNumber = carNumber;
        proposal.carMake = carMake;
        proposal.carModel = carModel;
        proposal.carYear = carYear;
        proposal.carValue = carValue;
        proposal.proposedDisposition = proposedDisposition;
        proposal.dispositionReason = dispositionReason;
        proposal.carCondition = carCondition;
        proposal.rentalFeedback = rentalFeedback;
        proposal.status = ApprovalStatus.PENDING;
        proposal.createdAt = LocalDateTime.now();

        proposal.persist();
        entityManager.flush();

        Log.infof("Created approval proposal ID=%d for car %d - %s %s %s (Value: %s, Proposed: %s)",
                proposal.id, carNumber, carYear, carMake, carModel, carValue, proposedDisposition);
        Log.info("⏸️  WORKFLOW PAUSED - Waiting for human approval decision");
        Log.infof("Proposal persisted with ID: %d, status: %s", proposal.id, proposal.status);
    }

    /**
     * Process a human's approval decision and complete the waiting CompletableFuture.
     * This resumes the workflow execution.
     * 
     * @param proposalId The proposal ID
     * @param approved Whether approved or rejected
     * @param reason Human's reasoning
     * @param approvedBy Who made the decision
     * @return The updated proposal
     */
    @Transactional(TxType.REQUIRES_NEW)
    public ApprovalProposal processDecision(Integer proposalId, boolean approved, String reason, String approvedBy) {
        ApprovalProposal proposal = ApprovalProposal.findById(proposalId);
        if (proposal == null) {
            throw new IllegalArgumentException("Proposal not found: " + proposalId);
        }

        if (proposal.status != ApprovalStatus.PENDING) {
            throw new IllegalStateException("Proposal is not pending: " + proposalId);
        }

        // Update proposal with decision
        proposal.status = approved ? ApprovalStatus.APPROVED : ApprovalStatus.REJECTED;
        proposal.decision = approved ? "APPROVED" : "REJECTED";
        proposal.approvalReason = reason;
        proposal.approvedBy = approvedBy;
        proposal.decidedAt = LocalDateTime.now();

        proposal.persist();

        Log.infof("Human decision received for car %d: %s - %s",
                proposal.carNumber, proposal.decision, reason);
        Log.info("▶️  WORKFLOW RESUMED - Continuing with approval decision");

        // Complete the CompletableFuture to resume workflow
        CompletableFuture<ApprovalProposal> future = pendingApprovals.remove(proposal.carNumber);
        if (future != null) {
            future.complete(proposal);
        }

        return proposal;
    }

    /**
     * Get all pending approval proposals.
     */
    public List<ApprovalProposal> getPendingProposals() {
        return ApprovalProposal.findAllPending();
    }

    /**
     * Get a specific proposal by ID.
     */
    public ApprovalProposal getProposal(Integer proposalId) {
        return ApprovalProposal.findById(proposalId);
    }

    /**
     * Check if there's a pending approval for a car.
     */
    public ApprovalProposal getPendingProposalForCar(Integer carNumber) {
        return ApprovalProposal.findPendingByCarNumber(carNumber);
    }
}

Key Points:

  • CompletableFuture<ApprovalProposal> is stored in a map keyed by car number
  • the createProposalAndWaitForDecision() method creates the future and returns it
  • The proposal is persisted in a separate transaction to ensure it’s visible to UI queries
  • The processDecision() method completes the future when human makes a decision
  • This completion resumes the workflow that was blocked on .get()

Create the ApprovalProposal Entity

This entity stores proposals in the database so the UI can display them.

Create src/main/java/com/carmanagement/model/ApprovalProposal.java:

ApprovalProposal.java
package com.carmanagement.model;

import io.quarkus.hibernate.orm.panache.PanacheEntity;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import jakarta.persistence.Column;
import java.time.LocalDateTime;

/**
 * Entity representing a disposition proposal awaiting human approval.
 * This is the core of the Human-in-the-Loop pattern - proposals are stored
 * and the workflow pauses until a human makes an approval decision.
 */
@Entity
public class ApprovalProposal extends PanacheEntity {

    /**
     * The car number this proposal is for
     */
    @Column(nullable = false)
    public Integer carNumber;

    /**
     * Car make
     */
    public String carMake;

    /**
     * Car model
     */
    public String carModel;

    /**
     * Car year
     */
    public Integer carYear;

    /**
     * Estimated car value
     */
    public String carValue;

    /**
     * Proposed disposition action (SCRAP, SELL, DONATE, KEEP)
     */
    @Column(nullable = false)
    public String proposedDisposition;

    /**
     * Reasoning for the proposed disposition
     */
    @Column(length = 2000)
    public String dispositionReason;

    /**
     * Current car condition
     */
    @Column(length = 1000)
    public String carCondition;

    /**
     * Rental feedback that triggered this proposal
     */
    @Column(length = 2000)
    public String rentalFeedback;

    /**
     * Current status of the approval
     */
    @Enumerated(EnumType.STRING)
    @Column(nullable = false)
    public ApprovalStatus status = ApprovalStatus.PENDING;

    /**
     * Human's decision (APPROVED or REJECTED)
     */
    public String decision;

    /**
     * Human's reasoning for their decision
     */
    @Column(length = 1000)
    public String approvalReason;

    /**
     * Who approved/rejected (for audit trail)
     */
    public String approvedBy;

    /**
     * When the proposal was created
     */
    @Column(nullable = false)
    public LocalDateTime createdAt = LocalDateTime.now();

    /**
     * When the decision was made
     */
    public LocalDateTime decidedAt;

    /**
     * Find pending proposal for a specific car
     */
    public static ApprovalProposal findPendingByCarNumber(Integer carNumber) {
        return find("carNumber = ?1 and status = ?2", carNumber, ApprovalStatus.PENDING).firstResult();
    }

    /**
     * Find all pending proposals
     */
    public static java.util.List<ApprovalProposal> findAllPending() {
        return find("status", ApprovalStatus.PENDING).list();
    }

    /**
     * Approval status enum
     */
    public enum ApprovalStatus {
        PENDING,
        APPROVED,
        REJECTED
    }
}

Create the ApprovalResource

This REST resource allows the UI to fetch pending approvals and submit decisions.

Create src/main/java/com/carmanagement/resource/ApprovalResource.java to create the following REST API endpoints:

  • GET /api/approvals/pending - Returns all pending approval proposals
  • POST /api/approvals/{id}/approve - Approve a proposal
  • POST /api/approvals/{id}/reject - Reject a proposal
ApprovalResource.java
package com.carmanagement.resource;

import com.carmanagement.model.ApprovalProposal;
import com.carmanagement.service.ApprovalService;
import jakarta.inject.Inject;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
import io.quarkus.logging.Log;

import java.util.List;
import java.util.Map;

/**
 * REST resource for managing approval proposals.
 * Provides endpoints for humans to view and approve/reject proposals.
 */
@Path("/api/approvals")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class ApprovalResource {

    @Inject
    ApprovalService approvalService;

    /**
     * Get all pending approval proposals.
     * This is called by the UI to display proposals awaiting human decision.
     */
    @GET
    @Path("/pending")
    public List<ApprovalProposal> getPendingProposals() {
        return approvalService.getPendingProposals();
    }

    /**
     * Get a specific proposal by ID.
     */
    @GET
    @Path("/{proposalId}")
    public Response getProposal(@PathParam("proposalId") Integer proposalId) {
        ApprovalProposal proposal = approvalService.getProposal(proposalId);
        if (proposal == null) {
            return Response.status(Response.Status.NOT_FOUND)
                    .entity(Map.of("error", "Proposal not found"))
                    .build();
        }
        return Response.ok(proposal).build();
    }

    /**
     * Approve a proposal.
     * This is called when a human clicks the "Approve" button in the UI.
     * 
     * @param proposalId The proposal ID
     * @param request Request body containing reason and approvedBy
     */
    @POST
    @Path("/{proposalId}/approve")
    public Response approveProposal(
            @PathParam("proposalId") Integer proposalId,
            Map<String, String> request) {

        try {
            String reason = request.getOrDefault("reason", "Approved by human reviewer");
            String approvedBy = request.getOrDefault("approvedBy", "Workshop User");

            Log.infof("Approval request received for proposal %d by %s", proposalId, approvedBy);

            ApprovalProposal proposal = approvalService.processDecision(
                    proposalId, true, reason, approvedBy);

            return Response.ok(proposal).build();
        } catch (IllegalArgumentException e) {
            return Response.status(Response.Status.NOT_FOUND)
                    .entity(Map.of("error", e.getMessage()))
                    .build();
        } catch (IllegalStateException e) {
            return Response.status(Response.Status.BAD_REQUEST)
                    .entity(Map.of("error", e.getMessage()))
                    .build();
        } catch (Exception e) {
            Log.error("Error approving proposal", e);
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
                    .entity(Map.of("error", "Error processing approval: " + e.getMessage()))
                    .build();
        }
    }

    /**
     * Reject a proposal.
     * This is called when a human clicks the "Reject" button in the UI.
     *
     * @param proposalId The proposal ID
     * @param request Request body containing reason and approvedBy
     */
    @POST
    @Path("/{proposalId}/reject")
    public Response rejectProposal(
            @PathParam("proposalId") Integer proposalId,
            Map<String, String> request) {

        try {
            String reason = request.getOrDefault("reason", "Rejected by human reviewer");
            String approvedBy = request.getOrDefault("approvedBy", "Workshop User");

            Log.infof("Rejection request received for proposal %d by %s", proposalId, approvedBy);

            ApprovalProposal proposal = approvalService.processDecision(
                    proposalId, false, reason, approvedBy);

            return Response.ok(proposal).build();
        } catch (IllegalArgumentException e) {
            return Response.status(Response.Status.NOT_FOUND)
                    .entity(Map.of("error", e.getMessage()))
                    .build();
        } catch (IllegalStateException e) {
            return Response.status(Response.Status.BAD_REQUEST)
                    .entity(Map.of("error", e.getMessage()))
                    .build();
        } catch (Exception e) {
            Log.error("Error rejecting proposal", e);
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
                    .entity(Map.of("error", "Error processing rejection: " + e.getMessage()))
                    .build();
        }
    }

    /**
     * Make a decision on a proposal with explicit KEEP_CAR or DISPOSE_CAR action.
     * Simplified approach where the UI directly specifies the desired outcome.
     *
     * @param proposalId The proposal ID
     * @param request Request body containing decision (KEEP_CAR or DISPOSE_CAR), reason, and approvedBy
     */
    @POST
    @Path("/{proposalId}/decide")
    public Response decideProposal(
            @PathParam("proposalId") Integer proposalId,
            Map<String, String> request) {

        try {
            String decision = request.get("decision"); // KEEP_CAR or DISPOSE_CAR
            String reason = request.getOrDefault("reason", "Decision by human reviewer");
            String approvedBy = request.getOrDefault("approvedBy", "Workshop User");

            if (decision == null || (!decision.equals("KEEP_CAR") && !decision.equals("DISPOSE_CAR"))) {
                return Response.status(Response.Status.BAD_REQUEST)
                        .entity(Map.of("error", "Decision must be either KEEP_CAR or DISPOSE_CAR"))
                        .build();
            }

            Log.infof("Decision '%s' received for proposal %d by %s", decision, proposalId, approvedBy);

            // Store the decision in the reason so the workflow can use it
            String fullReason = decision + ": " + reason;

            // We still use the approve/reject mechanism, but the decision is in the reason
            ApprovalProposal proposal = approvalService.processDecision(
                    proposalId, true, fullReason, approvedBy);

            return Response.ok(proposal).build();
        } catch (IllegalArgumentException e) {
            return Response.status(Response.Status.NOT_FOUND)
                    .entity(Map.of("error", e.getMessage()))
                    .build();
        } catch (IllegalStateException e) {
            return Response.status(Response.Status.BAD_REQUEST)
                    .entity(Map.of("error", e.getMessage()))
                    .build();
        } catch (Exception e) {
            Log.error("Error processing decision", e);
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR)
                    .entity(Map.of("error", "Error processing decision: " + e.getMessage()))
                    .build();
        }
    }
}

How the HITL Flow Works End-to-End

  1. The FleetSupervisorAgent detects a high-value vehicle and invokes the DispositionProposalAgent
  2. The proposal is passed to the HumanApprovalAgent, which is annotated with @HumanInTheLoop
  3. Inside the agent method, ApprovalService.createProposalAndWaitForDecision() persists the proposal to the database and returns a CompletableFuture
  4. The agent method blocks on future.get(5, TimeUnit.MINUTES) — the workflow pauses here
  5. The UI polls GET /api/approvals/pending and displays the proposal to the human reviewer
  6. The human clicks Approve or Reject, which calls the corresponding REST endpoint
  7. ApprovalService.processDecision() completes the CompletableFuture with the decision
  8. The agent method unblocks, formats the decision, and returns it
  9. The workflow resumes with the human’s decision

Update the FleetSupervisorAgent

Now we need to modify the supervisor to implement the new value-based routing with the approval workflow.

Update src/main/java/com/carmanagement/agentic/agents/FleetSupervisorAgent.java:

FleetSupervisorAgent.java
package com.carmanagement.agentic.agents;

import com.carmanagement.model.CarInfo;
import com.carmanagement.model.FeedbackAnalysisResults;
import dev.langchain4j.agentic.declarative.SupervisorAgent;
import dev.langchain4j.agentic.declarative.SupervisorRequest;

/**
 * Supervisor agent that orchestrates the entire car processing workflow.
 * Coordinates feedback analysis agents and action agents based on car condition.
 * Implements human-in-the-loop pattern for high-value vehicle dispositions.
 */
public interface FleetSupervisorAgent {

    @SupervisorAgent(
        outputKey = "supervisorDecision",
        subAgents = {
            PricingAgent.class,
            DispositionProposalAgent.class,
            HumanApprovalAgent.class,
            DispositionAgent.class,
            MaintenanceAgent.class,
            CleaningAgent.class
        }
    )
    String superviseCarProcessing(
        CarInfo carInfo,
        Integer carNumber,
        String feedback,
        FeedbackAnalysisResults feedbackAnalysisResults
    );

    @SupervisorRequest()
    static String request(
        CarInfo carInfo,
        Integer carNumber,
        String feedback,
        FeedbackAnalysisResults feedbackAnalysisResults
    ) {
        boolean dispositionRequired = feedbackAnalysisResults.dispositionAnalysis() != null &&
                                     feedbackAnalysisResults.dispositionAnalysis().toUpperCase().contains("DISPOSITION_REQUIRED");

        String noDispositionMessage = """
            Disposition is not required. 
            Proceed with normal maintenance and cleaning workflow. 
            If cleaning or maintenance is required, invoke the appropriate agents.
                """;

        String dispositionMessage = """
           DISPOSITION_REQUIRED

           Follow these steps:

           1. Get value from PricingAgent (keep $ format)
           2. IF value > $15,000 (HIGH-VALUE):
              - Invoke DispositionProposalAgent → HumanApprovalAgent (workflow pauses)
              - APPROVED: Use AI recommendation → KEEP→"KEEP_CAR", DISPOSE→"DISPOSE_CAR"
              - REJECTED: Opposite of AI → KEEP→"DISPOSE_CAR", DISPOSE→"KEEP_CAR"
           3. IF value ≤ $15,000 (LOW-VALUE):
              - Invoke DispositionAgent directly
              - KEEP→"KEEP_CAR", SCRAP/SELL/DONATE→"DISPOSE_CAR"
           4. IF "KEEP_CAR": Invoke MaintenanceAgent/CleaningAgent as needed

           CRITICAL: End with KEEP_CAR or DISPOSE_CAR
           """;

        return """
            You are a fleet supervisor for a car rental company. You coordinate action agents based on feedback analysis.

            The feedback has already been analyzed and you have these inputs:
            - cleaningAnalysis: What cleaning is needed (or "CLEANING_NOT_REQUIRED")
            - maintenanceAnalysis: What maintenance is needed (or "MAINTENANCE_NOT_REQUIRED")
            - dispositionAnalysis: Whether severe damage requires disposition (or "DISPOSITION_NOT_REQUIRED")

            Your job is to invoke the appropriate ACTION agents for this car

            Car: """ + carInfo.year + " " + carInfo.make + " " + carInfo.model + " (#" + carNumber + ")" + """

            Current Condition: """ + carInfo.condition + """

            Feedback: """ + feedback + """

            Cleaning Analysis: """ + feedbackAnalysisResults.cleaningAnalysis() + """

            Maintenance Analysis: """ + feedbackAnalysisResults.maintenanceAnalysis() + """

            Disposition Analysis: """ + (dispositionRequired ? dispositionMessage : noDispositionMessage);
    }
}

Update the CarConditions Model

We also need to add the approval tracking fields to the data model.

Update src/main/java/com/carmanagement/model/CarConditions.java:

CarConditions.java
package com.carmanagement.model;

/**
 * Record representing the conditions of a car.
 *
 * @param generalCondition    A description of the car's general condition
 * @param carAssignment       Indicates the action required (DISPOSITION, MAINTENANCE, CLEANING, or NONE)
 * @param dispositionStatus   Status of disposition decision (DISPOSITION_APPROVED, DISPOSITION_REJECTED, or DISPOSITION_NOT_REQUIRED)
 * @param dispositionReason   Reason for disposition decision
 */
public record CarConditions(
    String generalCondition,
    CarAssignment carAssignment,
    String dispositionStatus,
    String dispositionReason
) {
    /**
     * Constructor for backward compatibility without disposition fields.
     */
    public CarConditions(String generalCondition, CarAssignment carAssignment) {
        this(generalCondition, carAssignment, "DISPOSITION_NOT_REQUIRED", null);
    }
}

Update Application Configuration

And finally, add a new configuration value for the approval threshold so we can update it more easily when management changes their mind again.

Update src/main/resources/application.properties:

# Human-in-the-Loop configuration
# Threshold for requiring human approval on high-value dispositions
car-management.approval.threshold=15000

Try the Complete Solution

Now let’s see the Human-in-the-Loop pattern in action!

Start the Application

  1. Navigate to the step-05 directory:
cd section-2/step-05
  1. Start the application:
./mvnw quarkus:dev

mvnw quarkus:dev
  1. Open http://localhost:8080

Test HITL Scenarios

Try these scenarios to see how the approval workflow handles different vehicle values:

Scenario 1: High-Value Vehicle Requiring Approval

Enter the following text in the feedback field for the Honda Civic:

The car was in a serious collision. Front end is completely destroyed and airbags deployed.

What happens:

flowchart TD
    Start(["Input: Serious collision<br/>Front end destroyed"])

    Start --> FW["FeedbackAnalysisWorkflow<br/>Produces: FeedbackAnalysisResults"]

    FW --> FSA["FleetSupervisorAgent<br/>Orchestration"]
    FSA --> PA["PricingAgent"]
    PA --> Value["Estimate: ~$18,000<br/>2020 Honda Civic"]

    Value --> Check{"Value > $15,000?"}
    Check -->|Yes| Proposal["DispositionProposalAgent<br/>Creates Proposal"]
    Proposal --> PropResult["Proposed: SCRAP<br/>Reasoning: Severe damage"]

    PropResult --> Human["HumanApprovalAgent<br/>@HumanInTheLoop pauses workflow"]
    Human --> Decision{Decision}
    Decision -->|APPROVED| Execute["Execute SCRAP"]
    Decision -->|REJECTED| Fallback["Route to Maintenance"]

    Execute --> Result(["Status: PENDING_DISPOSITION<br/>Approval: APPROVED"])
    Fallback --> Result2(["Status: IN_MAINTENANCE<br/>Approval: REJECTED"])

    style FW fill:#FAE5D3,stroke:#333,stroke-width:2,color:#333
    style FSA fill:#D5F5E3,stroke:#333,stroke-width:2,color:#333
    style PA fill:#F9E79F,stroke:#333,stroke-width:2,color:#333
    style Proposal fill:#FFD700,stroke:#333,stroke-width:2,color:#000
    style Human fill:#FF6B6B,stroke:#333,stroke-width:2,color:#fff
    style Check fill:#FFA07A,stroke:#333,stroke-width:2,color:#000
    style Decision fill:#FFA07A,stroke:#333,stroke-width:2,color:#000
    style Result fill:#D2B4DE,stroke:#333,stroke-width:2,color:#333
    style Result2 fill:#D2B4DE,stroke:#333,stroke-width:2,color:#333
Hold "Alt" / "Option" to enable pan & zoom

Expected Result:

  • PricingAgent estimates value at ~$18,000 (above threshold)
  • DispositionProposalAgent creates SCRAP proposal
  • HumanApprovalAgent pauses the workflow (via @HumanInTheLoop) and waits for human input
  • Human reviews the proposal in the UI and clicks Approve or Reject
  • Workflow resumes with the decision
  • Status: PENDING_DISPOSITION if approved, IN_MAINTENANCE if rejected

Scenario 2: High-Value Vehicle - Approval Rejected

Enter the following text in the Mercedes Benz feedback field:

Minor fender bender, small dent in rear bumper

What happens:

flowchart TD
    Start(["Input: Minor fender bender<br/>small dent"])

    Start --> FW["FeedbackAnalysisWorkflow<br/>Produces: FeedbackAnalysisResults"]

    FW --> FSA["FleetSupervisorAgent"]
    FSA --> PA["PricingAgent"]
    PA --> Value["Estimate: ~$25,000<br/>2021 Mercedes Benz"]

    Value --> Check{"Value > $15,000?"}
    Check -->|Yes| Proposal["DispositionProposalAgent<br/>Creates Proposal"]
    Proposal --> PropResult["Proposed: SELL or KEEP<br/>Minor damage"]

    PropResult --> Human["HumanApprovalAgent<br/>@HumanInTheLoop pauses workflow"]
    Human --> Decision["Decision: REJECTED<br/>Too valuable for minor damage"]

    Decision --> Fallback["Route to Maintenance<br/>Repair instead"]
    Fallback --> Result(["Status: IN_MAINTENANCE<br/>Approval: REJECTED"])

    style FW fill:#FAE5D3,stroke:#333,stroke-width:2,color:#333
    style FSA fill:#D5F5E3,stroke:#333,stroke-width:2,color:#333
    style PA fill:#F9E79F,stroke:#333,stroke-width:2,color:#333
    style Proposal fill:#FFD700,stroke:#333,stroke-width:2,color:#000
    style Human fill:#FF6B6B,stroke:#333,stroke-width:2,color:#fff
    style Check fill:#FFA07A,stroke:#333,stroke-width:2,color:#000
    style Result fill:#D2B4DE,stroke:#333,stroke-width:2,color:#333
Hold "Alt" / "Option" to enable pan & zoom

Expected Result:

  • PricingAgent estimates value at ~$25,000 (high value)
  • DispositionProposalAgent suggests SELL or KEEP
  • HumanApprovalAgent pauses workflow, human REJECTS (too valuable for disposition with minor damage)
  • Fallback: Routes to MaintenanceAgent instead
  • Status: IN_MAINTENANCE
  • Disposition status: DISPOSITION_REJECTED with reasoning

Scenario 3: Low-Value Vehicle - No Approval Needed

Enter the following text in the Ford F-150 feedback field (status: In Maintenance) in the Fleet Status grid:

The truck is totaled, completely inoperable, very old

What happens:

flowchart TD
    Start(["Input: Totaled truck<br/>very old"])

    Start --> FW["FeedbackAnalysisWorkflow<br/>Produces: FeedbackAnalysisResults"]

    FW --> FSA["FleetSupervisorAgent"]
    FSA --> PA["PricingAgent"]
    PA --> Value["Estimate: ~$8,000<br/>2019 Ford F-150, totaled"]

    Value --> Check{"Value > $15,000?"}
    Check -->|No| Direct["DispositionAgent<br/>Direct Decision"]
    Direct --> Decision["Decision: SCRAP<br/>Beyond economical repair"]

    Decision --> Result(["Status: PENDING_DISPOSITION<br/>Approval: NOT_REQUIRED"])

    style FW fill:#FAE5D3,stroke:#333,stroke-width:2,color:#333
    style FSA fill:#D5F5E3,stroke:#333,stroke-width:2,color:#333
    style PA fill:#F9E79F,stroke:#333,stroke-width:2,color:#333
    style Direct fill:#87CEEB,stroke:#333,stroke-width:2,color:#000
    style Check fill:#FFA07A,stroke:#333,stroke-width:2,color:#000
    style Result fill:#D2B4DE,stroke:#333,stroke-width:2,color:#333
Hold "Alt" / "Option" to enable pan & zoom

Expected Result:

  • PricingAgent estimates value at ~$8,000 (below threshold)
  • Skips approval workflow entirely (low value)
  • DispositionAgent makes direct SCRAP decision
  • Status: PENDING_DISPOSITION
  • Disposition status: DISPOSITION_NOT_REQUIRED

Check the Logs

Watch the console output to see the approval workflow execution:

FeedbackAnalysisWorkflow executing...
  |- FeedbackAnalysisAgent(disposition): DISPOSITION_REQUIRED
FleetSupervisorAgent orchestrating...
  |- PricingAgent: Estimated value $18,000
  |- Value check: $18,000 > $15,000 -> Approval required
  |- DispositionProposalAgent: Proposed SCRAP
  |- HumanApprovalAgent (@HumanInTheLoop): Workflow paused...
  |- Waiting for human decision via UI...
  |- Human decision received: APPROVED
  |- Workflow resumed
CarConditionFeedbackAgent updating...
  |- Disposition status: DISPOSITION_APPROVED

Notice how the workflow pauses at the HumanApprovalAgent and only resumes after the human makes a decision in the UI.

Recapping the HITL pattern

The HITL pattern provides a safety net for autonomous systems. Human review catches edge cases that AI might miss, preventing costly mistakes. It also builds trust by allowing a gradual transition from manual to autonomous operations, while maintaining clear human accountability for critical decisions.

Many industries require human oversight by regulation or policy. Financial services need approval for large transactions. Healthcare requires physician review of treatment decisions. Legal departments need lawyers to approve contract terms. Beyond compliance, audit trails that track who approved what and when are essential for accountability.

The real power of HITL is that you can tune the automation level over time:

graph LR
    A["Fully Manual"] --> B["HITL - High Threshold"]
    B --> C["HITL - Low Threshold"]
    C --> D["Fully Autonomous"]

    style A fill:#FF6B6B,stroke:#333,stroke-width:2,color:#fff
    style B fill:#FFD700,stroke:#333,stroke-width:2,color:#000
    style C fill:#87CEEB,stroke:#333,stroke-width:2,color:#000
    style D fill:#90EE90,stroke:#333,stroke-width:2,color:#000
Hold "Alt" / "Option" to enable pan & zoom

Start with a low threshold where everything requires approval. As confidence grows, gradually increase the threshold so only higher-value decisions need human review. Eventually, routine cases can run fully autonomous while high-stakes decisions still go through HITL.

Experiment Further

  1. Try adjusting the approval threshold to see how it affects which vehicles require human review. Lower it to $10,000 to require approval for more vehicles, raise it to $25,000 to only catch the most expensive ones, or set it to $0 to require approval for all dispositions.

  2. You could implement multi-level approval where vehicles worth \(15,000-\)25,000 need a single approver, \(25,000-\)50,000 need two approvers, and anything over $50,000 requires manager approval.

  3. Add monitoring to track approval rates by value range, average approval time, rejection reasons, and approver performance. Implement approval timeouts that auto-reject after 24 hours, escalate to a manager after 48 hours, or send reminder notifications. Build an approval history that tracks who approved or rejected each decision, when they made it, what reasoning they provided, and what the outcome was.

Troubleshooting

All vehicles going through approval workflow

Check that the value threshold is correctly configured in application.properties and that the PricingAgent is returning numeric values that can be compared.

Workflow not pausing for human approval

Verify that:

  • The HumanApprovalAgent has the @HumanInTheLoop annotation
  • The ApprovalService is correctly creating the CompletableFuture
  • The agent method is calling .get() on the future to block
Approval status not being tracked

Verify that:

  • FleetSupervisorAgent stores disposition status in AgenticScope
  • CarConditions model has the dispositionStatus and dispositionReason fields
  • CarProcessingWorkflow retrieves these values from the scope
Low-value vehicles still requiring approval

Check the value comparison logic in FleetSupervisorAgent. Ensure the PricingAgent output is being parsed correctly as a number.

Timeout errors when waiting for approval

The HumanApprovalAgent has a 5-minute timeout by default. If you need more time, adjust the timeout value in the .get(5, TimeUnit.MINUTES) call. On timeout, the system defaults to REJECTED for safety.

Agent Observability with MonitoredAgent

Beyond the HITL workflow, step-05 also introduces agent observability. This gives you the ability to inspect what every agent in the system did, what inputs it received, what it produced, and how long it took.

LangChain4j provides the MonitoredAgent interface and an HtmlReportGenerator utility in the dev.langchain4j.agentic.observability package. Together, they give you a full execution report of your agentic system with zero manual instrumentation.

The MonitoredAgent interface has a single method that returns an AgentMonitor. When your top-level workflow interface extends MonitoredAgent, LangChain4j automatically attaches an AgentMonitor listener to the entire agent tree. This monitor implements AgentListener and records every agent invocation across the system.

Before each invocation, it captures the agent name, inputs, and start time. After each invocation, it captures the output and finish time. On errors, it captures exception details. For nested invocations, it tracks the full call hierarchy, such as when FleetSupervisorAgent calls PricingAgent which calls DispositionProposalAgent. The monitor groups executions by memory ID so you can inspect each independent workflow run separately.

Enabling this feature requires two simple steps. First, make your workflow interface extend MonitoredAgent. In CarProcessingWorkflow.java, the interface simply extends MonitoredAgent with no other changes needed. The framework handles everything automatically without requiring annotations on individual agents or manual tracking code.

Second, generate an HTML report from the monitor. In CarManagementService.java, the report() method uses the static HtmlReportGenerator.generateReport() helper to produce a self-contained HTML page. This page includes an agent topology showing a visual map of all agents and their relationships, an execution timeline with a detailed breakdown of every agent invocation including inputs, outputs, duration, and nesting level, and error tracking that highlights any failed invocations with their exception details.

The report is exposed via a REST endpoint in CarManagementResource.java. After processing one or more cars, click the “Generate Report” button in the UI (next to “Refresh Data”) to open the report in a new tab. The report shows the full agent topology of your system, every execution grouped by workflow run, and for each agent invocation, what went in, what came out, and how long it took. This is invaluable for debugging agent behavior, understanding why the supervisor made a particular routing decision, or verifying that the HITL workflow paused and resumed correctly.

What’s Next?

You’ve successfully implemented the Human-in-the-Loop pattern for safe, controlled autonomous decision-making. The system now routes high-value vehicles through human approval using LangChain4j’s @HumanInTheLoop annotation, creates proposals for human review via the DispositionProposalAgent, and pauses workflow execution in the HumanApprovalAgent until a human decides. It tracks approval decisions for audit trails, provides fallback paths for rejected proposals, and balances automation with human oversight.

In Step 06, you’ll learn about multimodal image analysis — allowing employees to upload car photos during rental returns, so the system can automatically enrich feedback with visual observations using a multimodal AI agent!

Continue to Step 06 - Multimodal Image Analysis