Relay Dispatch Job Repository

Overview

The Relay Dispatch Repository provides a persistent, transactional job queue for asynchronous tasks delegated from the API gateway to a backend "relay" component. It uses the s2r_relay_dispatch database table to manage the lifecycle of these jobs.

This component solves the problem of decoupling the gateway from backend services for operations that may be long-running. Instead of holding a request open, the gateway can enqueue a job, return an identifier to the API consumer, and allow a separate relay process to execute the work and post the result.

The repository manages the flow of jobs through several states:

1. Enqueue: The API gateway creates a job in pending status.
2. Claim: A relay process polls for and claims pending jobs, transitioning them to claimed.
3. Complete: After executing the task, the relay reports the outcome, transitioning the job to a terminal state of succeeded or failed.
4. Poll: The gateway can poll the job's status to track its progress.
5. Expire: Jobs that are not completed before their deadline can be marked as expired.

Job Lifecycle

A dispatch job progresses through a well-defined set of states, managed by the methods of the repository. The primary state transitions are from pending to claimed, and finally to a terminal state (succeeded, failed, or expired).

stateDiagram-v2
    direction LR
    [*] --> pending: enqueue()
    pending --> claimed: claim()
    pending --> expired: markExpired()

    claimed --> succeeded: complete()
    claimed --> failed: complete()
    claimed --> expired: markExpired()

    succeeded --> [*]
    failed --> [*]
    expired --> [*]

How It Works

The repository orchestrates the job lifecycle through a series of database operations. The interaction between the gateway, the repository, and the relay follows a specific sequence.

sequenceDiagram
    participant Gateway
    participant Relay Dispatch Repository
    participant Relay

    Gateway->>Relay Dispatch Repository: 1. enqueue(job details)
    note right of Gateway: Creates a 'pending' job
    Relay Dispatch Repository-->>Gateway: job_id

    loop Poll for work
        Relay->>Relay Dispatch Repository: 2. claim(relay_id)
    end
    note left of Relay: Atomically claims oldest 'pending' jobs
    Relay Dispatch Repository-->>Relay: List of DispatchJob

    note left of Relay: Executes the task defined in the job
    Relay->>Relay Dispatch Repository: 3. complete(job_id, result)
    note right of Gateway: Updates job to 'succeeded' or 'failed'

    loop Poll for result
        Gateway->>Relay Dispatch Repository: 4. pollStatus(job_id)
    end
    Relay Dispatch Repository-->>Gateway: DispatchResult

Tenancy and Security

All repository operations are strictly scoped to a single tenant to ensure data isolation in a multi-tenant environment.

• Tenant Context: Before any repository method is called, the system must establish a tenant context for the database connection (e.g., by setting app.tenant_id). All subsequent SQL queries within the repository are filtered by this bound tenant ID.
• SQL Predicates: Operations include an explicit SQL predicate (DISPATCH_TENANT_SCOPE_SQL) that enforces tenant boundaries. This acts as a defense-in-depth measure, ensuring that even a connection with elevated privileges cannot access another tenant's jobs. This predicate also requires that a tenant context is bound, preventing operations from unintentionally targeting a default tenant.
• Relay Isolation: The claim and complete methods are pinned to a specific relay_id in addition to the tenant ID. This prevents one relay from claiming or completing jobs intended for a different relay, even if they both belong to the same tenant.

Concurrency and Idempotency

The repository is designed to support concurrent relay workers and idempotent operations.

• Concurrency: The claim method uses a FOR UPDATE SKIP LOCKED clause in its SQL query. This allows multiple relay instances to poll the s2r_relay_dispatch table simultaneously. When one relay worker locks a set of rows to claim them, other workers will skip those locked rows and attempt to claim the next available ones, preventing race conditions and ensuring no job is processed more than once.
• Idempotency: The complete method is idempotent. It will only update a job if its current status is claimed. If a relay attempts to complete a job that has already been completed (or expired), the operation affects zero rows and is treated as a no-op.

API Reference

The RelayDispatchRepository exposes methods for managing the entire lifecycle of a dispatch job.

enqueue

Enqueues a new dispatch job in the pending state. This is typically called by the gateway when it receives a request to be processed asynchronously.

Returns: UUID - The unique identifier generated for the new job.

claim

Atomically claims one or more pending jobs for a specific relay, transitioning their status to claimed. Jobs are claimed in the order they were created (oldest first).

Returns: List<DispatchJob> - A list of the jobs that were successfully claimed. The list is empty if no pending jobs were available for the specified relay.

complete

Records the final result of a claimed job, transitioning its status to succeeded or failed.

Returns: int - The number of rows updated. Returns 1 on a successful completion or 0 if the job was not found, was not in the claimed state, or did not belong to the calling relay.

pollStatus

Retrieves the current status and result of a job. This is used by the gateway-side coordinator to monitor the job's progress.

Returns: Optional<DispatchResult> - An Optional containing the job's result if found, or an empty Optional if no job with the given ID exists for the current tenant.

markExpired

Transitions a non-terminal (pending or claimed) job to the expired state. This is a best-effort cleanup action typically called by the gateway-side coordinator when a job's expiresAt deadline has passed.

Returns: int - The number of rows updated. Returns 1 if the job was successfully marked as expired, or 0 if the job had already reached a terminal state.

Data Structures

The repository uses two primary record types to represent job data.

DispatchJob

Represents a job that has been claimed by a relay and is ready for execution. This is the data returned by the claim method.

DispatchResult

Represents a snapshot of a job's status and final result. This is the data returned by the pollStatus method.

The isTerminal() method on a DispatchResult returns true if the status is succeeded, failed, or expired.

Troubleshooting

Unbound Tenant Context

All repository methods require a tenant context to be bound to the database connection. If the context is missing, operations will fail. The source indicates this may result in an S2R-TENANCY-UNBOUND error. Ensure that the system correctly resolves and binds the tenant ID before calling any repository methods.

complete() or markExpired() Affects 0 Rows

If a call to complete() or markExpired() returns 0, it indicates that no database rows were updated. This is not necessarily an error and can occur under the following conditions:

• The job ID does not exist for the current tenant.
• The job was already in a terminal state (succeeded, failed, or expired).
• For complete(), the job's status was not claimed, or the relayId provided did not match the one that originally claimed the job.

Invalid JSON in Headers

The requestHeaders and responseHeaders maps are serialized to and from JSON. If the map contains data that cannot be serialized, enqueue or complete will throw an IllegalArgumentException. Similarly, if the JSON data in the database is malformed, claim or pollStatus will throw an IllegalArgumentException during deserialization.

See Also

• SaaS Multi-Tenant Architecture: For more context on the overall tenancy model and the role of components like the relay.

All SOAP-to-REST docs