Runtime Route Caching

Overview

The Runtime Route Cache Service is an in-memory caching layer that accelerates API request routing. It solves the problem of high latency and database load by caching the mapping between an incoming request's attributes (such as its HTTP method and path) and the configured backend route.

This service acts as an intermediary between the primary request handler and the route definition database. When a request arrives, the system queries this service first. If a valid cached route exists, it is returned immediately, avoiding a database query. If not, the service fetches candidate routes from the database, determines the correct match, and stores the result in the cache for subsequent requests.

The service maintains two levels of caching:

1. A cache of fully resolved routes.
2. A cache of candidate route lists, pre-filtered by environment, HTTP method, and direction.

This component is essential for system performance, especially in high-traffic environments.

How it Works

The route resolution process follows a specific sequence of operations to find a matching route for an incoming request, using an in-memory cache to optimize performance.

flowchart TD
    subgraph "Route Resolution Flow"
        A[Request Handler] -- "resolveRoute()" --> B{Check Resolved Route Cache};
        B -- "Hit" --> G[Return Cached Route];
        B -- "Miss" --> C{Check Candidate List Cache};
        C -- "Hit" --> E[Use Cached Candidate List];
        C -- "Miss" --> D[Query Database for Candidates];
        D -- "Routes" --> F[Sort & Store Candidate List];
        F --> E;
        E -- "Candidates" --> H[Perform Path & SOAPAction Matching];
        H -- "Resolved Route" --> I[Store in Resolved Route Cache];
        I --> G;
    end
    G -- "RouteMetadata" --> J[Request Processing Continues];

Route Resolution Sequence

1. Cache Key Generation: The service first constructs a unique cache key for the request. The key is a composite of the current tenant ID, direction (PUBLISH or CONSUME), environment, HTTP method, request path, and the normalized SOAPAction header value (if present).

   IMPORTANT

   The cache is tenant-aware. Each tenant's routes are cached separately to ensure strict data isolation. Requests without a tenant context are assigned a special unbound scope and cannot access tenant-specific cache entries.

2. Primary Cache Lookup: The service checks for a matching entry in the primary resolved route cache (cache). If a non-expired entry is found, it is returned immediately, and the process ends.

3. Candidate List Lookup: On a primary cache miss, the service checks a secondary cache (routeListCache) that stores lists of potential candidate routes. This cache is keyed by tenant, direction, environment, and HTTP method. If a valid list is found, it is used for the next step.

4. Database Query: If the candidate list is not cached, the service queries the database via RuntimeRouteRepository.findRoutes() to load all routes matching the request's environment, method, and direction. It then attaches the associated backend target configurations to each route.

5. Candidate Sorting: The loaded candidates are sorted to ensure that the most specific routes are evaluated first. The sorting order is:

   1. Descending number of static path segments (e.g., /a/b/{id} comes before /a/{id1}/{id2}).
   2. Descending total number of path segments.
   3. Alphabetical order of the route path template (restPath).

6. Candidate Caching: The sorted, immutable list of candidates is stored in the routeListCache with the configured Time-to-Live (TTL).

7. Final Resolution: The service iterates through the sorted candidate list to find a match.

   • Path Matching: It compares the incoming request path against each candidate's restPath template. It supports path parameters (e.g., {userId}).
   • Disambiguation:
     • If exactly one candidate's path matches, that route is chosen.
     • If multiple candidates' paths match (ambiguity), the behavior depends on the service direction:
       • For CONSUME services, it attempts a secondary match using the SOAPAction header from the request against the soapAction field of the candidate routes. If a unique match is found, that route is chosen.
       • If no SOAPAction match is found, or if the direction is PUBLISH, the system defaults to the first route in the sorted list. A warning is logged in ambiguous CONSUME scenarios.

8. Result Caching: The final ResolvedRoute (or an empty result if no match was found) is stored in the primary cache with the configured TTL.

Cache Invalidation

The service provides a mechanism to clear all its internal caches.

• The invalidateAll() method completely clears both the resolved route cache and the candidate list cache.
• This action also triggers a corresponding invalidation of the OpenAPI Schema Validator cache, ensuring that schema changes are picked up along with route changes.

This is necessary to force the system to reload route definitions from the database after configuration changes are made.

SOAPAction Normalization

To ensure consistent matching, the value of the SOAPAction HTTP header is normalized before being used in cache keys or comparisons. The normalization process involves:

1. Trimming leading and trailing whitespace.
2. Removing surrounding double-quote (") characters.
3. An empty or blank value is treated as null.

Configuration

The cache's Time-to-Live (TTL) is configurable. The system uses the first value it finds in the following order of precedence:

1. Database Setting: A value set in the settings table with the key runtime_route_cache_ttl_seconds.
2. Environment Variable: The S2R_RUNTIME_ROUTE_CACHE_TTL_SECONDS environment variable.
3. Code Default: A hardcoded default value.

The service periodically refreshes the TTL from the database every 60 seconds, allowing for dynamic adjustments without requiring a service restart.

API Reference

The RuntimeRouteCacheService exposes the following core methods for use by other internal components.

resolveRoute

Resolves an incoming request to a configured route. There are two overloads for this method.

resolveRoute(String direction, String environment, String method, String path)

Resolves a route without considering the SOAPAction header.

• Returns: Optional<ResolvedRoute> containing the matched route and extracted path parameters, or an empty Optional if no match is found.

resolveRoute(String direction, String environment, String method, String path, String soapAction)

Resolves a route with optional disambiguation using the SOAPAction header. This is primarily used for CONSUME services where multiple operations might share the same endpoint URL.

• Returns: Optional<ResolvedRoute> containing the matched route and extracted path parameters, or an empty Optional if no match is found.

invalidateAll

public void invalidateAll()

Immediately purges all entries from all internal caches, including the resolved route cache, the candidate list cache, and the linked OpenAPI validator cache. Subsequent requests will trigger a full reload from the database.

Troubleshooting

Ambiguous Route Resolution for CONSUME Services

The system logs a warning when it cannot unambiguously resolve a route for a CONSUME service.

• Log Code: S2R-RUN-0412
• Log Message: ambiguous-route-soapaction direction={} path={} soapAction={} candidates={} — ...
• Condition: An incoming request to a CONSUME service matches the path template (restPath) of multiple configured routes.
• Trigger: This occurs when either:
  1. The SOAPAction HTTP header is missing from the request.
  2. The SOAPAction header is present, but its value does not match the soapAction field of any of the matching candidate routes.
• Impact: The service falls back to selecting the first matching route based on the standard candidate sorting order. This may result in the request being processed by an unintended operation. The log message indicates which fallback was taken.

See Also

• OpenAPI Schema Validation: The route cache invalidation is linked to the schema validation cache.
• Service Directionality: The concept of PUBLISH and CONSUME directions is fundamental to how routes are fetched and cached.

All SOAP-to-REST docs