Runtime System Settings Cache

Overview

The RuntimeSettingRepository is a component responsible for reading operator-configurable system settings from the s2r_system_setting database table. Its primary purpose is to improve performance and reduce database load under high request volumes.

During the processing of a single API request, the runtime may need to resolve 15-20 different system settings. Querying the database for each setting on every request can create a significant performance bottleneck, leading to thousands of extra queries per second at high traffic rates.

To solve this, RuntimeSettingRepository implements a hot-path, time-to-live (TTL) cache. It stores a local, in-memory snapshot of settings, drastically reducing database round-trips. This component provides low-precedence default values to other parts of the request-processing pipeline, such as the ConversionOptionResolver. Per-operation overrides, which are loaded with a route, still take precedence over the system defaults provided by this repository.

How It Works

The repository caches the raw JSON text of a setting's value, not the parsed result. This allows for a flexible and efficient caching strategy that supports various data types. The caching mechanism is designed to be resilient, thread-safe, and aware of multi-tenant environments.

flowchart TD
    subgraph "Setting Accessor"
        A[Call for setting 'key']
    end
    
    A --> B{Value in cache for 'key'?}
    B -- "Yes, and not expired" --> C[Return cached value]
    
    subgraph "Cache Refresh"
        B -- "No, or expired" --> D[Query database for setting]
        D -- Success --> E[Update cache with new value and TTL]
        E --> C
        D -- "DB Error" --> F{Stale value in cache?}
        F -- "Yes" --> G[Serve stale value]
        G --> H[Extend stale value's TTL]
        H --> I[Log WARN]
        I --> C
        F -- "No" --> J[Throw exception]
        J --> K[Log WARN]
    
        subgraph "Special Case: Unbound Tenant"
            D -- "DB Error (S2R-TENANCY-UNBOUND)" --> L[Throw exception]
            L --> M[Remove key from cache]
            M --> N[Log WARN]
        end
    end

Cache TTL and Propagation

Settings are cached for a configurable duration, known as the Time-To-Live (TTL).

• Default TTL: 60 seconds.
• Configuration: The TTL can be changed via the S2R_RUNTIME_SETTING_CACHE_TTL_SECONDS environment variable.
• Propagation Delay: Any change made to a system setting will propagate to the runtime within one TTL window. This is consistent with the platform's accepted propagation delay for other global settings.

Tenant Isolation

The cache is designed for multi-tenant deployments and ensures that one tenant cannot access another's settings.

• Cache Key: The key used for caching is a combination of the current tenant's ID and the setting's key: (tenantId, settingKey).
• Single-Tenant Behavior: In a single-tenant deployment, the tenant ID is a constant value, and the cache behaves as a simple key-value store.
• Unbound Callers: A request without a tenant context is given a special unbound scope. This prevents it from being served a cached value belonging to the default tenant, forcing a database check which may be gated by security policies.

Concurrency and Thread Safety

The cache is implemented using a ConcurrentHashMap of immutable Snapshot records, ensuring high performance in multi-threaded environments.

• Non-Blocking Reads: Reads from the cache never lock.
• Atomic Updates: When a setting is refreshed, the old snapshot is atomically replaced with the new one.
• Stale Reads: If a request for a setting triggers a refresh, concurrent requests for the same setting will receive the existing stale-but-present value while the refresh completes. This avoids lock contention during updates.

Fail-Safe Behavior

The repository is designed to be resilient to transient database failures.

• Stale on Error: If a database query fails during a cache refresh, the repository will continue to serve the last-known-good (stale) value if one exists. The TTL of the stale entry is extended to prevent hammering a struggling database with refresh attempts on every subsequent request. A WARN level message is logged.
• No Cached Value: If a refresh fails and there is no previously cached value, the exception is thrown, and the calling component must handle the error.

Configuration

The cache's behavior can be tuned with the following environment variable.

API Reference

The RuntimeSettingRepository exposes several public accessor methods for retrieving settings of different types. These methods handle the logic of checking the cache, querying the database, and parsing the raw value.

getJsonStringList

Reads a field from a JSON object stored as a setting. The field's value is expected to be a JSON array of strings.

public List<String> getJsonStringList(String key, String field)

• Parameters:
  • key: The setting_key of the system setting.
  • field: The name of the field within the JSON object to read.
• Returns: A List<String> containing the elements of the JSON array. Returns null if the setting does not exist, the field is missing, or the value is not a JSON array.

Example

To retrieve a denylist of headers from a setting named conversion_defaults_header_propagation:

// Assuming the setting value is: {"headerDenylist": ["Security", "Authorization"]}
List<String> denylist = getJsonStringList("conversion_defaults_header_propagation", "headerDenylist");
// denylist will contain ["Security", "Authorization"]

getJsonStringField

Reads a field from a JSON object stored as a setting and returns its string representation.

public String getJsonStringField(String key, String field, String defaultValue)

• Parameters:
  • key: The setting_key of the system setting.
  • field: The name of the field within the JSON object to read.
  • defaultValue: The value to return if the setting or field is not found.
• Returns: The string value of the field. If the field contains a JSON object or array, its JSON text is returned. Returns defaultValue if the setting, field, or value is missing or null.

getInt

Reads a setting whose value is a single integer.

public int getInt(String key, int defaultValue)

• Parameters:
  • key: The setting_key of the system setting.
  • defaultValue: The value to return if the setting is not found or its value cannot be parsed as an integer.
• Returns: The int value of the setting or defaultValue.

getLong

Reads a setting whose value is a single long integer.

public long getLong(String key, long defaultValue)

• Parameters:
  • key: The setting_key of the system setting.
  • defaultValue: The value to return if the setting is not found or its value cannot be parsed as a long.
• Returns: The long value of the setting or defaultValue. If parsing fails, a WARN message is logged.

getBoolean

Reads a setting whose value is a single boolean.

public boolean getBoolean(String key, boolean defaultValue)

• Parameters:
  • key: The setting_key of the system setting.
  • defaultValue: The value to return if the setting is not found or its value is not a recognized boolean.
• Returns: The boolean value of the setting. The strings "true" (case-insensitive) and "1" are interpreted as true. All other values are false.

Troubleshooting

See Also

• ConversionOptionResolver: A primary consumer of the settings provided by this repository, which uses them to resolve final behavior for request processing.
• RuntimeRouteCacheService: A component that manages caching for routing information, using a similar tenant-scoped, TTL-based caching pattern.
• PayloadLimitFilter: Another component in the platform that employs a similar refresh-cache idiom for its configuration.

All SOAP-to-REST docs