Tenant-Keyed In-Process Cache

Overview

The TenantKeyedCache is a short-time-to-live (TTL), in-process cache designed for multi-tenant environments. Its primary purpose is to prevent data leakage between tenants, which can occur when using a simple, JVM-wide cache.

In a multi-tenant system, different API consumers operate within their own isolated data scopes, or "tenants". When caching data fetched from a tenant-scoped table or setting, it is critical that one tenant's data is never served to another. The TenantKeyedCache solves this by keying every cache entry to the specific tenant ID bound to the thread that requests the data.

This component is intended for use by internal services that need to cache tenant-specific data, such as configuration settings or frequently accessed resources. It relies on the system's TenantContext to identify the current tenant.

Key characteristics:

• Tenant-Scoped: A value cached for one tenant will never be served to a request for another tenant.
• In-Process: The cache state resides entirely in the memory of the service instance. A restart of the service container clears the cache. The database remains the source of truth.
• TTL-Based: Entries are automatically expired after a configured time-to-live.
• Safety by Default: If no tenant is bound to the current thread, the cache is bypassed entirely. The underlying data loader is called every time, preventing an unbound request from ever receiving a tenant's cached data.

How it Works

The TenantKeyedCache associates cached data with a tenant ID provided by TenantContext.current(). This is the same context used by TenantBindingDataSource to set the app.tenant_id on the database connection, ensuring data is fetched from and cached for the correct tenant.

Data Retrieval Flow

The get(Supplier<V> loader) method is the primary entry point for retrieving a value. The logic follows a clear sequence to ensure tenant isolation and data freshness.

flowchart TD
    subgraph "TenantKeyedCache.get() Flow"
        A[Start] --> B{Tenant bound via TenantContext?};
        B -- No --> C[Bypass cache];
        C --> D[Invoke loader function];
        D --> E[Return fresh value];

        B -- Yes --> F[Get tenant ID];
        F --> G{Valid entry exists in cache?};
        G -- Yes (Cache Hit) --> H[Return cached value];

        G -- No (Cache Miss / Expired) --> I[Record current generation number];
        I --> J[Invoke loader function to fetch fresh value];
        J --> K{Generation changed during load?};
        K -- Yes (Invalidated) --> L[Discard fresh value];
        L --> M[Return fresh value (not cached)];
        K -- No --> N[Store fresh value in cache];
        N --> M;
    end
    H --> Z[End];
    E --> Z;
    M --> Z;

1. Check Tenant Context: The cache first checks if a tenant is bound to the current thread.
2. Cache Bypass: If no tenant is bound, the cache is bypassed. The provided loader function is executed immediately, and its result is returned directly. Nothing is stored in the cache. In a properly configured system, an attempt to read tenant data without a bound tenant may fail at the database layer (e.g., with error S2R-TENANCY-UNBOUND).
3. Cache Hit: If a tenant is bound, the cache looks for a non-expired entry associated with that tenant's ID. If found, the cached value is returned immediately.
4. Cache Miss: If no entry exists or the existing entry has expired, the cache executes the loader function to fetch a fresh value from the source of truth (e.g., the database).

In-flight Load Protection

To prevent race conditions where a value is invalidated while a slow load operation is in progress, the cache uses a generation-tracking mechanism.

• An epoch counter is incremented every time invalidateAll() is called.
• A per-tenant generations counter is incremented when invalidateCurrentTenant() is called for a specific tenant.
• The effective generation for any tenant is the sum of the global epoch and its specific generations value.

When a cache miss occurs, the get method records the tenant's current generation number before calling the loader. After the loader returns, the value is only stored in the cache if the tenant's generation number has not changed. If an invalidation occurred during the load, the generation number will have increased, and the newly loaded (but now stale) value is discarded.

Invalidation

Cache invalidation is also tenant-aware.

• invalidateCurrentTenant(): Drops the cache entry for the tenant currently bound to the thread. If the special __platform__ tenant is bound (whose settings can overlay all tenants), or if no tenant is bound, this operation escalates to a full cache invalidation.
• invalidateAll(): Drops every entry for every tenant in the cache. This is a global invalidation.

Memory Management

When the number of cached entries exceeds the SWEEP_THRESHOLD (currently 1024), the next write operation will trigger a sweep to remove any expired entries, freeing up memory.

Usage Example

To use the cache, first instantiate it with a desired TTL. Then, use the get method, providing a Supplier function that knows how to load the data on a cache miss.

// 1. Instantiate the cache with a 5-minute TTL.
TenantKeyedCache<String> configCache = new TenantKeyedCache<>(300000L);

// ...

// 2. Use the cache to retrieve a setting.
// The lambda expression is the loader, which is only executed on a cache miss.
String tenantSetting = configCache.get(() -> {
    // Logic to fetch a setting from the database for the current tenant.
    return database.fetchSettingForCurrentTenant("some.setting.key");
});

// ...

// 3. Invalidate the cache when the underlying data changes.
// For example, after an API call updates the setting.
configCache.invalidateCurrentTenant();

API Reference

Constructor

The TenantKeyedCache is instantiated with a time-to-live for its entries.

Methods

Constants

Troubleshooting

• IllegalArgumentException: The constructor will throw this exception if ttlMillis is less than or equal to zero. The TTL must be a positive duration.
• Unbound Tenant Behavior: When no tenant is bound to a request thread, the cache is always bypassed. This is a safety feature. If you expect data to be cached but it is not, verify that the TenantContext is correctly populated for the request thread. In some configurations, an unbound read may result in a downstream error from the database (e.g., S2R-TENANCY-UNBOUND).

See Also

• Tenant Context Management: The mechanism responsible for identifying the current tenant and making it available via TenantContext.
• Data Source Tenant Binding: The component (TenantBindingDataSource) that uses the tenant context to bind database connections to a specific tenant.

All SOAP-to-REST docs