Master Discovery Refresh Service

Overview

The Master Discovery Refresh Service is a background process responsible for populating and maintaining the s2r_master_discovery materialized table. This table serves as a pre-computed cache of all APIs discovered through observed traffic, significantly improving the performance of user-facing discovery APIs by eliminating the need for expensive, real-time computations on each request.

The service operates on a configurable schedule, processing traffic data from both F5 and other generic gateway vendors. It transforms raw traffic logs into a structured, aggregated, and enriched format. For each API, it consolidates information about its endpoints, environments, consumers, and backend services.

In addition to the scheduled background refresh, the service exposes methods for triggering synchronous, on-demand refreshes. This is useful for bootstrapping a new gateway instance that has just started processing traffic or for manual administrative actions.

How It Works

The service's primary function is a scheduled job that runs periodically for each tenant. This job discovers APIs from two main sources: F5 instances and generic gateways distinguished by a gateway_type.

flowchart TD
    subgraph Scheduled Refresh Cycle
        A[Scheduler Trigger] --> B{Service Enabled?};
        B -- Yes --> C{Acquire Global Lock};
        B -- No --> Z[End];
        C -- Acquired --> D[For Each Tenant];
        C -- In Use --> Z;
        D --> E[Find Enabled F5 Instances];
        E --> F[For Each F5 Instance];
        F --> G["call refreshInstance(id)"];
        G --> F;
        F -- Done --> H[call refreshGenericVendors];
        H --> I[Find Generic Scopes];
        I --> J[For Each Generic Scope];
        J --> K["call refreshGenericVendor(type, id)"];
        K --> J;
        J -- Done --> D;
        D -- All Tenants Processed --> Y[Release Global Lock];
        Y --> Z;
    end

    subgraph "On-Demand Refresh (API Call)"
        M[External Call] --> G;
        N[External Call] --> K;
    end

Scheduled Refresh Cycle

The scheduledRefresh method orchestrates the entire process:

1. Check Prerequisites: The cycle begins only if the service is enabled (s2r.master-discovery.refresh.enabled is true) and no other refresh cycle is currently running. A global lock (cycleRunning) prevents concurrent cycles.
2. Tenant Iteration: The process runs independently for each tenant in the system.
3. F5 Instance Refresh: For the current tenant, the service retrieves all enabled F5 instances and calls refreshInstance() for each one. Errors during the refresh of one instance are logged but do not stop the process for other instances.
4. Generic Vendor Refresh: After processing all F5 instances, the service calls refreshGenericVendors() to handle traffic from non-F5 sources. This ensures that all observed traffic is reconciled into the master discovery table.
5. Release Lock: Once all tenants are processed, the global lock is released, allowing the next scheduled cycle to run.

F5 Instance Refresh Process

The refreshInstance(UUID instanceId) method executes a multi-step pipeline to process data for a single F5 instance.

1. Fetch Raw Data: It calls f5LearningRepository.computeMasterDiscoveryLive to get raw, unaggregated discovery rows from observed traffic for the specified instance.

   NOTE

   The number of rows computed in this step is capped at a fixed limit of 5,000 (REFRESH_COMPUTE_LIMIT) to manage resource consumption.

2. Filter Invalid URIs: Rows with empty or root-only (/) service URIs are discarded.
3. Aggregate Rows: The raw data is collapsed into a canonical form using MasterDiscoveryAggregator.collapseByUriAndEnvironment. This creates a single master row for each unique combination of a service's canonical URI and its environment.
4. Apply Manual Overrides: The service loads any manual overrides defined for the instance and applies them to the corresponding aggregated rows. This allows administrators to correct or enrich automatically discovered data.
5. Enrich with Service Inventory: It attempts to link each discovered API to a known service in the Service Inventory. If a row's serviceBackendUrl matches a backend URL in the inventory, the row is enriched with the corresponding serviceId and serviceStatus.
6. Set Discovery Status: Each row is marked with a discoveryStatus of either complete or incomplete based on the logic in the isComplete method (see Data Completeness).
7. Upsert Data: The final, enriched list of rows is passed to masterDiscoveryRepository.upsertAll, which atomically updates or inserts the master discovery rows and their associated consumer data into the database.
8. Clean Stale Overrides: Finally, the service removes any manual overrides that no longer correspond to any of the discovered APIs from the current refresh, preventing the accumulation of obsolete configuration.

Generic Vendor Refresh Process

The process for generic vendors mirrors the F5 process but is adapted for non-F5 data sources, which are identified by a gateway_type string.

1. Identify Scopes: The refreshGenericVendors method first queries for all unique discovery scopes that have observed traffic. A scope is defined by the combination of a gatewayType and a connectorInstanceId. This ensures that different gateways of the same vendor type are processed independently.

2. Refresh per Scope: For each scope, the refreshGenericVendor method is called. Failures are isolated to the specific scope, allowing other vendors to be processed.

3. Data Processing: The pipeline within refreshGenericVendor is a simplified version of the F5 flow:

   • Fetches raw data via f5LearningRepository.computeMasterDiscoveryLiveGeneric.
   • Filters invalid URIs.
   • Aggregates rows by URI and environment.
   • Enriches with Service Inventory links.
   • Sets the discoveryStatus.
   • Upserts data via masterDiscoveryRepository.upsertAllGeneric. The resulting rows are marked with source='generic'.

   This path does not include steps for F5-specific topology or manual overrides.

Data Completeness

A discovered API row is considered complete only if it meets specific criteria. This status helps distinguish between fully resolved and partially discovered services. A row is marked complete if it has:

• A non-blank serviceUri.
• A non-blank environment.
• AND at least one of the following is true:
  • It has a resolved serviceBackendUrl.
  • It has a fully populated F5 topology path (both frontVirtualServers and either backVirtualServers or a dataPowerDomain).

Rows that do not meet these criteria are still saved but are marked with discoveryStatus: "incomplete".

Concurrency and Locking

The service employs a multi-layered locking strategy to ensure data integrity and prevent redundant work:

• Global Cycle Lock: A single AtomicBoolean (cycleRunning) ensures that only one scheduledRefresh cycle can run at a time across the entire system.
• Per-Instance Lock: A ConcurrentHashMap<UUID, AtomicBoolean> (instanceLocks) provides a lock for each F5 instance ID. This prevents a scheduled refresh and a manual refreshInstance call for the same instance from running concurrently.
• Per-Vendor-Scope Lock: A ConcurrentHashMap<String, AtomicBoolean> (vendorLocks) provides a lock for each generic vendor scope (e.g., <gatewayType>:<connectorInstanceId>). This allows parallel refreshes of different generic gateways while serializing operations on the same one.

If a refresh is attempted while a lock is held, the new attempt is skipped, and a debug message is logged.

Configuration

The service's schedule and behavior can be configured via application properties.

| Property | Type | Default | Required | Description | --- | --- | --- | --- | | s2r.master-discovery.refresh.enabled | boolean | true | No | If true, the scheduled refresh job will run. If false, the job is disabled. | s2r.master-discovery.refresh.initial-delay-ms | long | 180000 | No | The delay in milliseconds before the first execution of the scheduled job after application startup. Defaults to 3 minutes. | s2r.master-discovery.refresh.delay-ms | long | 600000 | No | The delay in milliseconds between the end of one execution and the start of the next. Defaults to 10 minutes.

API Reference

While not a web-facing API, the service provides public methods that can be called internally by other components, such as administrative endpoints.

public void scheduledRefresh()

The entry point for the scheduled job. It orchestrates the refresh process for all tenants, F5 instances, and generic vendors. It is protected by a global lock to prevent overlapping runs.

public int refreshInstance(UUID instanceId)

Synchronously triggers a full refresh for a single F5 instance, identified by its instanceId.

• Parameters:
  • instanceId (UUID): The unique identifier of the F5 instance to refresh.
• Returns: int - The number of master discovery rows that were created or updated.
• Behavior: This method is idempotent and thread-safe. If a refresh for the specified instance is already in progress, the call is skipped.

public int refreshGenericVendor(String gatewayType, UUID connectorInstanceId)

Synchronously triggers a full refresh for a single generic vendor scope.

• Parameters:
  • gatewayType (String): The identifier for the vendor (e.g., 'my-custom-gateway').
  • connectorInstanceId (UUID): The specific instance of the gateway to refresh. A null value targets the "unattributed" bucket for that vendor type.
• Returns: int - The number of master discovery rows that were created or updated.
• Behavior: If a refresh for the specified scope is already in progress, the call is skipped.

public void refreshGenericVendors()

Triggers a refresh for all generic vendor scopes that have observed traffic. It discovers the scopes and then calls refreshGenericVendor for each one.

Troubleshooting

Skipped Refreshes

A refresh cycle or individual refresh operation may be skipped under the following conditions:

• Scheduled Cycle: The entire scheduledRefresh job will not run if s2r.master-discovery.refresh.enabled is false or if a previous cycle has not yet completed.
• Concurrent Instance/Scope Refresh: A call to refreshInstance or refreshGenericVendor will be skipped if a refresh for the same instance or scope is already in progress. A log message is generated: Master discovery refresh already in progress for...; skipping...

Error Conditions

The service is designed to be resilient, isolating failures to the specific instance or scope being processed.

| Log Message | Trigger | Impact | --- | --- | | Master discovery refresh failed for tenant {} instance {}: {} | An exception occurred within the refreshInstance method for a specific F5 instance. | The refresh for that single instance is aborted. The scheduled job continues to the next instance. | Master discovery generic-scope enumeration failed: {} | The database query to find generic vendor scopes with traffic failed. | The entire generic vendor refresh process for the current tenant is aborted. F5 instance refreshes are unaffected. | Master discovery refresh failed for generic scope {}/{}: {} | An exception occurred within the refreshGenericVendor method for a specific scope. | The refresh for that single scope is aborted. The process continues to the next generic scope. | Master discovery refresh error for {}: {} | A general exception was thrown from refreshInstance. The service records the error before re-throwing it. | The refresh for that instance is aborted. The exception propagates up, potentially affecting the calling process.

See Also

• Manual Discovery Overrides: For details on how to manually override or enrich discovered API data.
• Environment Definitions: For information on how environments are defined and used in the aggregation process.
• Service Inventory: To understand how discovered backends are linked to the central inventory of services.

All SOAP-to-REST docs