Learned Field Value Repository

Overview

The LearnedFieldValueRepository provides read access to data about individual SOAP message fields learned from captured traffic. This component is primarily used by the WSDL Onboarding user interface to auto-suggest per-field encodings based on real-world data observed for a service.

It aggregates field data from two distinct sources:

1. Traffic captured from F5 gateways (s2r_f5_learned_field_value table).
2. Traffic captured from all other gateway types (s2r_generic_learned_field_value table).

The repository queries these sources for a specified service, identified by its service_backend_key, and returns a consolidated list of learned fields, including their data fingerprints, usage counts, and optionally, a sample of observed values.

How it works

The repository's primary function is to query, merge, and rank learned field data from multiple underlying database tables. The process is orchestrated by the findByServiceBackendKey method.

flowchart TD
    subgraph "Learned Field Query Process"
        A[Input: service_backend_key, optional filters] --> B{Construct Query};
        B --> C[Query F5 Gateway Data<br/>(s2r_f5_learned_field_value)];
        B --> D[Query Generic Gateway Data<br/>(s2r_generic_learned_field_value)];
        C --> E[Union Results];
        D --> E;
        E --> F[Deduplicate by Field Identity];
        F --> G[Sort and Limit Results];
        G --> H[Return List of Learned Fields];
    end

1. Querying Data Sources: The repository constructs two parallel SQL queries, one for the s2r_f5_learned_field_value table and one for the s2r_generic_learned_field_value table. Both queries are filtered by the service_backend_key of the service in question. Additional optional filters for soap_operation_name and direction (request/response) can be applied to narrow the scope.

2. Tenant Scoping: Although the underlying tables have row-level security (RLS) policies, the connection used by this repository is the table owner, which bypasses those policies. To ensure strict tenant data isolation, the query explicitly includes a predicate (TENANT_SCOPE_SQL) to filter results for the current tenant.

3. Merging Results: The results from the F5 and generic gateway tables are combined using a UNION ALL operation. A source column is added to each record to indicate its origin (f5 or the specific gateway type).

4. Deduplication: A service backend may be observed through both an F5 and a generic gateway, resulting in duplicate entries for the same logical field (identified by soap_operation_name, direction, and field_xpath). The combined result set is deduplicated using DISTINCT ON. The query is ordered by seen_count desc and last_seen_at desc before deduplication, which ensures that for any given field, the record with the highest observation count is the one that is kept.

5. Final Result Set: The final, deduplicated list of fields is ordered by seen_count in descending order, so the most frequently observed fields appear first. The result set is limited to a maximum of 500 fields.

API Reference

The repository exposes a single public method for querying learned field data.

findByServiceBackendKey

This method retrieves all learned fields for a discovered service, identified by its service_backend_key. Results are aggregated from both F5 and generic gateway traffic sources.

Parameters

Returns

Returns a List<Map<String, Object>> where each map represents a single learned field. The list is ordered by seenCount descending and is limited to 500 entries.

Returned Fields

Each map in the returned list contains the following key-value pairs.

All SOAP-to-REST docs