Learned Examples from Live Traffic

Learn how the platform captures real request/response pairs from live traffic as learned examples for use in API onboarding and mapping validation.

Overview

Learned examples are real request and response pairs captured from production traffic at the gateway edge. They are a core component of the traffic-driven discovery feature, available since v1.0.

Instead of relying on synthetic or manually created sample payloads during API onboarding, learned examples provide concrete evidence of the actual messages your services exchange. This solves two key problems:

Onboarding Accuracy: API publishers can work with the true request and response formats their services use in production, including all headers and metadata.
Mapping Validation: Each request/response pair serves as a ground-truth check for the platform's SOAP-to-REST conversion mappings, allowing you to verify that the generated mapping correctly handles real-world data shapes and edge cases.

Learned examples are derived from raw traffic logs by a background process and are keyed by service and SOAP operation. They are surfaced in the platform's discovery and observability UIs to provide a view of representative live traffic for any given service.

How it works

The platform generates learned examples through an offline curation process that transforms raw transaction logs into a de-duplicated, representative dataset.

flowchart TD
    A[Live API Traffic] --> B[Raw Transaction Log]
    B --> C{Curation Process}
    C --> D[Learned Examples Dataset]
    C --> E[Discovery Aggregation Tables]
    D --> F[Onboarding & Discovery UI]

The process follows these steps:

Capture: Connectors at the gateway edge (such as the F5 BIG-IP connector) capture full transaction artifacts from live traffic. For each transaction, the raw traffic log receives the request body, response body, request headers, response headers, and associated metadata (e.g., endpoints, client IP, HTTP method, SOAPAction, status code, latency).
Curation: A background curator worker processes the raw logs. It identifies unique request/response pairs based on a hash of the request payload.
De-duplication: Instead of storing every captured transaction, the curator maintains a representative set. When a transaction with a previously seen request hash is found, it increments a seen_count on the existing learned example rather than creating a new entry. This keeps the dataset small, relevant, and fast to query.
Storage: The resulting learned example is stored in a derived dataset, keyed by service and SOAP operation. This dataset powers the "good examples" tile in the discovery and observability UIs.

IMPORTANT

The completeness of a learned example depends on the edge connector's configuration. If a connector is configured to log only URI-level data and not full request/response bodies, the learned examples derived from that traffic source will contain metadata but will have empty payload fields.

Use Cases

Learned examples are used in two primary ways within the platform:

As Onboarding Examples

During the API onboarding process, learned examples are displayed in the UI to provide real, per-operation sample payloads. This allows API publishers to see the actual XML that consumers send and the corresponding responses the backend service returns, eliminating the need to invent or source sample messages.

As Mapping Evidence

Because a learned example contains a real SOAP request paired with its actual response, it serves as ground truth for validating a conversion mapping. You can use these examples to confirm that the generated full-field mapping correctly handles all data shapes and edge cases present in production traffic, catching potential gaps before they impact API consumers.

NOTE

The capture and curation process preserves payloads end-to-end, including UTF-8 characters. Content in languages such as Hebrew, which may use right-to-left and gendered text, is captured faithfully.

Learned Example Data Structure

A learned example is a structured object derived from a raw transaction. The source does not provide concrete examples of payloads. The fields in a learned example are described below.

Field	Type	Description
Request Payload	String	The captured request body, typically XML.
Request Hash	String	A hash of the request payload, used for de-duplication and grouping.
Response Payload	String	The captured response body.
Request Headers	Object	The captured HTTP request headers.
Response Headers	Object	The captured HTTP response headers.
Request Method	String	The HTTP method of the request (e.g., `POST`).
SOAP Action	String	The `SOAPAction` header value from the request.
Front-end Identity	Object	Metadata identifying the front-end endpoint that received the request.
Back-end Identity	Object	Metadata identifying the back-end service that handled the request.
Seen Count	Integer	A counter that is incremented each time an identical request is observed, indicating how representative the example is.

Troubleshooting

Empty Response Bodies in Examples

If you observe a learned example with an empty response body for a service that is known to return data, this is typically a detail of the curation process, not an indication of a logging failure at the edge.

To diagnose the issue:

Find the correlation ID associated with the learned example.
Query the raw traffic log for the same correlation ID.
Check the raw log entry:
- If the raw log contains the response body, the data was successfully captured at the edge. The gap is downstream of ingest, within the curation and derivation logic.
- If the raw log is also missing the response body, the issue is likely with the edge device's logging configuration.

Note that some operations, such as fire-and-forget or acknowledgement-only patterns, legitimately have no response body. In these cases, an empty response in the learned example is expected and correct.