Operations Overview and Architecture

An overview of the operational model for the API platform, including its four main services, core principles, capacity baseline, and common failure modes.

NOTE

This component has been Generally Available (GA) since v1.0.

Overview

This document provides a high-level overview of the platform's operational model. It is intended for operators responsible for deploying, managing, and monitoring the platform in a production environment.

The platform consists of four cooperating services that rely on a single PostgreSQL database for all persistent configuration and state. This guide outlines the role of each component, the core principles for stable operation, and pointers to detailed runbooks for specific tasks.

For initial setup, refer to the Installation and Configuration reference guides before using the operational guides in this section.

Architecture and Core Principles

The platform is designed around a central database that acts as the single source of truth, with distinct services handling the control plane, data plane, background processing, and user interface.

Component Roles

The system is composed of four primary services, each with a specific role.

Service	Role	Operational Note
`admin-api`	Control plane	Manages database schema migrations via Flyway. This service must be deployed or upgraded first.
`runtime`	Data plane (conversion)	A stateless service that can be scaled horizontally. Maintain a warm minimum number of instances to avoid cold start latency.
`worker`	Aggregation, discovery derivation, learned examples	Performs background processing to generate aggregated data. The health of this service is critical for operational dashboards.
`admin-ui`	Operator web UI	A user interface that proxies API calls to the `admin-api` and `runtime` services.

Architectural Diagram

The following diagram illustrates the relationships between the services and the central database.

flowchart TD
    subgraph "API Platform Services"
        A[Control Plane<br>(admin-api)]
        B[Data Plane<br>(runtime)]
        C[Aggregation & Discovery<br>(worker)]
        D[Operator UI<br>(admin-ui)]
    end

    DB[(PostgreSQL Database)]

    A -- Manages schema & config --> DB
    B -- Reads config --> DB
    C -- Reads raw data --> DB
    C -- Writes aggregated data --> DB
    D -- Proxies to --> A
    D -- Proxies to --> B

Core Principles

Two fundamental principles govern the operation of the platform:

The database is the source of truth. All persistent configuration and application state are stored in the PostgreSQL database in tables prefixed with s2r_*. The application services are stateless and can be replaced, but the database must be backed up and protected.
Dashboards read aggregation tables, not raw logs. To ensure fast, sub-second load times, all metrics and time-bucketed data displayed in dashboards are served from pre-computed aggregation tables. The worker service is responsible for refreshing these tables. If the worker's aggregation processes fail, dashboards will become stale without explicit errors. Stale dashboards should be treated as a high-priority incident.

Capacity Baseline

The platform is load-tested to handle a baseline of approximately 1 million calls per day. This corresponds to a sustained load of about 12 requests per second (rps), with support for short bursts up to approximately 100 rps.

For information on monitoring current capacity and headroom, see the Daily operations and Metrics & SLO reports documentation.

Troubleshooting

The platform has several common failure modes. The Troubleshooting guide provides symptom-based diagnostic procedures for these issues.

Key issues to be aware of include:

Ingest Gaps: Missing data in the platform.
Schema Mismatch: Occurs after an upgrade if components are deployed out of order.
Credential/Key Errors: Indicated by error code S2R-ADM-0419.
Payload Too Large: Indicated by error code S2R-RUN-0413.
Account Lockout: Inability to access the system.
Stale or Slow Dashboards: This is a critical symptom indicating that the worker service may be failing to refresh its aggregation tables.

The Troubleshooting document also details where to locate service logs.

For detailed procedures and reference material, consult the following documents:

Installation: First-time installation instructions.
Configuration reference: Detailed configuration settings.
Deployment runbook: The correct sequence for deploying and upgrading services (admin-api → runtime → worker).
Daily operations: A checklist for day-to-day monitoring, including health checks and capacity verification.
Troubleshooting: Symptom-first diagnostics for common issues.
Backup & restore: Procedures for backing up and restoring the PostgreSQL database.
Metrics & SLO reports: Details on observability surfaces for monitoring capacity and performance.

Tagsguides

All SOAP-to-REST docs