Data retention

The product retains operational data — runtime logs, ingested traffic, metrics aggregations, and the audit trail — for a bounded window, then sweeps it.

The product retains operational data — runtime logs, ingested traffic, metrics aggregations, and the audit trail — for a bounded window, then sweeps it. Retention is configurable, with a sane default, and runs entirely inside your database. Nothing is shipped out for retention.

The default

The headline retention setting is default_retention_days, which defaults to 60 days. It is a mutable system setting, editable under Settings → Retention (read/write via GET / PATCH /admin/v1/settings). It governs the configured-retention sweeps for runtime logs and the audit trail.

Several data classes have their own retention knobs and defaults — the raw traffic archive in particular keeps a shorter window in the database and a longer window in object storage. The table below is the practical summary.

What is retained, and for how long

| Data | Default retention | Configurable | Notes | |---|---|---|---| | Runtime logs (per-call records) | default_retention_days (60 days) | Yes — Settings → Retention | Cleaned up by the configured retention-days sweep. | | Audit trail (every config change) | default_retention_days (60 days) | Yes | Who/what/before-after, with correlation ID. See Audit trail. | | Raw ingested traffic (gateway/connector feed) | 14 days in the database; up to 365 days when archived to object storage | Yes — separate knobs | Short DB window for drill-down; longer cold archive in your own storage bucket. | | Debug traces | 14 days in the database; up to 365 days in object storage | Yes | Same archive model as raw traffic. | | Generic-HTTP receiver raw log | 60 days | Yes | Bounded-delete sweep. | | Ingest parse-failure queue (DLQ) | 7 days | No — product invariant | Stale parse failures lose diagnostic value after a week. | | Metric aggregation tables (hourly rollups, topology) | 365 days | Yes | The longer-lived, pre-computed time-series that dashboards read from. |

All retention sweeps run on a background schedule inside the worker service; they delete in bounded chunks so a single sweep never holds a long database lock. The product documentation in docs/lld/data-retention.md is the canonical, code-backed reference for the exact per-table predicates and cadences.

Drill-down vs aggregation — why two windows

The product deliberately separates two kinds of data, and they are retained differently:

Raw logs are the per-event detail used for drill-down: "show me exactly what happened on this one correlation ID." These are kept for a short window in the database (then optionally archived to your object storage for the longer cold window), because raw detail is voluminous and most useful while recent.
Aggregation tables are the pre-computed rollups that dashboards, KPIs, and SLO reports read from. These are kept far longer (a year by default) because they are compact and power the long-range trend views. Dashboards never query raw logs at request time — they read the aggregations, which is what makes them fast.

This split is why a dashboard can show a year of per-service trend lines while the raw per-call records behind them only go back a couple of months in the database (and up to a year in cold archive).

Where retained data lives

Everything subject to retention lives in your own infrastructure — your database and, for the cold archive, your own object-storage bucket. Retention is a sweep over data you already hold; nothing is exported off your network as part of it. See Network & data residency.

Changing retention

default_retention_days and the other configurable windows can be changed under Settings; the change takes effect on the next scheduled sweep. Lengthening a window keeps data longer (and uses more storage); shortening it causes the next sweep to remove anything now past the new window. As with any setting change, the edit is recorded in the audit trail.

The DLQ's 7-day window and a small number of internal sampling tables are fixed product invariants and are not operator-tunable; everything in the "Configurable: Yes" rows above is.

All Specaria SOAP to REST docs

Loading…

The default

What is retained, and for how long

Drill-down vs aggregation — why two windows

The product deliberately separates two kinds of data, and they are retained differently:

Raw logs are the per-event detail used for drill-down: "show me exactly what happened on this one correlation ID." These are kept for a short window in the database (then optionally archived to your object storage for the longer cold window), because raw detail is voluminous and most useful while recent.

Aggregation tables are the pre-computed rollups that dashboards, KPIs, and SLO reports read from. These are kept far longer (a year by default) because they are compact and power the long-range trend views. Dashboards never query raw logs at request time — they read the aggregations, which is what makes them fast.

This split is why a dashboard can show a year of per-service trend lines while the raw per-call records behind them only go back a couple of months in the database (and up to a year in cold archive).

Changing retention

The DLQ's 7-day window and a small number of internal sampling tables are fixed product invariants and are not operator-tunable; everything in the "Configurable: Yes" rows above is.