SOAP-to-REST Conversion Platform: Architecture and Design

Overview

This document describes the architecture and core design principles of the SOAP-to-REST conversion platform. The platform is designed to convert a large number of backend SOAP web services into modern RESTful APIs in a scalable and maintainable way.

The system consists of a metadata-driven conversion engine built on a Java and Spring Boot stack. It separates the management of API conversions (the control plane) from the live request processing (the data plane).

Core Components:

• Control Plane: An administrative application with a UI and API that allows API publishers to manage API definitions, mappings, versions, and user roles. It also provides integrated observability features like dashboards, tracing, and analytics.
• Data Plane (Runtime): A stateless runtime service deployed on Cloud Run that executes the request and response transformations between REST and SOAP. It is designed for high concurrency and low-latency performance.
• Data Storage:
  • Cloud SQL for PostgreSQL: The source of truth for all persistent state, including API definitions, transformation mappings, version history, user roles, and audit logs. The database is named soap_to_rest.
  • Cloud Storage: Used for storing large binary artifacts, primarily WSDL and other specification files, as well as for staging large payloads.

The platform is deployed as an internal-only service, running on Cloud Run behind an internal Application Load Balancer. This ensures that the conversion runtime is not publicly reachable and integrates privately with API management layers like Apigee over VPC connectivity.

How it Works

The platform's core function is a metadata-driven conversion process that avoids the need for custom code for each of the 200+ services it is designed to support.

API Conversion and Publishing Workflow

1. Definition: An API publisher uses the control plane to define a new service conversion. They upload the source WSDL/XSD specification artifacts, which are stored in Cloud Storage.
2. Mapping Generation: The system automatically generates transformation mappings based on the service specification. These mappings are stored as versioned records in the Cloud SQL database. Publishers can create editable overrides for these generated mappings to handle special cases.
3. Artifact Generation: The platform generates an OpenAPI v3 specification for the new REST API.
4. Manual Deployment: The API publisher manually imports the generated OpenAPI v3 artifact into the Apigee API management layer. The platform does not directly integrate with Apigee deployment APIs.
5. Environment Targeting: The system treats environments like dev, test, and prod as distinct service targets within its metadata, allowing a single conversion definition to point to different backend endpoints for each environment.

Runtime Request Processing

1. An API consumer sends a REST request to an endpoint managed by Apigee.
2. Apigee forwards the request over a private VPC connection to the internal Application Load Balancer, which routes it to a stateless Cloud Run runtime instance.
3. The runtime service fetches the active conversion mapping and version information from Cloud SQL. For performance, it may use a short-lived, non-persistent, process-local in-memory cache for this configuration.
4. Using the mapping, the service transforms the inbound REST request (JSON) into a SOAP envelope (XML). The transformation logic relies on the standard Java javax.xml.transform library.
5. The service applies the configured backend authentication profile, such as adding an HTTP Authorization header or injecting a wsse:UsernameToken into the SOAP header.
6. The SOAP request is sent to the backend web service.
7. The runtime receives the SOAP response, transforms it into a RESTful JSON response, and returns it to the consumer.

Versioning and Rollback

The platform uses a versioning model based on immutable artifacts and an active pointer.

• All API definitions, mappings, and related artifacts are immutable once created.
• A pointer in the database designates which version of a service is "active".
• This design enables instantaneous rollbacks by simply updating the pointer to a previous, known-good version. It also ensures full auditability and reproducibility of every version.

Configuration and Operational Parameters

The system has several key operational parameters and default settings. Global defaults and IAM group-to-role bindings are managed in Parameter Manager.

Parameter	Default Value	Configurable	Description
max_sync_payload_mb	30	No	The maximum size in megabytes for a synchronous request payload. Requests exceeding this are rejected.
Backend Timeout	1000ms	Yes	The default timeout for requests to the backend SOAP service.
Retry Count	2	Yes	The default number of retries for failed requests to the backend service.
Trace Retention	60 days	Yes	The default retention period for audit and debug traces.

Backend Authentication

The platform supports multiple authentication mechanisms for connecting to backend SOAP services. Authentication is configured on a per-operation or per-backend profile basis using the auth_type parameter.

Credentials for authentication can be sourced from Secret Manager or provided directly in the configuration.

• Secret Manager: The recommended approach. The runtime retrieves secrets using a reference (e.g., projects/.../secrets/.../versions/... or sm://...). The runtime service account requires the roles/secretmanager.secretAccessor IAM role.
• Inline: A fallback mechanism (inline:<credential>) for use in controlled environments or when network policies temporarily prevent egress to Secret Manager.

The following auth_type values are supported:

auth_type	Description
none	No authentication is performed.
basic	Adds a standard HTTP Authorization header with Basic authentication credentials.
username_token	Injects a wsse:UsernameToken into the SOAP header with the password in wsse:Password as PasswordText.
username_token_digest	Injects a wsse:UsernameToken with the password digest in wsse:Password as PasswordDigest.
ws_security	An alias for username_token.
ws_security_timestamp	Same as username_token, but also adds a wsu:Timestamp element to the security header.
ws_security_digest	An alias for username_token_digest.
ws_security_digest_timestamp	Same as username_token_digest, but also adds a wsu:Timestamp element to the security header.

User Roles and Permissions

Access to the control plane is managed through a set of predefined roles. These roles are mapped to IAM groups. The data plane runtime does not use these roles for request processing.

Role	Permissions
admin	Full create, read, update, and delete (CRUD) permissions. Can publish services, perform rollbacks, and manage role mappings.
operator	Can enable or disable services, read configurations, run diagnostic tests, and view metrics. Cannot modify service definitions.
reader	Read-only access to all configurations and metrics.

Role mappings are configured in Parameter Manager, linking an IAM group to one of the roles (e.g., SOAP-To-REST_Admin@<iam-group-domain> -> admin).

Troubleshooting and Error Handling

• Large Payloads: The platform enforces a strict synchronous payload limit of max_sync_payload_mb = 30. Any request with a body larger than this limit will be rejected immediately with an HTTP 413 Payload Too Large status code and a structured error message. Asynchronous handling for larger objects is a deferred capability.
• Authentication Failures: The Admin UI provides tools to test endpoint connectivity and authentication, including WSDL pull diagnostics, to help debug connection issues with backend services.
• Secret Retrieval Errors: If the runtime fails to retrieve a secret from Secret Manager, it may be because the runtime service account is missing the roles/secretmanager.secretAccessor IAM role.
• Logging Policy:
  • All error transactions are logged automatically.
  • Payloads from successful transactions are logged only when debug mode is explicitly enabled for a service.
  • The system does not mask sensitive fields by default.

Platform and Deployment

• Technology Stack: Java 25, Spring Boot 3.5, Cloud SQL for PostgreSQL.
• Cloud Infrastructure:
  • Compute: Cloud Run with startup CPU boost (cpu-boost) enabled for fast scale-out. Minimum instances are set to 1.
  • Networking: Deployed within an existing VPC and subnet, using Direct VPC egress for outbound traffic. Communication between Cloud Run and Cloud SQL uses private IP routing, requiring no Serverless VPC Access connector.
  • Primary Region: The primary deployment region is me-west1.
• Database Authentication: The application runtime authenticates to Cloud SQL using service account-based IAM database authentication, not static passwords.
• Deployment Strategy: Services are built as local Docker images, deployed first via gcloud commands, and then infrastructure state is aligned using Terraform.

---

All SOAP-to-REST docs