Ad hoc proxies and mixed auth scattered APIs; Kong Gateway with OpenID Connect and OpenTelemetry governed route updates

Overview

An insurer’s APIs grew across lines of business with different authentication methods, ad hoc reverse proxies, and uneven logging. Some services used API keys, others used bespoke tokens, and tracing stopped at the edge. Route updates happened through ticketed scripts, so changes were hard to review and even harder to roll back. Intelligex introduced a centralized API gateway with OpenID Connect (OIDC), rate limits, and distributed tracing, and added governed change workflows for route and policy updates. Teams retired custom proxies, gained consistent visibility from client to service, and rolled out new endpoints in a controlled, auditable way—while keeping existing identity, services, and monitoring.

Client Profile

• Industry: Insurance (P&C, commercial, and partner distribution)
• Company size (range): Multiple lines of business with regional platforms and a shared integration team
• Stage: Hybrid estate across data centers and cloud; varied ingress patterns; mixed authentication and custom NGINX/LB configs; tracing limited to a subset of services
• Department owner: IT & Infrastructure (Platform/SRE and Integration)
• Other stakeholders: Application Owners, Security, Identity and Access Management, Architecture, Partner Management, Compliance, Service Desk, Change Management

The Challenge

API traffic crossed multiple borders with little consistency. Some teams relayed through locally managed NGINX boxes, others exposed services behind load balancers, and partner apps consumed endpoints with one?off headers. Authentication ranged from static keys to custom JWTs, and token validation code varied by service. Rate limits were implemented inconsistently, so a busy partner or batch job could stress downstream systems without a single place to intervene.

Observability was fragmented. Logs stopped at the edge, request IDs were not propagated end?to?end, and traces existed for only a subset of services. When an incident spanned mobile apps, partner gateways, and core services, engineers compared snapshots from several tools to reconstruct the path. Meanwhile, route changes were deployed by hand using per?team scripts. Approvals lived in tickets, but the commands and configs that actually shipped were not bound to the approval record, and rollbacks were manual edits.

Compliance and partner onboarding added pressure. New integrations required repeat conversations about auth, quotas, and endpoint stability. Each exception left a long?lived configuration drift. The insurer needed a way to standardize auth and throttling, see what traffic did in real time, and make route changes through a governed, versioned path—without rewriting services or replacing identity providers.

Why It Was Happening

Root causes were decentralized ingress and policy coded in many places. Teams solved local needs with proxies and service code, so auth, quotas, and headers diverged as services evolved. There was no authoritative control plane for routes and policies, and observability stopped at tool boundaries. Change workflows referenced intent, but there was no artifact that tied the approval to the exact gateway config or rollback plan.

Ownership and timing were split. Platform owned load balancers and DNS, Identity owned tokens and claims, Security set standards, and App teams shipped endpoints. Without a single gateway layer, policy?as?code, and a standard route lifecycle, new services appeared with bespoke patterns and minimal shared visibility.

The Solution

Intelligex deployed a centralized API gateway with unified auth, quotas, and tracing, and wired it to a change workflow that versioned route and policy updates. The gateway validated OIDC tokens issued by the existing identity provider, enforced rate limits per consumer, propagated correlation IDs and spans to downstream services, and produced structured logs. Route and policy changes were made as code, reviewed, tested, and promoted under approvals. The design used Kong Gateway with OIDC and rate limiting plugins (OpenID Connect, Rate Limiting), OIDC concepts from OpenID Connect, distributed tracing with OpenTelemetry and downstream systems such as Jaeger (Jaeger), and approvals through the existing change process.

• Integrations: Gateway in front of existing services and load balancers; Identity provider for OIDC token issuance and JWKS validation; rate limits per consumer and route; tracing headers propagated to services; logs and metrics forwarded to the current monitoring/observability stack; change approvals in the ITSM.
• Auth and authorization: Standard OIDC validation at the gateway; claims?based routing to distinguish internal, partner, and public paths; service?level scopes enforced via policy; deprecation of static API keys with time?bound exceptions.
• Traffic policy: Rate limits and burst controls per consumer, route, and tag; circuit?breaker patterns to protect backends; consistent response transforms and error models.
• Observability: Correlation IDs injected or forwarded; OpenTelemetry spans started at the gateway; request/response metadata logged in structured form; dashboards for top routes, error classes, latency, and quota usage.
• Route lifecycle: Routes, plugins, and consumers defined declaratively; environment?scoped configs and templates; linting and tests in CI; staged promotion through environments with approvals.
• Change workflow: Service catalog entries for new routes with owners, SLAs, and data classification; maker?checker approvals for high?risk changes; rollback manifests attached to the change record; audit logs for who approved and who applied.
• Partner onboarding: Self?service registration requests that create consumers with default quotas and scopes, reviewed by owners; contract terms reflected in policy; lifecycle events (renew, suspend, revoke) handled via workflow.
• Security and privacy: TLS everywhere; secrets stored in the existing vault; minimal logging of payloads; PII redaction in logs; role?based access to the gateway admin plane and configs.

Implementation

• Discovery: Cataloged entry points, ad hoc proxies, and route patterns; inventoried authentication methods and token issuers; sampled partner integrations and quotas; reviewed existing tracing and logging; gathered change approval paths and audit asks.
• Design: Selected gateway topology (central clusters with regional ingress); authored route templates, plugin bundles, and claims mappings; defined correlation ID and tracing standards; designed declarative config structure and CI checks; mapped approvals and rollback steps; planned dashboards and evidence exports.
• Build: Deployed the gateway alongside existing ingress; configured OIDC validation against the current identity provider; enabled rate limiting, response transforms, and circuit breakers; instrumented the gateway with OpenTelemetry and wired traces to Jaeger and logs to the SIEM/observability stack; created declarative route and plugin repositories; integrated ITSM approvals and change links.
• Testing/QA: Ran in mirror/shadow mode for selected routes; validated token acceptance and rejection paths; exercised quota limits, error transforms, and circuit breakers; verified trace propagation through sample services; tuned logging fields and redaction; rehearsed rollback apply and verification.
• Rollout: Migrated low?risk public routes first; onboarded internal and partner routes in waves by domain; deprecated bespoke proxies with redirects; turned on approvals for high?risk changes; tightened policy gates after stable cycles.
• Training/hand?off: Delivered sessions for App teams, Security, and Partner Management on registering routes, handling tokens and scopes, reading dashboards, and requesting changes; published examples and runbooks; updated SOPs for onboarding and deprecation; transferred config repos and dashboards to Platform/Integration under change control.
• Human?in?the?loop review: Established recurring reviews for policy exceptions, noisy routes, error classes, and partner quotas; decisions recorded with rationale and effective dates; improvements fed back into templates and policies.

Results

Authentication and throttling became consistent. The gateway validated OIDC tokens the same way for every route, rate limits applied predictably per consumer, and response formats matched patterns developers recognized. Partner and internal services stopped maintaining custom auth code and keys, which reduced drift and accelerated onboarding under policy.

Visibility improved across domains. Traces started at the edge and flowed through services, making it clear where latency and errors occurred. Dashboards showed top routes, error distributions, and quota usage in one place, and incident responders followed correlation IDs from the gateway to services. Changes to routes and policies moved through the same path every time, with reviewable artifacts and rollbacks that reverted config rather than editing in place. The identity provider, services, and monitoring stayed; the change was a governed gateway and configuration layer that standardized entry and amplified observability.

What Changed for the Team

• Before: Each service handled auth differently. After: OIDC validation and claims?based policy ran at the gateway.
• Before: Rate limits and quotas varied by team. After: Consistent per?consumer and per?route limits enforced centrally.
• Before: Tracing started mid?stack. After: Correlation IDs and spans began at the edge and followed the request through services.
• Before: Route changes were scripts and screenshots. After: Declarative configs moved through linting, tests, approvals, and staged promotion.
• Before: Partner onboarding was bespoke. After: Consumers were created through a standard workflow with scopes and quotas tied to contract terms.
• Before: Rollbacks were manual edits. After: Versioned gateway configs and rollback manifests restored prior policy cleanly.

Key Takeaways

• Centralize entry; a gateway enforces auth, quotas, and headers before services see traffic.
• Use standards; OIDC and claims mapping reduce custom auth code and drift.
• Instrument the edge; start correlation and tracing at the gateway to unify observability.
• Treat routes as code; declarative configs, tests, and approvals make changes predictable and auditable.
• Onboard partners through policy; scopes and quotas tied to contracts simplify lifecycle management.
• Integrate, don’t replace; keep identity, services, and monitoring—add a governed gateway and change flow around them.

FAQ

What tools did this integrate with? The solution used Kong Gateway with plugins for OpenID Connect and Rate Limiting. Tokens followed OpenID Connect standards from the existing identity provider. Traces were emitted with OpenTelemetry and viewed in tools such as Jaeger, while logs and metrics flowed to the current observability stack. Route changes and approvals were governed in the ITSM.

How did you handle quality control and governance? Route and policy configs lived in version control with owners and rationale. CI ran linting and tests for OIDC claims mapping, rate limit templates, and response transforms. High?risk changes required maker?checker approvals, and staged promotion enforced environment gates. Every apply and rollback generated audit logs with links to tickets and dashboards.

How did you roll this out without disruption? The gateway ran in mirror/shadow mode for selected routes first, validating auth, limits, and traces without affecting traffic. Low?risk routes migrated in waves with redirects from bespoke proxies. Approvals and rollback manifests were exercised during pilots, and the prior path remained as a controlled fallback until stable.

How were partners and internal consumers onboarded? Consumers were created through a request workflow that captured scopes, quotas, and owner contacts. The gateway issued credentials or registered client apps against the identity provider. Default policies applied per consumer type, and exceptions were time?bound with approvals and review dates.

How did this affect service teams? Services removed custom auth code and relied on standardized headers and claims. Tracing and correlation IDs arrived with requests, improving diagnostics. Route changes became pull requests against declarative configs, with predictable promotion gates and clear rollback paths.

What about legacy APIs and non?OIDC clients? Legacy routes used transitional policies with key?based auth and limited scopes, wrapped by rate limits and monitoring. These routes had deprecation timelines and were reviewed regularly. Where feasible, client updates moved to OIDC, and exceptions were tracked with compensating controls.