One REST Call Instead of a Three-Step Auth Dance

The platform sat on top of a set of older internal services: an accounts and auth service, the job scheduler, an ad-inventory service. They worked. They were also a pain to call.

Two reasons. First, they didn't speak plain REST. They spoke an RPC dialect over a bridge, so a "request" was a list of service-and-function calls wrapped in a specific envelope, and the responses came back nested a few layers deeper than anyone wanted to parse. Second, the auth was a three-step dance.

The dance

To make one real call against one of these services, you had to:

1. Get an OAuth token from our identity provider.
2. Sign in to the accounts service with your credentials to mint a short-lived JWT.
3. Attach that JWT to the actual call you wanted to make, against whichever downstream service.

None of that is hard once. The problem is that every consumer reimplemented all three steps, in whatever language they had, and most of them got some detail wrong: the wrong envelope, a token that expired mid-batch, a JWT passed to the wrong service. The auth flow was accidental complexity that had nothing to do with the question anyone was actually asking, which was usually "pause this stream" or "what's in this category."

The proxy

So I designed the data mesh API layer and built proxy APIs in MuleSoft to sit in front of these services. The deal for a consumer is simple: send one REST request, with your credentials and an OAuth bearer token, to one endpoint. That's it.

Behind that endpoint, the proxy does the parts everyone kept getting wrong. It runs the sign-in, mints the JWT, attaches it, forwards the call to the right backend, and unwraps the RPC response into plain JSON on the way back. The three-step dance still happens. It just happens once, server-side, written down correctly, instead of in every caller.

I shipped three of these: accounts and auth, the scheduler, and inventory. Each one runs across dev, QA, and prod, deployed through GitLab CI/CD. The routing is per-environment, with one wrinkle worth mentioning: some of the backend services only existed in prod, so the lower-environment proxies point at the prod backend on purpose. Pretending you have a dev instance you don't is worse than being honest about it in the routing config.

Why a gateway, not just a helper library

I could have shipped a client library and called it done. A gateway buys things a library can't.

It's one entry point, so a consumer sources what they need without knowing which database, filesystem, or host it lives behind. The auth, rate limiting, and logging live in one place instead of scattered across callers. And it gives you somewhere to stand for the next step: changing a backend without breaking everyone, because they all go through the same door.

Where it pointed next

Two prototypes came out of this that I still think were the right direction.

One was a unified GraphQL schema over the data products: instead of calling a specific service, you pick the fields you want and the layer sources and joins them for you. A buffet, not a menu of fixed endpoints.

The other was putting the same proxy pattern over the Snowflake SQL API, so loading data from object storage into a warehouse was one governed call through the gateway rather than a pile of credentials and SQL passed around by hand.

What it taught me

Most of the value here wasn't clever. It was taking a real but boring sequence that everyone reimplemented badly and doing it once, correctly, behind a clean interface. That is most of what an API layer is for. The interesting engineering was in the honest parts: routing that admits which environments are real, tokens that refresh before they bite you, and error responses that say what actually went wrong instead of leaking the RPC envelope underneath.