jrdnhnz.dev
← Back to Blog

September 1, 2026 · 8 min read

Designing a Multi-Provider AI Gateway: Abstraction, Billing & a Reseller Tier

How I architected MecutinAI — one /v1 endpoint across many LLM providers, with token accounting, QRIS billing, and a reseller (agency) tier.

aiarchitecturellmmecutinai

Every app that wants to use AI ends up juggling provider SDKs — OpenAI here, Anthropic there, an image model somewhere else. Each has its own auth, its own payload shape, its own error conventions. MecutinAI exists to collapse that into one surface, and to let resellers sell it as their own.

The unified contract

The gateway exposes a small, stable surface: POST /v1/chat, /v1/embeddings, /v1/images, GET /v1/models. Clients integrate once. Adding or swapping a provider is a backend-only change that no client ever sees.

The trick is a dual-layer provider abstraction. The outer layer speaks the unified contract; the inner layer is a per-provider adapter that translates between the unified shape and each upstream SDK. A request comes in, the router picks the provider for that model, the adapter normalizes the payload, calls upstream, then normalizes the response back.

// Simplified: a provider adapter implements one normalize-in, one call, one normalize-out.
interface ProviderAdapter {
  toProviderRequest(unified: ChatRequest): ProviderPayload;
  call(payload: ProviderPayload): Promise<ProviderResponse>;
  fromProviderResponse(resp: ProviderResponse): ChatResponse;
}

New provider = new adapter. The rest of the system doesn't move.

Token accounting and rate limiting

Billing only works if you can trust the numbers. Token counting lives in one place (js-tiktoken), applied identically across every model, so a credit deduction is consistent whether the call hit Provider A or Provider B. Rate limiting is per API key with a token bucket — per identity, not just per IP.

The reseller (agency) tier

This is the part I'm proudest of. An agency is a reseller account: it gets its own branded surface, its own plans and pricing, its own key pool, and its own dashboard — but it rides on the same gateway. Checkout is QRIS (Indonesian standard), vouchers work, and usage rolls up to both the agency and the platform.

The hard part isn't the billing — it's keeping tenant boundaries clean. An agency's keys must never leak into another agency's quota, and a plan change must propagate atomically. That's enforced at the adapter layer, not in UI code, so it can't be bypassed.

Why it pays off

  • Clients integrate once, against one contract.
  • Provider switching is invisible to clients.
  • Billing, rate limiting, and observability live in one place.
  • Resellers get a turnkey product; the platform gets distribution.

Building this taught me that the value of an API gateway isn't the proxying — it's the contract you commit to, and the discipline of keeping every provider-specific ugliness behind one adapter.