FluxRouter

Enterprise LLM hub: routes prompts to OpenAI, Anthropic, Gemini, and self-hosted models behind one API — with team budgets, SSO/SCIM, and tamper-evident audit logs

By the numbers

LLM providers

SSO + SCIM

Auth standards

hash-chain

Audit log

WebSocket

Streaming protocol

The Problem

What I was solving

Engineering teams want to use the best LLM for the job — Claude for code review, GPT for general reasoning, Gemini for long context, a self-hosted Llama for sensitive data. In practice they juggle four browser tabs, four billing relationships, and four sets of API keys. There's no single dashboard showing who spent what, no shared prompt library, no audit trail when compliance asks. Existing tools either lock you to one vendor or skip the enterprise side entirely.

My Approach

How I built it

Multi-tenant from day one. Laravel 12 backend with DDD/CQRS — provider adapters, billing, team management, and audit logging live in separate domains and cannot leak into each other. Four providers (OpenAI, Anthropic, Gemini, self-hosted OSS via vLLM/Ollama) sit behind a single internal interface; adding a fifth is ~50 lines. WebSocket streaming gives true two-way control — clients cancel generation mid-stream without dropping the connection. Frontend is Next.js 16 + React 19 with Feature-Sliced Design enforced by Steiger lint. Token budgets are enforced server-side, not just displayed; SSO/SCIM via standard SAML2 + OIDC; audit log uses append-only PostgreSQL 17 tables with hash-chained rows for tamper evidence.

Tech choices

Laravel 12 + DDD/CQRS— Domain-Driven Design separates provider adapters from billing and team logic — swapping a provider doesn't touch audit code. CQRS keeps the read side fast for dashboards while writes go through strict domain events.
WebSocket streaming— SSE was tempting, but WebSocket gives two-way control — clients cancel generation mid-stream without closing the connection. Critical for enterprise UX where stopping a runaway response saves real money.
Next.js 16 + React 19 + FSD— Feature-Sliced Design with Steiger lint catches architectural drift before it ships. React 19's new transitions and Suspense boundaries make streaming UI feel native. Next.js 16 server components keep the admin panel fast even with thousands of audit rows.
PostgreSQL 17 + hash-chained audit— Append-only tables with each row's hash including the previous row's hash give tamper-evidence without a separate ledger. Compliance teams can verify the chain offline. Postgres 17 incremental backups make point-in-time recovery cheap.

Outcome

What came out of it

[in development] Provider routing layer — all four backends streaming through one API; team/role/budget data model done; SSO + SCIM working end-to-end with Okta and Azure AD; hash-chained audit log with verification tooling. Frontend chat, prompt library, and team admin shipped. In flight: per-org cost-attribution analytics, prompt-quality A/B testing, on-prem deployment package.