How it is actually built.
Written for the technically curious and for anyone thinking about building on top of it. No architecture diagram that flatters the architecture.
- Engine
- A Python service speaking the OpenAI chat-completions dialect.Each of the six agents maps to its own provider and model lane, with per-lane fallbacks and key rotation when a provider rate-limits.
- Orchestration
- Agentic multi-step work runs as a durable plan, not a prompt.Steps, budgets and recovery are recorded, so an interrupted run resumes instead of restarting.
- Execution truth
- A ledger of which provider and model actually served each request.Responses are validated against their own claims — a download link that does not resolve never reaches you.
- Image studio
- FLUX on RafayGen's own GPU through a ComfyUI pipeline.Text-to-image, img2img, edit, upscale, describe.
- Frontend
- Next.js App Router; the voice orb is an STT → chat → streaming-TTS loop.
- Urdu OCR
- A multi-pass pipeline producing searchable text and rebuilt PDFs.Nastaliq is the hard part; general-purpose OCR reliably fails at it.
- Infrastructure
- Self-operated, down to the authoritative nameservers.One person holds the pager.
A single-provider assistant inherits that provider's outages, rate limits and price changes as if they were its own.
Routing per task means a fast lane stays fast, a reasoning lane stays deep, and one upstream failure degrades a corner of the product instead of taking it down. It also means the engineering worth talking about is the routing, the agents, the artifact pipeline and the Urdu handling — not the base models, which are somebody else's work and are credited as such.
The cost is real: six lanes are six sets of quirks, six failure modes and six ways for a provider to change something without telling you. Most of the engine's defensive code exists because of that trade.
API access
The engine speaks OpenAI-compatible chat completions internally, and a public API surface runs at api.rafaygen.com. Self-serve keys are not open yet — if you want to build on RafayGen, say what you are building and you will get a straight answer about whether it is ready for that.
The most useful thing here for a sceptic.
Deterministic grading, published raw answers, and RafayGen ranked alongside the models it routes to rather than exempt from the comparison.