Can I self-host for free? +
Yes. The full source is Apache 2.0 on GitHub. Deploy it anywhere — your own servers, Docker, or Kubernetes — free, with no restrictions on commercial use.
Does Cloud Growth include a HIPAA BAA? +
Not by default. A signed BAA is available on Enterprise (on-prem/VPC). If you're a covered entity handling PHI on Cloud, contact us — we evaluate this case by case.
What counts as a "request"? +
One call to /v1/chat/completions or /v1/embeddings. Streaming counts as one request. Cache hits count as one request but save the LLM cost entirely. We never bill per-token — you pay your LLM provider directly for the underlying model call.
Can I move from self-hosted to Cloud without downtime? +
Yes. The gateway is a drop-in proxy — point your base_url at the Cloud endpoint. Config and provider integrations import directly; no code changes in your applications.
Do you store my LLM responses? +
On Cloud, audit logs are stored encrypted at rest for compliance. The semantic cache stores a hash of the request/response, never raw PHI, and caching can be disabled for PHI traffic in config. On self-hosted or Enterprise, you control all storage.
What LLM providers are supported? +
20+, including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Groq, Mistral, DeepSeek, Cohere, Fireworks, Together AI, Ollama, Perplexity, xAI, AI21, and more. Open a GitHub issue to request one.
Does it support Microsoft Entra ID? +
Yes — native Entra ID OIDC with RS256/ES256 JWT validation against Microsoft's JWKS endpoint, App Role enforcement, and multi-tenant support. Three auth modes: api_key, entra_id, or any (accept both, useful for zero-downtime migration). Included in every edition.
Is there a self-hosted admin portal? +
Yes — the Enterprise edition bundles the full governance portal (cost dashboards, request logs, governance rules, A/B experiments, team management) with the gateway engine, in a separate repository built for regulated industries that need everything on their own infrastructure. Deploys with docker compose up.