openai-nim-proxy (LLM Gateway)

The gateway never fakes OpenAI model aliases. Each provider's models are exposed under explicit, prefixed IDs — nim:, chutes:, ollama:, g4f:, openrouter: — so a client always knows where a request is going, and the gateway never maintains a fragile mapping table of fake OpenAI names to real upstream models.

A model registry discovers models from every configured provider in parallel (one provider being down never breaks the union), dedupes by ID, and persists to a cache with a configurable TTL. A round-robin key pool selects across multiple keys per provider with exponential cooldown and Retry-After honoring, so many free-tier OpenRouter keys can sit behind one stable endpoint.

On top of the routing core it ships a built-in browser chat UI backed by SQLite with per-message provider/model attribution, a TTS benchmark UI probing local Kokoro/KittenTTS/Piper engines through one OpenAI-style /v1/audio/speech endpoint, an optional GPT4Free sidecar (chat + image generation) as a separate Docker container, and a Responses-API translator so Codex CLI 0.136+ can keep talking to chat-completions-only providers.

Key features

• OpenAI-compatible /v1/models, /v1/chat/completions, /v1/responses, /v1/audio/speech, /v1/images/generations.
• Explicit provider-prefixed model IDs (nim:, chutes:, ollama:, g4f:, openrouter:) — no alias impersonation.
• Round-robin key pool with exponential cooldown + Retry-After for free-tier pooling.
• SQLite-backed browser chat UI with per-message provider/model attribution.
• TTS benchmark UI against Kokoro / KittenTTS / Piper via one OpenAI-style endpoint.
• Responses-API translator for Codex CLI 0.136+ (chat-completions-only providers).
• Tool-call pass-through (streaming + non-streaming) + a mock-upstream probe.
• Bounded debug logging (ring buffer + capped file); systemd VPS install scripts.

Stack