proxy: translate chat requests between openai/anthropic/ollama protocols#666
proxy: translate chat requests between openai/anthropic/ollama protocols#666nightscape wants to merge 2 commits into
Conversation
Add a new proxy/translate package that decodes inbound chat requests into a canonical IR and re-encodes them for the upstream model's native protocol. When the request's protocol is already supported, forwarding is still a zero-copy pass-through. - new ModelConfig.Protocols field (default [openai, anthropic], preserving pre-translation behavior bit-for-bit) - parsers + emitters for OpenAI, Anthropic and Ollama chat endpoints - stateless reversible tool-call ID mapping across protocols - streaming translation via an http.ResponseWriter wrapper that rewrites Content-Type, strips Content-Length, and translates SSE/NDJSON frames - Ollama API surface: /api/chat, /api/generate, /api/tags, /api/show - header bridging (x-api-key <-> Authorization Bearer, anthropic-version) - sendErrorResponse now emits the client's native error envelope Peer-routed models fall back to 501 when the inbound protocol is Ollama; cross-peer translation is a follow-up once peers advertise their protocols. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
TestConfig_LoadWindows mirrors TestConfig_LoadPosix but was missed in the earlier fixture update, causing CI to fail on the windows job. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Hi, see the llama-swappo fork which has these endpoints. That would the place to contribute ollama features. |
Hey, while i can kinda understand this, i would love to have the ollama functionality here instead of some fork which might or might not get updated with upstream. |
Summary
proxy/translatepackage: canonical OpenAI-shaped IR, with parsers/emitters for OpenAI, Anthropic and Ollama chat endpoints (both non-streaming and streaming).ModelConfig.Protocolsdeclares which wire formats each model natively supports. Default[openai, anthropic]preserves existing behaviour bit-for-bit./api/chat,/api/generate,/api/tags(filtered byprotocols),/api/show.x-api-key↔Authorization: Bearer, defaultanthropic-version) and protocol-aware error envelopes fromsendErrorResponse.Status
Draft. Covered by
go test ./proxy/translate/...(IR, cross-protocol, ID round-trip, stream parsers/emitters) andmake test-devis green. End-to-end integration tests against a real upstream andresponse_format=json_schematranslation are listed as follow-ups in the plan.