Skip to content

proxy: translate chat requests between openai/anthropic/ollama protocols#666

Draft
nightscape wants to merge 2 commits into
mostlygeek:mainfrom
nightscape:protocol-translation
Draft

proxy: translate chat requests between openai/anthropic/ollama protocols#666
nightscape wants to merge 2 commits into
mostlygeek:mainfrom
nightscape:protocol-translation

Conversation

@nightscape

Copy link
Copy Markdown

Summary

  • New proxy/translate package: canonical OpenAI-shaped IR, with parsers/emitters for OpenAI, Anthropic and Ollama chat endpoints (both non-streaming and streaming).
  • ModelConfig.Protocols declares which wire formats each model natively supports. Default [openai, anthropic] preserves existing behaviour bit-for-bit.
  • When a client's protocol is supported: zero-cost pass-through. When it isn't: translate request + response (including SSE/NDJSON streaming and tool-use), with stateless reversible tool-call ID mapping.
  • Ollama API surface registered: /api/chat, /api/generate, /api/tags (filtered by protocols), /api/show.
  • Header bridging (x-api-keyAuthorization: Bearer, default anthropic-version) and protocol-aware error envelopes from sendErrorResponse.
  • Peer-routed models return 501 on Ollama mismatch for now; cross-peer translation is a follow-up.

Status

Draft. Covered by go test ./proxy/translate/... (IR, cross-protocol, ID round-trip, stream parsers/emitters) and make test-dev is green. End-to-end integration tests against a real upstream and response_format=json_schema translation are listed as follow-ups in the plan.

Add a new proxy/translate package that decodes inbound chat requests into a
canonical IR and re-encodes them for the upstream model's native protocol.
When the request's protocol is already supported, forwarding is still a
zero-copy pass-through.

- new ModelConfig.Protocols field (default [openai, anthropic], preserving
  pre-translation behavior bit-for-bit)
- parsers + emitters for OpenAI, Anthropic and Ollama chat endpoints
- stateless reversible tool-call ID mapping across protocols
- streaming translation via an http.ResponseWriter wrapper that rewrites
  Content-Type, strips Content-Length, and translates SSE/NDJSON frames
- Ollama API surface: /api/chat, /api/generate, /api/tags, /api/show
- header bridging (x-api-key <-> Authorization Bearer, anthropic-version)
- sendErrorResponse now emits the client's native error envelope

Peer-routed models fall back to 501 when the inbound protocol is Ollama;
cross-peer translation is a follow-up once peers advertise their protocols.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a3638c56-2f27-4665-ad03-1ef0b358c269

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

TestConfig_LoadWindows mirrors TestConfig_LoadPosix but was missed in the
earlier fixture update, causing CI to fail on the windows job.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mostlygeek

Copy link
Copy Markdown
Owner

Hi,

see the llama-swappo fork which has these endpoints. That would the place to contribute ollama features.

@ojsef39

ojsef39 commented May 9, 2026

Copy link
Copy Markdown

Hi,

see the llama-swappo fork which has these endpoints. That would the place to contribute ollama features.

Hey, while i can kinda understand this, i would love to have the ollama functionality here instead of some fork which might or might not get updated with upstream.
Thank you for your consideration! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants