feat(gateway): add Anthropic Messages API support to openrouter proxy#1122
feat(gateway): add Anthropic Messages API support to openrouter proxy#1122kilo-code-bot[bot] wants to merge 14 commits intomainfrom
Conversation
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| const usage = responseJson?.usage as MessagesApiUsage | undefined; | ||
|
|
||
| const responseContent = | ||
| responseJson?.content |
There was a problem hiding this comment.
WARNING: Non-streaming error responses throw during usage parsing
When OpenRouter returns an error JSON payload for /messages, content is usually missing. The unguarded .filter().map().join() chain throws before hasError can be recorded, so failed non-streaming requests skip usage accounting and can bubble an unexpected exception out of countAndStoreUsage.
Code Review SummaryStatus: 8 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
Other Observations (not in diff)None. Files Reviewed (13 files)
|
| : Array.isArray(systemContent) | ||
| ? systemContent.map(b => b.text).join('\n') | ||
| : null; | ||
| const lastUserMessage = request.body.messages.filter(m => m.role === 'user').at(-1); |
There was a problem hiding this comment.
WARNING: Tool-result turns erase the user prompt for abuse checks
Anthropic tool loops send tool results as a user message. When that is the latest user turn, this picks it, filters out the non-text blocks, and returns null even though an earlier user text prompt is still in history. That leaves the abuse classifier blind on follow-up tool calls.
| const cacheWriteTokens = usage?.cache_creation_input_tokens ?? 0; | ||
|
|
||
| // OpenRouter path: cost fields are present directly in usage | ||
| if (usage?.cost != null || usage?.is_byok != null) { |
There was a problem hiding this comment.
WARNING: Vercel-routed Messages requests are recorded with zero cost
This only understands OpenRouter's extra usage.cost / is_byok fields. The PR also allows GatewayMessagesRequest to route through Vercel for BYOK, and Anthropic-compatible responses there only carry token counts, so these requests fall through to the zero-cost path and market_cost is underreported.
| ? body.system.map(b => b.text).join('\n') | ||
| : ''; | ||
|
|
||
| const lastUserMessage = body.messages.filter(m => m.role === 'user').at(-1); |
There was a problem hiding this comment.
WARNING: Follow-up tool calls lose the logged user prompt
The Messages API encodes tool results as a user message with tool_result blocks. When that message is last, this extractor emits an empty prefix instead of the previous natural-language user turn, so prompt logging and downstream analytics lose the actual request text.
src/lib/kilo-auto-model.ts
Outdated
| const resolved = resolveAutoModel(model, modeHeader); | ||
| request.body.model = resolved.model; | ||
| if (resolved.reasoning) request.body.reasoning = resolved.reasoning; | ||
| if (resolved.reasoning && request.kind === 'chat_completions') { |
There was a problem hiding this comment.
WARNING: Auto-model reasoning is dropped on /messages
resolveAutoModel() still returns reasoning for modes like plan, general, and debug, but this guard only copies it into chat-completions requests. GatewayMessagesRequest already has a thinking field, so /messages requests using kilo-auto/frontier or kilo-auto/balanced will silently lose the extra reasoning configuration and behave differently from the other gateway APIs.
# Conflicts: # src/app/api/openrouter/[...path]/route.ts # src/lib/processUsage.ts
| const userId = generateProviderSpecificHash(user.id, provider); | ||
| if (requestBodyParsed.kind === 'messages') { | ||
| requestBodyParsed.body.metadata = { user_id: userId }; | ||
| requestBodyParsed.body.user = userId; |
There was a problem hiding this comment.
WARNING: Non-standard Messages fields are sent to Anthropic-compatible backends
GatewayMessagesRequest is widened with user/session_id, but Anthropic's Messages request schema does not define either field. getProvider() can still route /messages through Vercel/BYOK, so this branch will forward those keys to Anthropic-compatible /messages endpoints, where they can be rejected as unknown parameters even when the model otherwise supports the Messages API.
| let messageId: string | null = null; | ||
| let model: string | null = null; | ||
| let responseContent = ''; | ||
| const reportedError = statusCode >= 400; |
There was a problem hiding this comment.
WARNING: Streaming error events are recorded as successful usage
reportedError is fixed from the initial HTTP status and never flips once SSE processing starts. Anthropic-style /messages streams can surface failures as in-band type: "error" events after the response has already started, so this path still returns hasError: false and logs the request as a successful completion.
| return; | ||
| } | ||
|
|
||
| //if (json.type === 'error') { |
There was a problem hiding this comment.
WARNING: Streamed Messages errors are recorded as successful usage
Anthropic can send { type: 'error' } inside a 200 SSE response. With this branch commented out, reportedError never flips to true, so failed /messages streams can still be logged with hasError: false and look like successful requests in usage tracking.
1 participant
Summary
Adds support for the Anthropic Messages API to the OpenRouter proxy route (
/api/openrouter/messagesand/api/gateway/messages), in addition to the existing OpenAI chat completions (/chat/completions) and Responses API (/responses) support.Key changes:
types.ts: AddedGatewayMessagesRequesttype (Anthropic Messages format withmodel,max_tokens,messages,system,stream,tools, etc.) and extended theGatewayRequestdiscriminated union with themessageskind.route.ts: ExtendedvalidatePathto accept/messages, added body parsing for the messages format, applied the same admin-only guard as the Responses API, handled prompt info extraction and free-model rewriting for the new kind.processUsage.messages.ts(new file): Streaming and non-streaming usage parsing for Anthropic's SSE format (message_start→ input tokens,message_delta→ output tokens + stop reason), with OpenRouter cost field handling mirroring the existing chat completions and responses parsers.processUsage.ts: Wired the newmessagesapi_kind intocountAndStoreUsage.request-helpers.ts:getMaxTokensnow handles the messages kind (returnsmax_tokens).api-metrics.server.ts:getToolsAvailableandgetToolsUsedhandle Anthropic tool format (tools have a top-levelname, tool use appears astool_usecontent blocks in assistant messages).abuse-service.ts:extractFullPromptshandles the messages kind (top-levelsystemfield + user message content extraction).providers/index.ts:openRouterRequestbody parameter type updated to includeGatewayMessagesRequest.The Messages API endpoint is gated behind
is_admin(same as the Responses API) while it's experimental.Verification
message_startcarriesusage.input_tokens,message_deltacarriesusage.output_tokens.Visual Changes
N/A
Reviewer Notes
/messages) is forwarded as-is to OpenRouter at${provider.apiUrl}/messages— OpenRouter supports the native Anthropic Messages API format at this path.wrapInSafeNextResponse.customLlmRequestonly handles chat completions.applyAnthropicModelSettingsare already guarded behindkind === 'chat_completions'; clients using the native Messages API are responsible for their own cache control markup.