Skip to content

test: add comprehensive structured output support coverage#1140

Merged
homanp merged 1 commit into
superagent-ai:mainfrom
frankentini:test/structured-output-coverage
Apr 11, 2026
Merged

test: add comprehensive structured output support coverage#1140
homanp merged 1 commit into
superagent-ai:mainfrom
frankentini:test/structured-output-coverage

Conversation

@frankentini
Copy link
Copy Markdown
Contributor

Summary

Adds 37 unit tests covering the supportsStructuredOutput logic across all supported providers and model combinations. Previously only 2 tests existed for this surface (OpenAI + one Groq negative case).

What's tested

Provider Tests Coverage
OpenAI 2 gpt-4o, gpt-4o-mini
Anthropic 4 sonnet-4-5 ✓, opus-4-1 ✓, haiku-4-5 ✗, claude-3-haiku ✗
Google 2 gemini-2.0-flash, gemini-1.5-pro
Bedrock 1 claude-haiku on bedrock
Vercel 1 openai via vercel gateway
Groq 4 gpt-oss-20b ✓, llama-4-maverick ✓, llama-3.3-70b ✗, mixtral ✗
Fireworks 3 gpt-oss-120b ✓, llama-4-scout ✓, llama-v3p1-8b ✗
OpenRouter 15 OpenAI, Anthropic, Google Gemini/Gemma, Meta Llama, Mistral, DeepSeek, Qwen, Cohere, x-ai sub-providers
OpenAI-compatible 3 env var enabled/disabled/default
Unknown 1 fallback behavior

All tests use the existing mock pattern (vi.mock callProvider) and verify whether responseFormat is passed as the 3rd argument.

Motivation

Relates to #1111 — moving test coverage toward mocked unit tests. The structured output routing logic is complex with many provider-specific branches, and was previously under-tested.

Test results

All 37 new tests pass. No changes to source code.

Add 37 unit tests verifying structured output (responseFormat) behavior
across all supported providers and model combinations.

Covers: OpenAI, Anthropic, Google, Bedrock, Vercel, Groq, Fireworks,
OpenRouter (with sub-provider variants), and OpenAI-compatible.

Tests both positive cases (responseFormat passed) and negative cases
(responseFormat omitted) to ensure the supportsStructuredOutput logic
is correct for each provider/model combination.

Relates to superagent-ai#1111
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 3, 2026

@frankentini is attempting to deploy a commit to the Superagent Team on Vercel.

A member of the Team first needs to authorize it.

@homanp
Copy link
Copy Markdown
Collaborator

homanp commented Apr 3, 2026

@frankentini thanks!

@homanp homanp merged commit 5adc62d into superagent-ai:main Apr 11, 2026
3 of 5 checks passed
richardokonicha pushed a commit to fugoku/superagent that referenced this pull request May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants