Skip to content

feat: Chat & Responses APIs, core improvements, and docs overhaul#1

Merged
codybrom merged 17 commits into
mainfrom
feat/openai-compat
Mar 12, 2026
Merged

feat: Chat & Responses APIs, core improvements, and docs overhaul#1
codybrom merged 17 commits into
mainfrom
feat/openai-compat

Conversation

@codybrom

@codybrom codybrom commented Mar 11, 2026

Copy link
Copy Markdown
Owner

Summary

Adds Chat-style and Responses-style API interfaces (tsfm-sdk/chat), a complete documentation site overhaul, and core SDK improvements for the 0.3.0 release.

import Client from "tsfm-sdk/chat";

const client = new Client();
const response = await client.chat.completions.create({
  model: "SystemLanguageModel",
  messages: [{ role: "user", content: "Hello" }],
});

Chat & Responses APIs (tsfm-sdk/chat)

  • Chat Completions API (client.chat.completions.create()) — full message history, streaming, structured output (json_schema), and tool calling
  • Responses API (client.responses.create()) — string or structured input, function tools, and streaming via ResponseStream
  • Parameter mapping: temperature, max_tokens/max_completion_tokens, top_p, seed → native GenerationOptions; unsupported params warned at runtime
  • Error mapping: ExceededContextWindowSizeErrorfinish_reason: "length", GuardrailViolationErrorfinish_reason: "content_filter", RefusalErrormessage.refusal, RateLimitedError → HTTP 429
  • Stream and ResponseStream async iterables with toReadableStream(), close(), Symbol.dispose, and FinalizationRegistry cleanup
  • Tool calling via structured output with $defs/$ref schemas to prevent parameter name collisions
  • JSON key reordering utility to match schema-defined property order

Core improvements

  • SystemLanguageModel — named public class with availability checks and model warmup
  • ServiceCrashedError — detects crashed generativeexperiencesd service with recovery instructions
  • Symbol.dispose on SystemLanguageModel, LanguageModelSession, Tool, and Client for TC39 Explicit Resource Management
  • Typed transcript entries: TranscriptEntry, TranscriptContent, TranscriptToolCall, etc. with transcript.entries() method
  • JsonSchema and JsonObject exported types
  • Automatic session cleanup on process.exit, SIGINT, and SIGTERM
  • Enhanced afmSchemaFormat() with recursive normalization, $defs/$ref support, and x-order fields
  • Tool callback error handling: synchronous errors in call() now invoke FMBridgedToolFinishCall() with error message to prevent session hang
  • Enhanced statusToError(): maps ModelManagerError Code=1041 to InvalidGenerationSchemaError

Documentation

  • Complete Chat & Responses API guide, API reference, and examples page
  • Docs site visual overhaul: brand colors shifted to teal, Apple-style typography, WCAG AA contrast fixes
  • Landing page redesigned with code examples and API showcase
  • Swift-equivalent references extracted into caption-style info boxes across all guide pages
  • 6 new examples in examples/compat/ demonstrating Chat Completions and Responses API

Known limitations

  • Single tool call per turn (on-device structured output generates one response object)
  • Streaming + structured output / tools: response is buffered internally, emitted as a single chunk
  • usage is always null (native API doesn't expose token counts)
  • choices always contains exactly one entry (n > 1 not supported)
  • Non-text content parts (image_url, input_audio, file, refusal) are accepted in types but ignored at runtime with a warning

Test plan

  • npm run test — 399 tests passing (unit + integration)
  • npm run typecheck — clean
  • npm run lint — clean
  • All 6 compat examples run successfully

Drop-in replacement for the OpenAI Node SDK — users swap their import
to `import OpenAI from "tsfm-sdk/openai"` and existing
`client.chat.completions.create()` code runs against Apple Intelligence
on-device inference.

Implements: message/transcript mapping, parameter mapping, streaming,
tool calling via structured output, json_schema/json_object response
formats, and OpenAI-style error mapping.

Consolidates vitest config into a single file with named projects
(unit/integration), replacing the deprecated vitest.workspace.ts.

100% test coverage across all metrics (252 unit tests).
…del warmup

- Add Symbol.dispose to LanguageModelSession, SystemLanguageModel, and Tool
  for TC39 Explicit Resource Management (`using` keyword)
- Add SIGINT/SIGTERM handlers to clean up native sessions on forced termination
- Add ServiceCrashedError with detection and recovery instructions for crashed
  Apple Intelligence safety service
- Replace synchronous isAvailable() with waitUntilAvailable() in integration
  tests to handle model warmup after service restarts
- Rewrite tools integration test with secret-based assertion to guarantee
  tool invocation
- Add ESNext.Disposable lib to tsconfig
… reliability

- Restructure tool schema to use $defs/$ref (fixes Apple Code=1041 rejections)
- Improve tool-calling prompt reliability with structured instructions
- Handle tool-as-last-message flow (OpenAI tool result continuation)
- Add JSON key reordering to match OpenAI schema property order
- Add try/catch in tool callback to prevent session deadlocks
- Fix integration tools test hanging with Promise.race timeout pattern
- Map ModelManagerError Code=1041 to InvalidGenerationSchemaError
- Add example runner script, cross-provider example, and OpenAI comparison
- Add typecheck and example npm scripts; fix test:coverage to run all projects
- 300 tests, 100% coverage
@codybrom codybrom self-assigned this Mar 11, 2026
…st reliability

- GuardrailViolationError now returns finish_reason: "content_filter" instead
  of throwing, matching OpenAI's behavior for content policy violations
- Widened compat types to match OpenAI SDK: content part types (input_audio,
  file, refusal), assistant/tool message content arrays, content_filter in
  finish_reason enums
- extractText now warns on all non-text content parts generically
- Added 7 missing unsupported params (verbosity, web_search_options,
  prompt_cache_key, prompt_cache_retention, safety_identifier, function_call,
  functions)
- Tools integration test: 1-in-5 retry with debug logging and transcript dump
- Rewrite getting-started with intro paragraph, Key Concepts using Apple's
  terminology (Apple Intelligence, Foundation Models, SystemLanguageModel)
- Expand all error-handling descriptions with causes and recovery steps
- Add Apple developer doc links across guide pages (model-configuration,
  sessions, streaming, structured-output, tools, transcripts, generation-options)
- Merge JSON Schema page into structured-output
- Add interface comments for OpenAI class, Completions.create(), Stream,
  ChatCompletionCreateParams, UNSUPPORTED_PARAMS, ServiceCrashedError
- Fix duplicate JSDoc on afmSchemaFormat, fix ServiceCrashedError message
- Update terminology: "Apple" → "Foundation Models" / "AFM" in source comments
- Style: transparent inline code backgrounds, heading code inherits font size
Add TranscriptEntry types and entries() accessor to Transcript class,
enabling direct inspection of transcript history without manual JSON
parsing. Export all new types from the public API. Update guide pages
with improved terminology, guardrails docs, and tool best practices.
Implement OpenAI Responses API (client.responses.create()) alongside
existing Chat Completions API with text, streaming, structured output,
tool calling, and error mapping. Add side-by-side integration tests
comparing both APIs, comprehensive unit tests (100% coverage on
responses.ts), full documentation (guide, API reference, examples),
and fix lint issues (no-this-alias, varsIgnorePattern, MD024 headings).
… improvements

- Rename MODEL_APPLE_INTELLIGENCE to MODEL_DEFAULT with value "SystemLanguageModel"
- Add Responses API examples for both real OpenAI and tsfm (basic + advanced)
- Add content clues to doc sections identifying which API is discussed
- Fix MD024 duplicate headings across API and guide docs
- Update all tests and examples for new model identifier
…d Swift callouts

- Shift brand from green to teal (light) and mint to #29d297 (dark) for readability
- Darken text-3 to #707074 for WCAG AA contrast
- Add Apple-style font rendering, sizing, and letter-spacing
- Code blocks word-wrap, inline code uses inherited color with subtle bg
- Subtler table alternating rows
- Landing page: hero + code example + OpenAI compat box replacing feature grid
- Extract Swift-equivalent references into caption-style info boxes across all guide pages
- Rewrite structured-output "Picking a Schema Format" section
- OpenAI compat: add Responses API, extract shared utils, tool param validation, stream error propagation, Symbol.dispose, integration tests
@codybrom codybrom changed the title feat: OpenAI-compatible API layer with tool calling feat: OpenAI-compatible API layer, Responses API, and docs overhaul Mar 12, 2026
@codybrom codybrom marked this pull request as ready for review March 12, 2026 04:51
@codybrom codybrom requested a review from Copilot March 12, 2026 04:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands the SDK with an OpenAI-compatible API surface (tsfm-sdk/openai), improves schema/transcript ergonomics, and updates test/CI configuration to support the new functionality.

Changes:

  • Added an OpenAI compatibility layer (Chat Completions + Responses APIs), including streaming wrappers, tool-calling helpers, and OpenAI-style type definitions.
  • Improved SDK ergonomics: typed transcript entries accessors, Symbol.dispose support, and more robust JSON schema normalization.
  • Reworked test setup (unit vs integration), added extensive new unit/integration coverage, and updated CI workflow/docs/examples accordingly.

Reviewed changes

Copilot reviewed 75 out of 78 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
vitest.workspace.ts Removed Vitest workspace file (previously enumerated unit/integration configs).
vitest.config.ts Updated Vitest configuration to define unit/integration projects + coverage settings.
tsconfig.json Added ESNext.Disposable lib support for Symbol.dispose.
tests/unit/transcript.test.ts Updated transcript JSON shape expectations; added entries() tests.
tests/unit/tool.test.ts Added tests for sync throws in tool callback + Symbol.dispose.
tests/unit/session.test.ts Added tests for exit/signal cleanup and transcript guard; removed unused-vars suppression.
tests/unit/schema.test.ts Expanded tests for recursive JSON schema normalization behavior.
tests/unit/helpers/mock-bindings.ts Adjusted mock signature for transcript creation binding.
tests/unit/errors.test.ts Added coverage for ServiceCrashedError mapping and hierarchy.
tests/unit/core.test.ts Added Symbol.dispose coverage for SystemLanguageModel.
tests/unit/compat/transcript.test.ts New: tests for OpenAI-messages → native transcript conversion.
tests/unit/compat/tools.test.ts New: tests for tool instruction/schema builders + tool response parsing.
tests/unit/compat/stream.test.ts New: tests for compat Stream async iteration + cleanup semantics.
tests/unit/compat/responses-stream.test.ts New: tests for compat ResponseStream iteration + cleanup semantics.
tests/unit/compat/params.test.ts New: tests for OpenAI params → GenerationOptions mapping + warnings.
tests/integration/vitest.config.ts Removed dedicated integration Vitest config (migrated to project config).
tests/integration/transcript.test.ts Switched to waitUntilAvailable before running integration suite.
tests/integration/tools.test.ts Strengthened tool integration test with retries/timeouts and forced tool usage.
tests/integration/structured-output.test.ts Switched to waitUntilAvailable before running integration suite.
tests/integration/streaming.test.ts Switched to waitUntilAvailable before running integration suite.
tests/integration/responses.test.ts New: integration tests for compat Responses API and parity with Chat Completions.
tests/integration/helpers/retry.ts New: retry helper for flaky on-device integration tests.
tests/integration/compat.test.ts New: integration coverage for OpenAI compat layer flows (incl. tool calling).
tests/integration/basic.test.ts Switched to waitUntilAvailable before running integration suite.
src/transcript.ts Added typed transcript entry/content/tool-call types + entries() helper; tightened dict typing.
src/tool.ts Ensured tool calls finish on synchronous errors; added Symbol.dispose.
src/session.ts Added exit/signal cleanup tracking; added transcript initialization guard; added Symbol.dispose.
src/schema.ts Added JsonSchema/JsonObject types; made afmSchemaFormat recursively normalize nested schemas.
src/options.ts Tightened serialization typing for generation options/sampling.
src/index.ts Exported new transcript/schema types and new error type.
src/errors.ts Added ServiceCrashedError; improved statusToError mapping for specific crash signatures.
src/core.ts Added Symbol.dispose for SystemLanguageModel.
src/compat/utils.ts New: JSON key reordering helper + compat error utilities.
src/compat/types.ts New: OpenAI-compatible Chat Completions type definitions.
src/compat/transcript.ts New: OpenAI messages → Foundation Models transcript conversion.
src/compat/tools.ts New: tool instructions/schema builder + structured tool-call parsing.
src/compat/stream.ts New: OpenAI-like Stream wrapper with cleanup semantics.
src/compat/responses-types.ts New: OpenAI Responses API type definitions.
src/compat/responses-stream.ts New: Responses streaming wrapper with cleanup semantics.
src/compat/params.ts New: mapping OpenAI params to SDK generation options + warnings.
src/compat/index.ts New: main OpenAI compat client implementation (tsfm-sdk/openai).
scripts/run-example.ts New: helper to run one/all examples via npm run example.
README.md Updated CI badge link to new workflow.
package.json Bumped version to 0.3.0; added ./openai export; added scripts; added dev dep openai.
package-lock.json Updated lock for 0.3.0 + added openai dependency tree.
examples/compat/responses-real.ts New: example using real OpenAI Responses API (requires API key).
examples/compat/responses-local.ts New: example using local compat Responses API.
examples/compat/responses-advanced-real.ts New: advanced real OpenAI Responses API example.
examples/compat/responses-advanced-local.ts New: advanced local compat Responses API example.
examples/compat/openai-real.ts New: side-by-side real OpenAI Chat Completions example.
examples/compat/openai-compat.ts New: side-by-side local compat Chat Completions example.
examples/compat/cross-compat.ts New: cross-provider transcript handoff example (cloud → local → cloud).
eslint.config.js Adjusted unused-vars rule to ignore vars prefixed with _.
docs/index.md Updated docs home layout with OpenAI compat callout and quick links.
docs/guide/transcripts.md Expanded transcript docs; documented entries() and entry fields.
docs/guide/tools.md Expanded tools docs; added best practices + compat tool calling note.
docs/guide/structured-output.md Expanded structured output docs; added JSON schema usage and guidance.
docs/guide/streaming.md Expanded streaming docs; added compat streaming usage + cleanup notes.
docs/guide/sessions.md Expanded sessions docs; added state checking + compat mention.
docs/guide/model-configuration.md Expanded model config docs; clarified availability waiting behavior.
docs/guide/json-schema.md Removed standalone JSON schema guide (folded into structured output docs).
docs/guide/getting-started.md Expanded getting-started, clarified platform constraints and next steps.
docs/guide/generation-options.md Expanded options docs; clarified sampling semantics + Swift equivalents.
docs/guide/error-handling.md Expanded error docs; documented ServiceCrashedError and recovery steps.
docs/examples/openai-compat.md New: example guide for OpenAI compat layer usage.
docs/examples/index.md Added OpenAI compat examples entry.
docs/api/openai-compat.md New: API reference for tsfm-sdk/openai.
docs/api/index.md Linked OpenAI compat API docs and showed import usage.
docs/.vitepress/theme/custom.css Major theme tweaks + home page layout styles + accessibility adjustments.
docs/.vitepress/config.ts Added OpenAI compat links in nav/sidebars; enabled external link icon.
.gitignore Added .env.local to ignore.
.github/workflows/test.yml Removed old Tests workflow.
.github/workflows/ci.yml Added new CI workflow (tests + dylib build verification).
.env.local.example New: template for OpenAI API key for real OpenAI comparison examples.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread vitest.config.ts
Comment thread src/session.ts
Comment thread src/session.ts
Comment thread src/schema.ts
Comment thread src/compat/utils.ts
Comment thread src/compat/tools.ts
Comment thread src/transcript.ts
Drop "OpenAI Compatible" branding in favor of industry-standard
terminology. The compat layer is now framed as Chat-style and
Responses-style APIs rather than an OpenAI-specific feature.

- Export path: tsfm-sdk/openai → tsfm-sdk/chat
- Class: OpenAI → Client (import Client from "tsfm-sdk/chat")
- Doc files renamed: openai-compatibility → chat-api, openai-compat → chat
- Remove cross-compat example and cross-provider doc sections
- Update all imports, tests, examples, and JSDoc comments
@codybrom codybrom changed the title feat: OpenAI-compatible API layer, Responses API, and docs overhaul feat: Chat & Responses APIs, core improvements, and docs overhaul Mar 12, 2026
- Add Array.isArray guard for tuple schemas in afmSchemaFormat
- Handle arrays of objects in reorderJson key ordering
- Fix misleading JSDoc in buildToolSchema (merged params, not namespaced)
- Validate entries() return with Array.isArray
- Use process.once for SIGINT/SIGTERM to prevent handler loop
- Use WeakRef for session tracking to allow GC of forgotten sessions
@codybrom codybrom merged commit c87cb65 into main Mar 12, 2026
3 checks passed
@codybrom codybrom deleted the feat/openai-compat branch March 12, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants