feat: Chat & Responses APIs, core improvements, and docs overhaul#1
Merged
Conversation
Drop-in replacement for the OpenAI Node SDK — users swap their import to `import OpenAI from "tsfm-sdk/openai"` and existing `client.chat.completions.create()` code runs against Apple Intelligence on-device inference. Implements: message/transcript mapping, parameter mapping, streaming, tool calling via structured output, json_schema/json_object response formats, and OpenAI-style error mapping. Consolidates vitest config into a single file with named projects (unit/integration), replacing the deprecated vitest.workspace.ts. 100% test coverage across all metrics (252 unit tests).
…del warmup - Add Symbol.dispose to LanguageModelSession, SystemLanguageModel, and Tool for TC39 Explicit Resource Management (`using` keyword) - Add SIGINT/SIGTERM handlers to clean up native sessions on forced termination - Add ServiceCrashedError with detection and recovery instructions for crashed Apple Intelligence safety service - Replace synchronous isAvailable() with waitUntilAvailable() in integration tests to handle model warmup after service restarts - Rewrite tools integration test with secret-based assertion to guarantee tool invocation - Add ESNext.Disposable lib to tsconfig
… reliability - Restructure tool schema to use $defs/$ref (fixes Apple Code=1041 rejections) - Improve tool-calling prompt reliability with structured instructions - Handle tool-as-last-message flow (OpenAI tool result continuation) - Add JSON key reordering to match OpenAI schema property order - Add try/catch in tool callback to prevent session deadlocks - Fix integration tools test hanging with Promise.race timeout pattern - Map ModelManagerError Code=1041 to InvalidGenerationSchemaError - Add example runner script, cross-provider example, and OpenAI comparison - Add typecheck and example npm scripts; fix test:coverage to run all projects - 300 tests, 100% coverage
…st reliability - GuardrailViolationError now returns finish_reason: "content_filter" instead of throwing, matching OpenAI's behavior for content policy violations - Widened compat types to match OpenAI SDK: content part types (input_audio, file, refusal), assistant/tool message content arrays, content_filter in finish_reason enums - extractText now warns on all non-text content parts generically - Added 7 missing unsupported params (verbosity, web_search_options, prompt_cache_key, prompt_cache_retention, safety_identifier, function_call, functions) - Tools integration test: 1-in-5 retry with debug logging and transcript dump
- Rewrite getting-started with intro paragraph, Key Concepts using Apple's terminology (Apple Intelligence, Foundation Models, SystemLanguageModel) - Expand all error-handling descriptions with causes and recovery steps - Add Apple developer doc links across guide pages (model-configuration, sessions, streaming, structured-output, tools, transcripts, generation-options) - Merge JSON Schema page into structured-output - Add interface comments for OpenAI class, Completions.create(), Stream, ChatCompletionCreateParams, UNSUPPORTED_PARAMS, ServiceCrashedError - Fix duplicate JSDoc on afmSchemaFormat, fix ServiceCrashedError message - Update terminology: "Apple" → "Foundation Models" / "AFM" in source comments - Style: transparent inline code backgrounds, heading code inherits font size
Add TranscriptEntry types and entries() accessor to Transcript class, enabling direct inspection of transcript history without manual JSON parsing. Export all new types from the public API. Update guide pages with improved terminology, guardrails docs, and tool best practices.
Implement OpenAI Responses API (client.responses.create()) alongside existing Chat Completions API with text, streaming, structured output, tool calling, and error mapping. Add side-by-side integration tests comparing both APIs, comprehensive unit tests (100% coverage on responses.ts), full documentation (guide, API reference, examples), and fix lint issues (no-this-alias, varsIgnorePattern, MD024 headings).
… improvements - Rename MODEL_APPLE_INTELLIGENCE to MODEL_DEFAULT with value "SystemLanguageModel" - Add Responses API examples for both real OpenAI and tsfm (basic + advanced) - Add content clues to doc sections identifying which API is discussed - Fix MD024 duplicate headings across API and guide docs - Update all tests and examples for new model identifier
…d Swift callouts - Shift brand from green to teal (light) and mint to #29d297 (dark) for readability - Darken text-3 to #707074 for WCAG AA contrast - Add Apple-style font rendering, sizing, and letter-spacing - Code blocks word-wrap, inline code uses inherited color with subtle bg - Subtler table alternating rows - Landing page: hero + code example + OpenAI compat box replacing feature grid - Extract Swift-equivalent references into caption-style info boxes across all guide pages - Rewrite structured-output "Picking a Schema Format" section - OpenAI compat: add Responses API, extract shared utils, tool param validation, stream error propagation, Symbol.dispose, integration tests
There was a problem hiding this comment.
Pull request overview
This PR expands the SDK with an OpenAI-compatible API surface (tsfm-sdk/openai), improves schema/transcript ergonomics, and updates test/CI configuration to support the new functionality.
Changes:
- Added an OpenAI compatibility layer (Chat Completions + Responses APIs), including streaming wrappers, tool-calling helpers, and OpenAI-style type definitions.
- Improved SDK ergonomics: typed transcript entries accessors,
Symbol.disposesupport, and more robust JSON schema normalization. - Reworked test setup (unit vs integration), added extensive new unit/integration coverage, and updated CI workflow/docs/examples accordingly.
Reviewed changes
Copilot reviewed 75 out of 78 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| vitest.workspace.ts | Removed Vitest workspace file (previously enumerated unit/integration configs). |
| vitest.config.ts | Updated Vitest configuration to define unit/integration projects + coverage settings. |
| tsconfig.json | Added ESNext.Disposable lib support for Symbol.dispose. |
| tests/unit/transcript.test.ts | Updated transcript JSON shape expectations; added entries() tests. |
| tests/unit/tool.test.ts | Added tests for sync throws in tool callback + Symbol.dispose. |
| tests/unit/session.test.ts | Added tests for exit/signal cleanup and transcript guard; removed unused-vars suppression. |
| tests/unit/schema.test.ts | Expanded tests for recursive JSON schema normalization behavior. |
| tests/unit/helpers/mock-bindings.ts | Adjusted mock signature for transcript creation binding. |
| tests/unit/errors.test.ts | Added coverage for ServiceCrashedError mapping and hierarchy. |
| tests/unit/core.test.ts | Added Symbol.dispose coverage for SystemLanguageModel. |
| tests/unit/compat/transcript.test.ts | New: tests for OpenAI-messages → native transcript conversion. |
| tests/unit/compat/tools.test.ts | New: tests for tool instruction/schema builders + tool response parsing. |
| tests/unit/compat/stream.test.ts | New: tests for compat Stream async iteration + cleanup semantics. |
| tests/unit/compat/responses-stream.test.ts | New: tests for compat ResponseStream iteration + cleanup semantics. |
| tests/unit/compat/params.test.ts | New: tests for OpenAI params → GenerationOptions mapping + warnings. |
| tests/integration/vitest.config.ts | Removed dedicated integration Vitest config (migrated to project config). |
| tests/integration/transcript.test.ts | Switched to waitUntilAvailable before running integration suite. |
| tests/integration/tools.test.ts | Strengthened tool integration test with retries/timeouts and forced tool usage. |
| tests/integration/structured-output.test.ts | Switched to waitUntilAvailable before running integration suite. |
| tests/integration/streaming.test.ts | Switched to waitUntilAvailable before running integration suite. |
| tests/integration/responses.test.ts | New: integration tests for compat Responses API and parity with Chat Completions. |
| tests/integration/helpers/retry.ts | New: retry helper for flaky on-device integration tests. |
| tests/integration/compat.test.ts | New: integration coverage for OpenAI compat layer flows (incl. tool calling). |
| tests/integration/basic.test.ts | Switched to waitUntilAvailable before running integration suite. |
| src/transcript.ts | Added typed transcript entry/content/tool-call types + entries() helper; tightened dict typing. |
| src/tool.ts | Ensured tool calls finish on synchronous errors; added Symbol.dispose. |
| src/session.ts | Added exit/signal cleanup tracking; added transcript initialization guard; added Symbol.dispose. |
| src/schema.ts | Added JsonSchema/JsonObject types; made afmSchemaFormat recursively normalize nested schemas. |
| src/options.ts | Tightened serialization typing for generation options/sampling. |
| src/index.ts | Exported new transcript/schema types and new error type. |
| src/errors.ts | Added ServiceCrashedError; improved statusToError mapping for specific crash signatures. |
| src/core.ts | Added Symbol.dispose for SystemLanguageModel. |
| src/compat/utils.ts | New: JSON key reordering helper + compat error utilities. |
| src/compat/types.ts | New: OpenAI-compatible Chat Completions type definitions. |
| src/compat/transcript.ts | New: OpenAI messages → Foundation Models transcript conversion. |
| src/compat/tools.ts | New: tool instructions/schema builder + structured tool-call parsing. |
| src/compat/stream.ts | New: OpenAI-like Stream wrapper with cleanup semantics. |
| src/compat/responses-types.ts | New: OpenAI Responses API type definitions. |
| src/compat/responses-stream.ts | New: Responses streaming wrapper with cleanup semantics. |
| src/compat/params.ts | New: mapping OpenAI params to SDK generation options + warnings. |
| src/compat/index.ts | New: main OpenAI compat client implementation (tsfm-sdk/openai). |
| scripts/run-example.ts | New: helper to run one/all examples via npm run example. |
| README.md | Updated CI badge link to new workflow. |
| package.json | Bumped version to 0.3.0; added ./openai export; added scripts; added dev dep openai. |
| package-lock.json | Updated lock for 0.3.0 + added openai dependency tree. |
| examples/compat/responses-real.ts | New: example using real OpenAI Responses API (requires API key). |
| examples/compat/responses-local.ts | New: example using local compat Responses API. |
| examples/compat/responses-advanced-real.ts | New: advanced real OpenAI Responses API example. |
| examples/compat/responses-advanced-local.ts | New: advanced local compat Responses API example. |
| examples/compat/openai-real.ts | New: side-by-side real OpenAI Chat Completions example. |
| examples/compat/openai-compat.ts | New: side-by-side local compat Chat Completions example. |
| examples/compat/cross-compat.ts | New: cross-provider transcript handoff example (cloud → local → cloud). |
| eslint.config.js | Adjusted unused-vars rule to ignore vars prefixed with _. |
| docs/index.md | Updated docs home layout with OpenAI compat callout and quick links. |
| docs/guide/transcripts.md | Expanded transcript docs; documented entries() and entry fields. |
| docs/guide/tools.md | Expanded tools docs; added best practices + compat tool calling note. |
| docs/guide/structured-output.md | Expanded structured output docs; added JSON schema usage and guidance. |
| docs/guide/streaming.md | Expanded streaming docs; added compat streaming usage + cleanup notes. |
| docs/guide/sessions.md | Expanded sessions docs; added state checking + compat mention. |
| docs/guide/model-configuration.md | Expanded model config docs; clarified availability waiting behavior. |
| docs/guide/json-schema.md | Removed standalone JSON schema guide (folded into structured output docs). |
| docs/guide/getting-started.md | Expanded getting-started, clarified platform constraints and next steps. |
| docs/guide/generation-options.md | Expanded options docs; clarified sampling semantics + Swift equivalents. |
| docs/guide/error-handling.md | Expanded error docs; documented ServiceCrashedError and recovery steps. |
| docs/examples/openai-compat.md | New: example guide for OpenAI compat layer usage. |
| docs/examples/index.md | Added OpenAI compat examples entry. |
| docs/api/openai-compat.md | New: API reference for tsfm-sdk/openai. |
| docs/api/index.md | Linked OpenAI compat API docs and showed import usage. |
| docs/.vitepress/theme/custom.css | Major theme tweaks + home page layout styles + accessibility adjustments. |
| docs/.vitepress/config.ts | Added OpenAI compat links in nav/sidebars; enabled external link icon. |
| .gitignore | Added .env.local to ignore. |
| .github/workflows/test.yml | Removed old Tests workflow. |
| .github/workflows/ci.yml | Added new CI workflow (tests + dylib build verification). |
| .env.local.example | New: template for OpenAI API key for real OpenAI comparison examples. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Drop "OpenAI Compatible" branding in favor of industry-standard terminology. The compat layer is now framed as Chat-style and Responses-style APIs rather than an OpenAI-specific feature. - Export path: tsfm-sdk/openai → tsfm-sdk/chat - Class: OpenAI → Client (import Client from "tsfm-sdk/chat") - Doc files renamed: openai-compatibility → chat-api, openai-compat → chat - Remove cross-compat example and cross-provider doc sections - Update all imports, tests, examples, and JSDoc comments
- Add Array.isArray guard for tuple schemas in afmSchemaFormat - Handle arrays of objects in reorderJson key ordering - Fix misleading JSDoc in buildToolSchema (merged params, not namespaced) - Validate entries() return with Array.isArray - Use process.once for SIGINT/SIGTERM to prevent handler loop - Use WeakRef for session tracking to allow GC of forgotten sessions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Chat-style and Responses-style API interfaces (
tsfm-sdk/chat), a complete documentation site overhaul, and core SDK improvements for the 0.3.0 release.Chat & Responses APIs (
tsfm-sdk/chat)client.chat.completions.create()) — full message history, streaming, structured output (json_schema), and tool callingclient.responses.create()) — string or structured input, function tools, and streaming viaResponseStreamtemperature,max_tokens/max_completion_tokens,top_p,seed→ nativeGenerationOptions; unsupported params warned at runtimeExceededContextWindowSizeError→finish_reason: "length",GuardrailViolationError→finish_reason: "content_filter",RefusalError→message.refusal,RateLimitedError→ HTTP 429StreamandResponseStreamasync iterables withtoReadableStream(),close(),Symbol.dispose, andFinalizationRegistrycleanup$defs/$refschemas to prevent parameter name collisionsCore improvements
SystemLanguageModel— named public class with availability checks and model warmupServiceCrashedError— detects crashedgenerativeexperiencesdservice with recovery instructionsSymbol.disposeonSystemLanguageModel,LanguageModelSession,Tool, andClientfor TC39 Explicit Resource ManagementTranscriptEntry,TranscriptContent,TranscriptToolCall, etc. withtranscript.entries()methodJsonSchemaandJsonObjectexported typesprocess.exit,SIGINT, andSIGTERMafmSchemaFormat()with recursive normalization,$defs/$refsupport, andx-orderfieldscall()now invokeFMBridgedToolFinishCall()with error message to prevent session hangstatusToError(): mapsModelManagerError Code=1041toInvalidGenerationSchemaErrorDocumentation
examples/compat/demonstrating Chat Completions and Responses APIKnown limitations
usageis alwaysnull(native API doesn't expose token counts)choicesalways contains exactly one entry (n > 1not supported)Test plan
npm run test— 399 tests passing (unit + integration)npm run typecheck— cleannpm run lint— clean