ai-client

Client API for accessing various AI services

This is highly experimental while I figure out a nice API.

It also needs a catchy name.

Features

• Gemini API: Support for Google's Gemini API

  • Text generation
  • Token counting
  • Model listing
  • Streaming responses (with stream feature)

• OpenAI API: Support for OpenAI's API

  • Full pinned Responses resource operations and typed protocol coverage
  • Conversations state and item operations used by Responses
  • Typed embeddings with model-bound output dimensions
  • Model listing
  • Streaming responses (with stream feature)
  • Deprecated legacy chat completions (with chat-completions feature)

• OpenAI-compatible APIs: A separate typed Chat Completions family

  • Explicit custom endpoint and authentication configuration
  • Provider/dialect and model capability bounds
  • Streaming responses (with both openai-compatible and stream)

OpenAI Responses

OpenAI Responses is the primary API for new OpenAI integrations. Successful calls return AiResponse<T>, which keeps the provider body and HTTP metadata together:

use ai_client::openai::OpenAIClient;
use ai_client::openai::responses::{CreateResponseRequest, Gpt5Mini};

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let client = OpenAIClient::builder()
    .api_key(std::env::var("OPENAI_API_KEY")?)
    .build()?;
let request = CreateResponseRequest::builder()
    .input_text("Explain typed builders briefly.")
    .build();

let response = client
    .responses()
    .create(Gpt5Mini::config(), request)
    .await?;
println!("response id: {}", response.data().id);
println!("request id: {:?}", response.metadata().request_id);

let response_body = response.into_inner();
println!("status: {:?}", response_body.status);
# Ok(())
# }

Model-independent request content and model-specific configuration are separate. Known-model configs expose only settings supported by their marker and track mutually exclusive reasoning/sampling modes in the config type. The endpoint checks that the selected model supports every capability used by the request, including item input, structured output, cache keys, tool controls, and each tool type.

This keeps runtime model routing small without moving failures to production:

let request = CreateResponseRequest::builder()
    .instructions(instructions)
    .input_items(input)
    .json_schema(schema)
    .build();

let model: Box<dyn ResponseModelFor<_>> = match selected_model {
    AppModel::Fast => Box::new(Gpt5_4Nano::config().reasoning_none()),
    AppModel::Strong => Box::new(
        Gpt5_2::config().reasoning(ExtendedReasoningEffort::High),
    ),
};

let response = client.responses().create(model, request).await?;

The request is constructed once. ResponseModelFor<Request> is an object-safe erasure boundary, so a Box or Arc can hold configs with different model and reasoning/sampling typestates. Each coercion is available only if that model supports the request's schema, input, cache, tools, and tool controls. This also makes conditional configuration straightforward: select one of several valid typed configs into the trait object instead of mutating a builder across typestate changes. Static callers can pass a concrete config directly with no allocation, and reusable callers can pass &config or Arc::clone(&config).

There is deliberately no string-model native Responses builder. Custom and fine-tuned models can define an OpenAIResponsesModel marker, opt into capabilities, and start with ResponseModelConfig::<MyModel>::new(). See specs/migration-0.6.md for a focused migration guide.

OpenAI Embeddings

Embedding requests bind dimensions to the selected model marker and return vectors together with provider usage and HTTP metadata:

use ai_client::openai::{
    OpenAIClient,
    embeddings::{
        CreateEmbeddingRequest, EmbeddingDimensions, EmbeddingInput,
        TextEmbedding3Large,
    },
};

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let client = OpenAIClient::builder()
    .api_key(std::env::var("OPENAI_API_KEY")?)
    .build()?;
let request = CreateEmbeddingRequest::<TextEmbedding3Large>::new(
    EmbeddingInput::batch(["first document", "second document"])?
)
.dimensions(EmbeddingDimensions::new(1_024)?);
let response = client.embeddings().create(request).await?;
for embedding in &response.data().data {
    println!("{}: {} dimensions", embedding.index, embedding.embedding.len());
}
# Ok(())
# }

Stored responses use validated opaque IDs and encoded path segments:

use ai_client::openai::OpenAIClient;
use ai_client::openai::responses::ResponseId;

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let client = OpenAIClient::builder()
    .api_key(std::env::var("OPENAI_API_KEY")?)
    .build()?;
let id = ResponseId::new("resp_123")?;
let response = client.responses().retrieve(&id).await?;
println!("status: {:?}", response.data().status);
# Ok(())
# }

Native OpenAI scope

Resource	Status
Responses	7/7 pinned operations, including distinct create/retrieve streaming methods
Conversations	8/8 pinned operations for conversation and nested item state
Embeddings	Typed create operation with model-bound dimensions and float vectors
Standalone Images	Next planned native API resource
Files, Audio, Realtime, Batches, Videos, administration/control-plane	Deferred and out of the active product scope

Responses protocol types still represent documented file/audio/tool content and stream events where the Responses API itself requires them. That does not imply standalone support for those other API resources.

Streaming Support

To enable streaming support, add the stream feature to your Cargo.toml:

[dependencies]
ai_client = { version = "0.6", features = ["stream"] }

Streaming is available via:

• GeminiClient::generate_content_streamed() for Gemini
• OpenAIClient::responses().create_stream(model, request) and retrieve_stream() for OpenAI Responses
• OpenAIClient::generate_response_streamed() as a migration forwarding method
• OpenAIClient::generate_content_streamed() for legacy OpenAI chat completions when both stream and chat-completions are enabled

Streaming methods return AiResponse<AiStream<_>>. The outer response exposes request and rate-limit metadata from the successful HTTP handshake; its inner AiStream yields crate-owned AiStreamError values. OpenAI SSE items are SseJsonEvent<T>, preserving event metadata and the complete raw JSON value alongside typed provider data. Read response.metadata() before calling response.into_inner() to obtain and poll the stream.

Legacy OpenAI Chat Completions

OpenAI recommends the Responses API for new work, so chat completions are disabled by default and deprecated since 0.4.0. Enable them only while migrating a downstream app that intentionally needs the native legacy API:

[dependencies]
ai_client = { version = "0.6", features = ["chat-completions"] }

For OpenAI-shaped third-party endpoints, use the separate openai-compatible feature instead. CustomDialect does not claim that an endpoint implements any particular option: callers provide model markers and capability implementations for the contract they have verified.

OpenAI-compatible Chat Completions

The compatibility family preserves a Chat-Completions-shaped protocol without making it part of native OpenAI. Base URL and authentication are always explicit, and the prepared request remains bound to its dialect:

use ai_client::openai_compatible::{
    chat::{ChatMessage, ChatRole, DynamicChatModel, DynamicChatRequest},
    CompatibleAuth, CustomDialect, OpenAICompatibleClient,
};

let client = OpenAICompatibleClient::<CustomDialect>::builder()
    .base_url("http://localhost:8080/v1")
    .auth(CompatibleAuth::bearer(std::env::var("COMPATIBLE_API_KEY")?))
    .build()?;
let model = DynamicChatModel::new("my-runtime-model")?;
let request = DynamicChatRequest::<CustomDialect>::builder(model)
    .messages(vec![ChatMessage::new(ChatRole::User, "Hello")])
    .build()?;
let response = client.chat().create(request).await?;
println!("{}", response.data().id());

The dynamic builder validates only structural safety and intentionally makes no model-capability guarantees. For compile-time checking, define a CompatibleChatModel<CustomDialect> marker and implement only the relevant capability traits. extra_body is an explicit forward-compatibility escape hatch; collisions with typed or dialect option fields are rejected. Simple messages use ChatMessage::new; multimodal, tool, and endpoint-specific messages use the explicit object-preserving ChatMessage::from_object path. Downstream dialects can instead define their own associated message type.

Product direction

Provider-native APIs remain full fidelity. OpenAI work is intentionally focused on Responses and standalone Images; OpenAI-compatible Chat Completions remains a separate typed dialect family rather than a lowest-common-denominator provider interface.

Minimum Supported Rust Version

The MSRV for this crate is likely to be close to the latest at least for now.

Live provider tests

The normal test suite never contacts an AI provider. Real-provider coverage has two deliberate gates: compile it with the default-off live-tests feature, then select ignored tests with --ignored. This keeps routine cargo test and cargo test --all-features runs token-free.

# OpenAI's cheap core operations and tiny prompts
cargo test --all-features --test live_openai live_openai_core \
  -- --ignored --test-threads=1

# Gemini's cheap core operations and tiny prompts
cargo test --all-features --test live_gemini live_gemini_core \
  -- --ignored --test-threads=1

The model matrices, entitlement-dependent options, provisioned resources, and image/hosted-tool tests are separate filters so their cost and prerequisites are explicit. See tests/LIVE_PROVIDERS.md for the environment variables, coverage map, and exact commands. An explicitly selected live test fails with a clear error when its credential or resource environment is missing; it does not silently pass without testing anything.

LICENSE

This work is dual-licensed under MIT or Apache 2.0. You can choose either license if you use this work.

SPDX-License-Identifier: MIT OR Apache-2.0