Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions openai-python/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# OpenAI API key — https://platform.openai.com/api-keys
OPENAI_API_KEY=

# Archestra LLM Proxy URL — copy from your LLM Proxy's "Connect" page in Archestra
# Example: http://localhost:9000/v1/openai
ARCHESTRA_PROXY_URL=http://localhost:9000/v1/openai
54 changes: 54 additions & 0 deletions openai-python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# OpenAI Python SDK + Archestra

A minimal Python chat app showing how to route OpenAI requests through the
[Archestra LLM Proxy](https://archestra.ai) for security guardrails,
observability, and policy enforcement — with zero changes to your existing
OpenAI SDK code beyond the `base_url`.

## How it works

```
Your app → Archestra LLM Proxy → OpenAI API
(security policies,
usage logs, rate limits)
```

The only change from a standard OpenAI integration is setting `base_url` to
your Archestra proxy URL. The SDK, models, and API surface stay identical.

## Prerequisites

- Python 3.9+
- An Archestra instance running locally (`http://localhost:9000`) or in the cloud
- An OpenAI API key

## Setup

```bash
# 1. Copy the env template and fill in your values
cp .env.example .env

# 2. Install dependencies
pip install -r requirements.txt

# 3. Run the interactive chat
python main.py
```

## Configuration

| Variable | Description |
|---|---|
| `OPENAI_API_KEY` | Your OpenAI API key |
| `ARCHESTRA_PROXY_URL` | Archestra proxy URL (from LLM Proxy → Connect in the UI) |

## Using a different model

Change the `model` parameter in `main.py`:

```python
response = client.chat.completions.create(
model="gpt-4o-mini", # or any model available in your Archestra setup
messages=history,
)
```
96 changes: 96 additions & 0 deletions openai-python/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
"""
Archestra LLM Proxy — OpenAI Python SDK example.

Routes OpenAI requests through the Archestra platform for security
guardrails, observability, and policy enforcement.

Usage:
cp .env.example .env # fill in OPENAI_API_KEY and ARCHESTRA_PROXY_URL
pip install -r requirements.txt
python main.py # streaming (default)
python main.py --no-stream # non-streaming
"""

import os
import sys
from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

api_key = os.environ["OPENAI_API_KEY"]
proxy_url = os.environ.get("ARCHESTRA_PROXY_URL", "http://localhost:9000/v1/openai")

# Point the OpenAI client at the Archestra proxy instead of api.openai.com.
# All other SDK usage stays identical — Archestra is transparent to the caller.
client = OpenAI(
api_key=api_key,
base_url=proxy_url,
)


def chat_stream(user_message: str, history: list[dict]) -> str:
"""Send a message and stream the assistant reply token by token."""
history.append({"role": "user", "content": user_message})

full_reply = ""
with client.chat.completions.create(
model="gpt-4o",
messages=history,
stream=True,
) as stream:
for chunk in stream:
delta = chunk.choices[0].delta.content or ""
print(delta, end="", flush=True)
full_reply += delta
print() # newline after streaming finishes

history.append({"role": "assistant", "content": full_reply})
return full_reply


def chat(user_message: str, history: list[dict]) -> str:
"""Send a message and return the complete assistant reply."""
history.append({"role": "user", "content": user_message})

response = client.chat.completions.create(
model="gpt-4o",
messages=history,
)

reply = response.choices[0].message.content
history.append({"role": "assistant", "content": reply})
return reply


def main() -> None:
use_stream = "--no-stream" not in sys.argv

mode = "streaming" if use_stream else "non-streaming"
print(f"Archestra + OpenAI Python chat [{mode}] (type 'quit' to exit)\n")
history: list[dict] = []

while True:
try:
user_input = input("You: ").strip()
except (EOFError, KeyboardInterrupt):
print("\nGoodbye!")
break

if user_input.lower() in {"quit", "exit", "q"}:
print("Goodbye!")
break

if not user_input:
continue

if use_stream:
print("Assistant: ", end="")
chat_stream(user_input, history)
else:
reply = chat(user_input, history)
print(f"Assistant: {reply}\n")


if __name__ == "__main__":
main()
2 changes: 2 additions & 0 deletions openai-python/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
openai>=1.0.0
python-dotenv>=1.0.0