Skip to content

feat: add tzafon integration#4110

Open
eddieogola wants to merge 29 commits intocrewAIInc:mainfrom
eddieogola:feature/tzafon-integration
Open

feat: add tzafon integration#4110
eddieogola wants to merge 29 commits intocrewAIInc:mainfrom
eddieogola:feature/tzafon-integration

Conversation

@eddieogola
Copy link
Copy Markdown

@eddieogola eddieogola commented Dec 17, 2025

Affiliation: Tzafon

This pull request introduces the new TzafonLoadTool, a tool for programmatic web scraping and browser automation using the Tzafon platform. It adds support for Tzafon in the codebase, updates documentation in English, Korean, and Portuguese, and ensures the tool and its dependencies are properly integrated and documented for end users.

Key changes include:

Tool Implementation and Integration

  • Added the TzafonLoadTool implementation to provide programmatic control and data extraction from web pages via the Tzafon browser API, supporting both synchronous and asynchronous usage.
  • Registered TzafonLoadTool in the crewai_tools package for import and use, and included it in the __all__ exports. [1] [2]
  • Added Tzafon and Playwright as optional dependencies in the pyproject.toml under the [project.optional-dependencies] section.
  • Added a test environment variable for the Tzafon API key in .env.test.

Documentation Updates

  • Added English documentation for TzafonLoadTool, including installation, usage, arguments, and example code. [1] [2]
  • Added Korean and Portuguese documentation for TzafonLoadTool, with translated descriptions, installation instructions, and usage examples. [1] [2]
  • Updated tool listings and overview pages in English, Korean, and Portuguese to include TzafonLoadTool and its description. [1] [2] [3] [4] [5] [6] [7]

These changes collectively enable users to leverage Tzafon for advanced web scraping and browser automation tasks, with comprehensive multilingual documentation and seamless integration into the existing tool suite.


Note

Medium Risk
Adds a new native LLM provider and a Playwright-based web loading tool that invoke external services, which can affect runtime behavior, dependency resolution, and error handling paths. Risk is mitigated by added model constants and a fairly comprehensive mocked test suite for routing, streaming, and tool-calling.

Overview
Adds native Tzafon LLM support by introducing TzafonCompletion, registering tzafon as a supported native provider in LLM routing, and adding TZAFON_MODELS plus context-window entries for tzafon.sm-1 and tzafon.northstar-cua-fast.

Introduces a new web scraping/automation tool TzafonLoadTool (sync + async) that uses Tzafon’s Computer + Playwright CDP connection to load a URL and return either text or full HTML; it’s exported via crewai_tools, added to tool.specs.json, and wired with a TZAFON_API_KEY env var.

Updates packaging and docs: adds tzafon optional dependency group (and uv.lock entries), adds TZAFON_API_KEY to .env.test, and documents the provider/tool in EN/KO/pt-BR (including new tzafonloadtool.mdx pages and navigation/listing updates).

Written by Cursor Bugbot for commit cca0861. This will update automatically on new commits. Configure here.

cursor[bot]

This comment was marked as outdated.

@eddieogola eddieogola changed the title Feature/tzafon integration feat: add tzafon integration Jan 30, 2026
await page.close()
await browser.close()
finally:
await computer.terminate()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Async method incorrectly awaits synchronous terminate call

Medium Severity

The _arun method uses await computer.terminate() but the computer object is created via a synchronous call to self.tzafon.create(kind="browser") (line 136, not awaited). Since create() is synchronous, terminate() is also likely synchronous, making the await incorrect. This will cause a TypeError at runtime when using the async execution path. The sync _run method correctly calls computer.terminate() without await on line 121.

Fix in Cursor Fix in Web

content = str(page_html)

page.close()
browser.close()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant explicit page and browser cleanup calls

Low Severity

The explicit page.close() and browser.close() calls are redundant. Playwright's context manager (with sync_playwright() / async with async_playwright()) automatically handles cleanup on exit. Additionally, computer.terminate() in the finally block destroys the entire remote environment, making browser resource cleanup unnecessary. No other tool in the codebase includes these explicit cleanup calls. This was noted in the PR discussion as well.

Additional Locations (1)

Fix in Cursor Fix in Web

eddieogola and others added 3 commits February 11, 2026 09:28
Add native Tzafon provider with OpenAI-compatible chat completions,
streaming, tool calling, and token usage tracking. Includes docs
(EN, KO, PT-BR), provider constants, and unit tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

"completion_tokens": getattr(usage, "completion_tokens", 0),
"total_tokens": getattr(usage, "total_tokens", 0),
}
return {"total_tokens": 0}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TzafonCompletion is large near-copy of OpenAI provider

Medium Severity

TzafonCompletion is a ~730-line class that duplicates nearly all logic from the existing OpenAICompletion provider (completion handling, streaming, tool calling, token tracking, error handling) with only the API key env var name and base URL differing. Since Tzafon's API is explicitly OpenAI-compatible, this could be achieved by configuring the existing OpenAICompletion with a different base_url and api_key source, avoiding a large maintenance burden and risk of the copy diverging over time.

Fix in Cursor Fix in Web

@eddieogola eddieogola marked this pull request as draft February 28, 2026 09:38
@eddieogola eddieogola marked this pull request as ready for review February 28, 2026 10:30
@eddieogola eddieogola marked this pull request as draft March 11, 2026 13:04
@eddieogola eddieogola marked this pull request as ready for review March 11, 2026 13:05
@eddieogola eddieogola marked this pull request as draft March 31, 2026 18:19
eddieogola and others added 2 commits March 31, 2026 21:42
- Add Arabic (ar) documentation translations
- Add skills system with loader, parser, and validation
- Add agent planning configuration and step executor
- Add flow serializer and observation events
- Update MCP tools, memory storage, and LLM providers
- Add OpenAI-compatible provider support
- Update crewai-tools and crewai-files libraries
- Add comprehensive test coverage for new features

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@eddieogola eddieogola marked this pull request as ready for review March 31, 2026 18:51
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0181542292

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"hosted_vllm",
"cerebras",
"dashscope",
"tzafon",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Route tzafon through its native completion class

Adding tzafon to openai_compatible_providers makes _get_native_provider return OpenAICompatibleCompletion for this provider, but that class has no tzafon entry in OPENAI_COMPATIBLE_PROVIDERS, so LLM("tzafon/tzafon.sm-1") (or provider="tzafon") fails with an unknown-provider error instead of using the new TzafonCompletion. This leaves the advertised Tzafon integration unreachable at runtime.

Useful? React with 👍 / 👎.

await page.close()
await browser.close()
finally:
await computer.terminate()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not await synchronous termination in async tool path

The async implementation calls await computer.terminate(), but the same SDK object is terminated synchronously in _run and the Tzafon Computer API uses a non-async terminate() method. Awaiting it will raise a TypeError during cleanup, causing _arun calls to fail even after successful navigation/extraction.

Useful? React with 👍 / 👎.

@eddieogola eddieogola marked this pull request as draft March 31, 2026 18:56
@eddieogola eddieogola marked this pull request as ready for review March 31, 2026 19:09
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c2e54d6abb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +699 to +701
def supports_stop_words(self) -> bool:
"""Check if the model supports stop words."""
return self._supports_stop_words_implementation()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Implement supports_function_calling on TzafonCompletion

TzafonCompletion implements native tool-call handling but never defines supports_function_calling(), unlike the other native providers. Multiple core paths call this method directly (for example Converter.to_pydantic and task evaluation flows), so using a Tzafon LLM there will raise AttributeError before any model request is made. Adding this capability method is necessary to keep the provider compatible with the expected native LLM interface.

Useful? React with 👍 / 👎.

Comment on lines +622 to +624
response: ChatCompletion = await self.async_client.chat.completions.create(
**params
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Route async streaming calls to a streaming handler

When self.stream is enabled, _prepare_params() adds stream=True, but acall() still always routes into _ahandle_completion(), which expects a non-streaming ChatCompletion. In async mode this call returns a stream object, so the later response.choices[0] access fails and breaks acall() for streaming configurations. This path needs a dedicated async streaming handler (as done in other native providers).

Useful? React with 👍 / 👎.

@eddieogola eddieogola marked this pull request as draft March 31, 2026 19:35
@eddieogola eddieogola marked this pull request as ready for review March 31, 2026 19:35
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a52be34f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +558 to +559
if tool_calls and available_functions:
for call_data in tool_calls.values():
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Return streamed tool calls when no functions are provided

This streaming path only handles tool calls under if tool_calls and available_functions, so when tools are enabled but available_functions is intentionally omitted, the method falls through and returns full_response (often empty) instead of the tool-call payload. That breaks the same workflow supported by non-streaming calls (return raw tool calls for external execution) and causes streamed tool invocations to be silently dropped; the async streaming method has the same omission.

Useful? React with 👍 / 👎.

Comment on lines +844 to +846
content = message.content or ""
content = self._apply_stop_words(content)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate async structured output before returning content

In _ahandle_completion, the async non-streaming flow goes straight from message.content to stop-word truncation and return, but it never runs the structured-output validation that the sync _handle_completion path applies for self.response_format model types. This makes acall() return raw text for typed response_format configurations, causing type mismatches and skipped schema enforcement for async users.

Useful? React with 👍 / 👎.

@eddieogola eddieogola marked this pull request as draft March 31, 2026 19:46
@eddieogola eddieogola marked this pull request as ready for review March 31, 2026 19:50
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant