Skip to content

Add MiniMax as LLM provider for web agent evaluation#250

Open
octo-patch wants to merge 1 commit into
web-arena-x:mainfrom
octo-patch:feature/add-minimax-provider
Open

Add MiniMax as LLM provider for web agent evaluation#250
octo-patch wants to merge 1 commit into
web-arena-x:mainfrom
octo-patch:feature/add-minimax-provider

Conversation

@octo-patch

Copy link
Copy Markdown

Summary

  • Add MiniMax (M2.7 / M2.5 / M2.5-highspeed) as a third LLM provider alongside OpenAI and HuggingFace
  • Uses the OpenAI-compatible Chat Completions API via direct HTTP requests (avoids interfering with the existing openai==0.27.0 global state)
  • Includes temperature clamping, think tag stripping, async batch support, and retry with exponential backoff
  • 28 unit tests + 3 integration tests covering config, dispatch, tokenizer, prompt construction, and API calls

Changes

File Description
llms/providers/minimax_utils.py New MiniMax provider module (sync + async generation)
llms/lm_config.py Add minimax branch in construct_llm_config()
llms/utils.py Add minimax dispatch in call_llm()
llms/tokenizers.py Add minimax tokenizer (cl100k_base encoding)
llms/init.py Export generate_from_minimax_chat_completion
agent/prompts/prompt_constructor.py Support minimax in prompt formatting (same chat format as OpenAI)
README.md Add MiniMax usage examples and supported providers table
tests/test_minimax_provider.py 28 unit tests
tests/test_minimax_integration.py 3 integration tests (require MINIMAX_API_KEY)

Usage

export MINIMAX_API_KEY=your_key
python run.py --provider minimax --model MiniMax-M2.7 --mode chat --temperature 0.8 --result_dir results_minimax

Test plan

  • All 28 unit tests pass (pytest tests/test_minimax_provider.py)
  • All 3 integration tests pass against real MiniMax API
  • Existing OpenAI/HuggingFace providers unaffected (no changes to their code paths)
  • End-to-end evaluation with MiniMax on WebArena tasks

Add MiniMax (MiniMax-M2.7, MiniMax-M2.5, MiniMax-M2.5-highspeed) as a
third LLM provider alongside OpenAI and HuggingFace. Uses the
OpenAI-compatible Chat Completions API via direct HTTP requests to avoid
interfering with the existing openai==0.27.0 global state.

Key changes:
- New provider module: llms/providers/minimax_utils.py with sync/async
  generation, temperature clamping, think-tag stripping, and retry logic
- Updated lm_config.py, utils.py, tokenizers.py, __init__.py for
  minimax provider dispatch and configuration
- Updated prompt_constructor.py to format prompts for minimax (same
  chat format as OpenAI)
- Updated README.md with MiniMax usage examples and supported providers
  table
- 28 unit tests + 3 integration tests

Usage:
  export MINIMAX_API_KEY=your_key
  python run.py --provider minimax --model MiniMax-M2.7 --mode chat
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant