FastAPI Prompt Evaluation API

This FastAPI application evaluates a user-supplied prompt across multiple AI models Gemini(gemini-2.0-flash-001), ChatGPT(gpt-4o-mini), Llama(llama-3.3-70b-instruct) and returns the best response based on evaluation scores.

Prompt Format

The JSON prompt can have the following fields:

• messages: An array of message objects.
• tools: An array of tool objects.
• response_format: The desired response format.
• temperature: The temperature setting for the model.

API Keys

To run this application, you will need the following API keys:

• NVIDIA_NIM_API_KEY: For the Llama model.
• OPENAI_API_KEY: For the ChatGPT model.
• GEMINI_API_KEY: For the Gemini model.

Response Format

The response from the API will be in the following format:

{
  "outputs": [
    {
      "model_name": "string",
      "output": {}
    }
  ],
  "best_model": "string"
}