[FEATURE] LLM-powered Test Generation — Automatic test case creation from requirements

[gelluisaac]

Description

Use LLMs to automatically generate test cases, test data, and test assertions
from code, documentation, and requirements.

Scope

Build AI-powered testing assistant that creates comprehensive test suites.

Files to Touch/Create

• astroml/llm/testing/__init__.py
• astroml/llm/testing/generator.py — Test generation orchestrator
• astroml/llm/testing/strategies/ — Generation strategies
• astroml/llm/testing/assertions/ — Assertion generators
• astroml/llm/testing/fixtures/ — Test data generation
• astroml/llm/testing/reviewer.py — Test quality reviewer
• tools/test_generator/ — CLI tool for test generation

Test Generation Capabilities

1. Unit Tests: Generate from function signatures and docstrings
2. Integration Tests: Generate from API specs
3. Property-based Tests: Generate from type specifications
4. Edge Case Discovery: Identify boundary conditions
5. Regression Tests: Generate from bug reports
6. Performance Tests: Generate load test scenarios

Implementation Details

• Analyze code structure and documentation
• Generate pytest-compatible test functions
• Create realistic test fixtures and mocks
• Include edge cases and error conditions
• Generate test documentation
• Suggest test coverage improvements

Acceptance Criteria

• Generated tests pass 80% of the time initially
• Human review reduces fixing time by 50%
• Test coverage suggestions are actionable
• Generated edge cases find real bugs
• Supports pytest, unittest, and FastAPI TestClient
• Test quality improves with feedback

Example Output

def test_fraud_score_with_high_risk_account():
    account = create_test_account(risk_score=0.9)
    result = await fraud_service.score(account.id)
    assert result.score > 0.8
    assert result.alert_created is True

Labels

enhancement, llm, testing, developer-tools