[FEATURE] LLM Fine-tuning Pipeline — Custom model fine-tuning for domain-specific tasks

[gelluisaac]

Description

Build infrastructure for fine-tuning LLMs on AstroML-specific data to improve
performance on domain tasks like fraud explanation, transaction classification, and SQL generation.

Scope

Create fine-tuning pipeline with data preparation, training orchestration, and model deployment.

Files to Touch/Create

• astroml/llm/fine_tuning/__init__.py
• astroml/llm/fine_tuning/pipeline.py — Fine-tuning orchestration
• astroml/llm/fine_tuning/dataset.py — Dataset preparation and formatting
• astroml/llm/fine_tuning/trainer.py — Training wrapper (OpenAI, Anthropic, LoRA)
• astroml/llm/fine_tuning/evaluator.py — Post-tuning evaluation
• astroml/llm/fine_tuning/registry.py — Fine-tuned model registry
• astroml/data/fine_tuning/ — Training data storage
• configs/llm/fine_tuning/ — Fine-tuning configs

Fine-tuning Targets

1. Fraud Explanation Model: Better at explaining Stellar fraud patterns
2. SQL Generation Model: Improved NL-to-SQL for AstroML schema
3. Transaction Classification: Categorize transaction types accurately
4. Support Chatbot: Domain-specific customer support

Implementation Details

• Support OpenAI fine-tuning API
• Support LoRA/QLoRA for open-source models (Llama, Mistral)
• Dataset versioning and lineage tracking
• Automated evaluation against holdout sets
• A/B testing: fine-tuned vs base model
• Rollback capability

Acceptance Criteria

• Fine-tuned models show >15% improvement on domain tasks
• Training pipeline runs unattended
• Model evaluation compares to baseline automatically
• Fine-tuned models deployable via model registry
• Dataset preparation is reusable across tasks
• Cost tracking per fine-tuning run

Dataset Requirements

• Minimum 500 examples per task
• 80/10/10 train/validation/test split
• Data quality validation (no duplicates, correct format)
• Synthetic data generation support

Labels

enhancement, llm, fine-tuning, ml