Skip to content

feat(agents): add task quality loop auditor - #76

Draft
irvineoy wants to merge 1 commit into
codex/pr44-curated-tasksfrom
codex/quality-loop-agent
Draft

feat(agents): add task quality loop auditor#76
irvineoy wants to merge 1 commit into
codex/pr44-curated-tasksfrom
codex/quality-loop-agent

Conversation

@irvineoy

@irvineoy irvineoy commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add the repository-level quality_loop agent, defaulting to Codex
  • require GitHub authentication, repository write access, Issues, a clean checkout, and Git author identity before task work starts
  • run validator -> one repair attempt for blocking failures -> exactly one optimization iteration -> centralized evaluation -> independent Codex review
  • record WARN findings without repair and create fingerprinted, idempotent issues for unrepairable tasks
  • harden reproducibly easy (median >=5x) standalone baselines and targeted cases only behind fresh validation and dual original/candidate correctness gates
  • keep GitHub credentials outside the GPU container, mount the main checkout read-only, verify the accepted diff on the host, and bundle task commits into one draft PR
  • add resumable state, MI300/MI355X examples, Docker/Makefile entry points, documentation, and tests

Validation

  • python3 -m unittest discover -s tests -p 'test_quality_loop.py' -v — 15 passed
  • pinned ROCm Docker image: python -m pytest -q tests — 81 passed, 3 subtests passed
  • bash tests/test_docker_benchmark.sh — passed
  • make docker-smoke — passed on MI355X/gfx950
  • real Codex task-validator smoke on three representative tasks:
    • hip2hip/gpumode/GELU — PASS
    • triton2triton/vllm/triton_rms_norm — PASS
    • flydsl2flydsl/fp8_gemm_4wave_kernel — WARN only (reference timing methodology); compilation/correctness/kernel timing passed

Dependency

This draft is based on #75 so its diff contains only the quality-loop implementation. Retarget it to main after #75 merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant