feat(checkup lint): implement pdd checkup lint prompt linter#1154
Draft
DianaTao wants to merge 8 commits into
Draft
feat(checkup lint): implement pdd checkup lint prompt linter#1154DianaTao wants to merge 8 commits into
DianaTao wants to merge 8 commits into
Conversation
…ve package and mock
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #829
Overview
This PR implements
pdd checkup lint, a comprehensive, prompt-native static analysis engine that enforces strict prompt engineering guidelines and checks prompts and user stories for ambiguity before executing downstream agentic actions.The linter is designed with two distinct, complementary phases:
Technical Details & Architecture
1. Deterministic Linter Engine (
pdd/prompt_lint.py)<contract_rules>,<requirements>,<acceptance_tests>) and comment blocks (%prose lines).valid,successful,authorized, etc.) within rules.<vocabulary>,## Glossary, or## Definitionsblock in the prompt/story.--strict): Elevates all warnings to errors (exit code2) and expands the vocabulary check to strict english terms (proper,correct,normal).returns,raises,writes,emits) to ensure formal testability.2. PDD Cloud LLM Ambiguity Review (
pdd/prompts/prompt_lint_LLM.prompt)pdd checkup lint --ambiguity.prompt_lint_LLM.promptto prompt the LLM for potential semantic misinterpretations.3. Subprocess venv Parity & Unit Test Fixes
To prevent execution discrepancy inside local environments and CI pipelines:
tests/test_operation_logging_e2e.pyto usesys.executable -m pddinstead of"pdd". This prevents path priority issues where a stale global installation ofpddtakes precedence over the active workspace wheel in the virtual environment.unfinished_promptmethod intest_generate_test.py'stest_generate_test_uses_example_templateto ensure unit tests execute completely offline and complete under<0.2swithout trying to connect to Google Vertex AI.Files Added/Modified
pdd/prompt_lint.py— Linter and glossary parsing core engine.pdd/commands/prompt.py— Prompt CLI subcommand forlint.pdd/prompts/prompt_lint_LLM.prompt— LLM prompt instructions for ambiguity review.tests/commands/test_checkup_prompt_lint.py— CLI integration unit tests.pdd/commands/checkup.py— Integratedpdd checkup lintsubcommand routing.tests/test_operation_logging_e2e.py— Subprocess command execution venv parity.tests/test_generate_test.py— Offline unit test execution fixes.Verification & Staging Results
A. Deterministic Lint Unit Tests (
pytest tests/commands/test_checkup_prompt_lint.py)tests/commands/test_checkup_prompt_lint.py::test_checkup_lint_clean_prompt_json PASSED [ 33%] tests/commands/test_checkup_prompt_lint.py::test_checkup_lint_reports_warning PASSED [ 66%] tests/commands/test_checkup_prompt_lint.py::test_checkup_lint_strict_promotes_warning_to_error PASSED [100%] ========================= 3 passed in 0.19s =========================B. Prompt Generation Offline Unit Tests (
pytest tests/test_generate_test.py)tests/test_generate_test.py::test_generate_test_uses_example_template PASSED [ 80%] ... ======================== 21 passed in 0.18s ========================