Add Agent Playbook section for automated updates - #29
Conversation
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Pull Request Overview
The PR introduces a comprehensive 'Agent Playbook' to the README.md, successfully covering the required versioning locations, procedures, and failure modes. However, there are critical inconsistencies in the documented build commands that will likely lead to incorrect container builds. Specifically, the build argument used in the examples does not match the argument name or format required by the Dockerfile as specified in the playbook's own reference table.
Additionally, while the playbook provides a useful end-to-end guide, it creates significant redundancy by duplicating existing 'Usage' and 'Test' sections. These should be consolidated to maintain a single source of truth and prevent future documentation rot.
About this PR
- The addition of the 'Agent Playbook' creates multiple sources of truth for core procedures like building, testing, and generating documentation. To ensure long-term maintainability, these should be unified rather than duplicated across the README.
Test suggestions
- Instructions for bumping .tool_version (no leading 'v') are present.
- Instructions for bumping Dockerfile OPENGREP_VERSION (leading 'v') are present.
- Instructions for updating the opengrep-rules commit pin in internal/docgen/parsing.go are present.
- Verification that docgen (go run ./cmd/docgen) and unit tests are included in the procedure.
- Inclusion of the 'Definition of Done' checklist.
- Verification that pipes within Markdown table cells are escaped (e.g., in unit test command).
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| 2. **Update the opengrep-rules commit pin** in `internal/docgen/parsing.go` if the task calls for picking up new/changed upstream rules (find the latest commit on the `release` branch of `opengrep/opengrep-rules`). | ||
| 3. **Regenerate the docs**: `go run ./cmd/docgen`. This produces `docs/patterns.json`, `docs/rules.yaml`, and `docs/description/*` fresh — since these are git-ignored, you won't see them in the diff, but the run itself is the validation that the rule set still parses cleanly (it `panic`s on unrecognized severities/categories/languages, so a clean exit is a meaningful signal). | ||
| 4. **Run the Go unit tests**: `go test $(go list ./... | grep -v /docs/)` (this is the exact command CI's `unit_tests` job runs). | ||
| 5. **Build the Docker image**: `docker build --build-arg TOOL_VERSION=$(cat .tool_version) -t codacy-opengrep:latest .` — this exercises the full chain: Go build, `go run ./cmd/docgen` inside the builder stage, and the architecture-specific Opengrep binary download, so a failure here can also reveal a bad version/commit pin. |
There was a problem hiding this comment.
🔴 HIGH RISK
Critical inconsistency in build commands: The build command uses the argument TOOL_VERSION, but the versioning table (line 59) and failure mode documentation (line 81/87) specify that the Dockerfile requires OPENGREP_VERSION. Furthermore, the table states the version requires a leading v (e.g., v1.24.0), which $(cat .tool_version) lacks. This will likely cause the image build to fail or use a stale binary.
Try running the following prompt in your coding agent:
Verify if the Dockerfile uses
TOOL_VERSIONorOPENGREP_VERSION. Update the build commands inREADME.md(lines 13, 70, and 91) to use the correct argument name and ensure thevprefix is included if required (e.g.,--build-arg OPENGREP_VERSION=v$(cat .tool_version)).
| We use the [codacy-plugins-test](https://github.com/codacy/codacy-plugins-test) to test our external tools integration. | ||
| You can follow the instructions there to make sure your tool is working as expected. | ||
|
|
||
| ## Agent Playbook: Updating This Repository End-to-End |
There was a problem hiding this comment.
🟡 MEDIUM RISK
Suggestion: This new playbook section largely duplicates instructions found in the existing 'Usage', 'Generate Docs', and 'Test' sections (lines 8-36). To prevent documentation rot, consider consolidating these into the playbook or removing the redundant top-level headings.
Try running the following prompt:
Consolidate the 'Usage', 'Generate Docs', and 'Test' sections into the new 'Agent Playbook' to provide a single, comprehensive guide and remove the redundant headings.
Summary
.tool_version, DockerfileOPENGREP_VERSIONARG, opengrep-rules commit pin ininternal/docgen/parsing.go), the step-by-step update/test/CI-verification procedure, common failure modes, and a definition-of-done checklist.Test plan