Modularization plan, ucr_hpcc institutional profile, and CI-green fix#15
Merged
Conversation
- Extract ASM_STATS process into modules/asm_stats.nf for reuse by both funannotate.nf and earlgrey_mask.nf - Generate asm_stats.tsv automatically from clean genomes if --gen_asm_stats=true (default) - Make SELECT_REPS optional in earlgrey_mask.nf via --skip_select_reps flag - Add params for gen_asm_stats, skip_select_reps, and tables_dir to control these features - When skip_select_reps=true, all genomes are processed for EarlGrey without size filtering Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Create reusable module for annotation workflows: - ANTISMASH_RUN: Secondary metabolite detection - INTERPROSCAN_RUN: Protein domain annotation - SIGNALP_RUN: Signal peptide prediction Document Phase 1-3 modularization plan for progressive extraction of funannotate pipeline components into independent modules. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Reorganize refactoring plan to align with actual workflow: GENOME PREPROCESSING: - Clean: FCS_GX, sourpurge, vecscreen (modules/AAFTF/) - Mask: TANTAN, REPEATMODELER, REPEATMASKER, EARLGREY, NONE (modules/repeatmasking/) - Summary Stats: ASM_STATS (modules/AAFTF/) GENE PREDICTION: - RNA-seq: SRA_QUERY, SRA_FETCH, RNASEQ_PREPARE (modules/rnaseq_fetch/) - Funannotate: TRAIN, PREDICT, UPDATE (modules/funannotate/) ANNOTATION: - Tools: ANTISMASH, SIGNALP, INTERPROSCAN (modules/annotate/) - Final: FUNANNOTATE_ANNOTATE (modules/annotate/) Add modules/README.md with development guidelines, testing procedures, and step-by-step rollout plan for progressive extraction. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Create detailed ASCII diagram showing: - Phase-by-phase module organization - Data flow through all stages - Input/output contracts for each process - Current status and implementation priority - Masking strategy options (TANTAN, REPEATMODELER, REPEATMASKER, EARLGREY, NONE) Provides visual reference for implementation planning and code review. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Create detailed review document covering: - Complete list of deliverables (4 working implementations) - Full modularization plan (4 phases with directory structures) - Implementation priority and rationale - Critical decision points requiring expert review: * RNA-seq fetch extraction strategy * Repeat masking strategy selector design * AAFTF contamination tool extraction * Setup database ordering pattern - Testing and compatibility concerns - Rollout timeline and next steps Ready for architecture review before implementation begins. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Two config-layer changes:
1. Define params.genome_dir = "${launchDir}/input_clean_genomes" in
profile_annotate.config. It was referenced by the ASM_STATS call site
(funannotate.nf:2063) but never defined, so `-profile test -stub-run`
crashed with "Argument of file() function cannot be null" — i.e. CI on
main was red. The value matches GENOME_CLEAN's storeDir target and
earlgrey_mask.nf's identical param. Stub-run gate is now green (exit 0).
2. Rename the `module` provisioning profile to `ucr_hpcc` to mark it as an
institutional (site-specific Lmod) profile, the nf-core/configs model.
Portable runs use singularity/pixi. Updates nextflow.config, the renamed
conf/provision_ucr_hpcc.config header, run_annotate.sh default, and doc refs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFGQVQDdSLEgpSJehG9qx5
…acking Rewrite REFACTORING_PLAN.md as the single source of truth: the meta-map data contract (Principle 0), target main.nf/workflows/subworkflows/modules layout, per-process extraction checklist gated on stub-run, corrected migration order, and the nf-core distance scorecard. Remove IMPLEMENTATION_SUMMARY.md and MODULE_STRUCTURE.txt — status reports that described aspiration as completed work and would rot. Add .github/REFACTOR_ISSUES.md (epic + 12 ordered issues) and the idempotent scripts/create_refactor_issues.sh that creates them via gh. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFGQVQDdSLEgpSJehG9qx5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sets up the DSL2 modularization effort and fixes the broken stub-run/CI gate.
Note this branch also carries the 5 previously unpushed modularization
commits from local
main(4c50fd8…b115dbc) —origin/mainwas behind, sothey ride in here.
Changes
params.genome_dir(CI-breaking). TheASM_STATScall site(
funannotate.nf:2063) referencedparams.genome_dir, which was neverdeclared, so
-profile test -stub-runcrashed at graph build (file()cannotbe null). Declared it as
${launchDir}/input_clean_genomes— theGENOME_CLEANstoreDir target, matching
earlgrey_mask.nf. Stub-run now passes (exit 0).module→ucr_hpccas an institutional(site-specific Lmod) profile, the nf-core/configs model. Portable runs use
singularity/pixi.REFACTORING_PLAN.md— meta-map data contract (Principle 0),target
main.nf/workflows/subworkflows/moduleslayout, per-processextraction checklist gated on stub-run, corrected migration order, nf-core
scorecard. Removes the two rot-prone status docs.
.github/REFACTOR_ISSUES.md(epic + 12 issues) and theidempotent
scripts/create_refactor_issues.sh.Verification
nextflow config . -profile annotate,slurm,ucr_hpcc— parsesnextflow run . -profile test -stub-run— exit 0, 14 processesnextflow run . --help— renders