feat(#6): extract GENOME_CLEAN + GENOME_CLEAN_BATCH → CLEAN_GENOMES subworkflow#20
Merged
Merged
Conversation
…+ CLEAN_GENOMES subworkflow
Two processes moved out of the funannotate.nf monolith:
modules/local/genome_clean.nf — per-genome FCS-GX purge + length-filter
modules/local/genome_clean_batch.nf — batched variant (amortizes /dev/shm staging)
Composed into subworkflows/local/clean_genomes.nf (CLEAN_GENOMES), which
encapsulates the batching decision (clean_batch_size / skip_fcs), the
jobs_to_clean pre-filter, the sentinel-gate clean_done_ch, and the
clean_genome_ch reconstruction. funannotate.nf reduces to:
CLEAN_GENOMES(jobs, taxondb_ch)
def clean_genome_ch = CLEAN_GENOMES.out.genomes
Dual-label pattern: 'genome_clean' (provisioning) + 'process_high'/'process_long'
(tier defaults); inline cpus/memory/time removed (overridden by withName in
conf/profile_annotate.config).
withName patterns in profile_annotate.config updated from 'GENOME_CLEAN' to
'.*:GENOME_CLEAN' (regex) to match the subworkflow-namespaced process names.
genomeFile() is a top-level def in funannotate.nf, not accessible from included
subworkflow files. clean_genomes.nf inlines the equivalent two-liner logic
(prefer .fa.gz if non-empty, else .fa) rather than refactoring to lib/ now.
Stub-run gate: 15 completed, 0 failed
(CLEAN_GENOMES:GENOME_CLEAN_BATCH correctly namespaced; SETUP_AUGUSTUS_CONFIG skips)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa
…E subworkflow
Process moved to modules/local/maskrepeat_tantan_run.nf; labels updated to dual
pattern: 'funannotate' (provisioning) + 'process_medium' (tier default, overridden
by withName '.*:MASKREPEAT_TANTAN_RUN' in profile_annotate.config).
MASK_GENOME subworkflow (subworkflows/local/mask_genome.nf) encapsulates the
run_repeatmasker if/else and the fallback-to-prior-masked-or-clean logic.
funannotate.nf reduces to:
MASK_GENOME(clean_genome_ch)
def predict_genome_ch = MASK_GENOME.out.genomes
genomeFile() unavailability in subworkflows: fallback path inlines equivalent
logic (prefer .masked.fasta.gz if non-empty, else .masked.fasta).
Stub-run gate: 15 completed, 0 failed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa
…RNASEQ subworkflow
Six processes moved out of funannotate.nf monolith:
modules/local/sra_query.nf — single-species SRA query (storeDir)
modules/local/sra_query_batch.nf — batched query (publishDir, maxForks 4)
modules/local/collect_sra_query.nf — merge per-species CSVs into manifest
modules/local/write_empty_reads.nf — zero-byte read placeholders for no-data species
modules/local/sra_fetch.nf — PE download + normalize (EBI FTP fallback)
modules/local/sra_fetch_se.nf — SE download + normalize
Composed into subworkflows/local/fetch_rnaseq.nf (FETCH_RNASEQ):
take: ch_species (species_tag, taxonid)
- Step 1: SRA query (skip_sra_query reuses cached CSVs; else SRA_QUERY_BATCH)
- Step 2: COLLECT_SRA_QUERY — always runs
- Step 3: three-way branch (SRA_FETCH / SRA_FETCH_SE / WRITE_EMPTY_READS)
skipped when stop_after_sra_query=true
emit: reads (species_tag, r1, r2, se), sra_manifest
funannotate.nf: ~110 lines of query/routing orchestration replaced by:
FETCH_RNASEQ(sra_input)
reads_ch = FETCH_RNASEQ.out.reads
withName patterns updated to '.*:SRA_QUERY_BATCH' etc. in profile_annotate.config.
SRA_QUERY (standalone, unused in current workflow) retains bare name pattern.
Stub-run gate: 15 completed, 0 failed (run_sra_fetch=false default)
Dual-stub gate: 17 completed, 0 failed (run_sra_fetch=true via params-file)
FETCH_RNASEQ:SRA_QUERY_BATCH, COLLECT_SRA_QUERY, SRA_FETCH all correctly namespaced
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa
…EDICT → TRAIN_PREDICT subworkflow Three processes moved out of funannotate.nf monolith: modules/local/rnaseq_prepare.nf — Trinity-GG once per species (storeDir rnaseq_data/) modules/local/funannotate_train.nf — PASA alignment per assembly (training_target/<id>/) modules/local/funannotate_predict.nf — gene prediction (target/<id>/, Option B persistence) Composed into subworkflows/local/train_predict.nf (TRAIN_PREDICT): take: ch_genomes (meta, genome_fa), ch_reads (species_tag, r1, r2, se) - When run_sra_fetch+reads available: RNASEQ_PREPARE (repr only) → FUNANNOTATE_TRAIN (all) - Species with no reads bypass TRAIN entirely; predict still runs - FUNANNOTATE_PREDICT: skipped when gbk exists and not stale (staleRnaseq logic inlined) emit: metadata (val meta), done (*.predict.done marker) funannotate.nf: ~130 lines of orchestration replaced by TRAIN_PREDICT(predict_genome_ch, reads_ch) staleRnaseq/gbkResult inlined in train_predict.nf as top-level function declarations (same pattern as genomeFile in clean_genomes.nf and mask_genome.nf — these top-level `def` functions from funannotate.nf are not accessible from subworkflow files). withName patterns updated: RNASEQ_PREPARE/FUNANNOTATE_TRAIN/FUNANNOTATE_PREDICT → '.*:RNASEQ_PREPARE' etc. in conf/profile_annotate.config. Stub-run gate: 15 completed, 0 failed (run_sra_fetch=false) Dual-stub gate: 17 completed, 0 failed (run_sra_fetch=true) TRAIN_PREDICT:FUNANNOTATE_PREDICT correctly namespaced in both Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa
…TE_GENOME subworkflow
Two processes moved out of funannotate.nf monolith:
modules/local/funannotate_annotate.nf — funannotate annotate (functional annotation)
modules/local/funannotate_update.nf — funannotate update (PASA re-alignment post-predict)
Composed with existing tool modules into subworkflows/local/annotate_genome.nf (ANNOTATE_GENOME):
take: ch_predict_meta (val meta), ch_reads (species_tag, r1, r2, se)
- Optional ANTISMASH_RUN → INTERPROSCAN_RUN → SIGNALP_RUN gating chain (unchanged logic)
- Optional FUNANNOTATE_UPDATE (joins reads_ch; gated by gbk existence; update→annotate)
- Optional FUNANNOTATE_ANNOTATE (gated behind all requested pre-annotate steps)
emit: versions (ch_versions from ANTISMASH/INTERPRO/SIGNALP)
funannotate.nf: ~100 lines of post-predict orchestration replaced by:
ANNOTATE_GENOME(predict_meta, reads_ch)
ch_versions = ch_versions.mix(ANNOTATE_GENOME.out.versions)
withName patterns updated: ANTISMASH_RUN, INTERPROSCAN_RUN, SIGNALP_RUN,
FUNANNOTATE_ANNOTATE, FUNANNOTATE_UPDATE → '.*:NAME' in conf/profile_annotate.config.
_gbkResult inlined as top-level function in annotate_genome.nf (same pattern as other subworkflows).
funannotate.nf now contains: only includes, 3 utility defs (gbkResult/genomeFile/staleRnaseq),
and the workflow{} block. No inline process definitions remain.
Stub-run gate: 15 completed, 0 failed
Dual-stub gate: 17 completed, 0 failed
ANNOTATE_GENOME:ANTISMASH_RUN, INTERPROSCAN_RUN, SIGNALP_RUN, FUNANNOTATE_ANNOTATE
all correctly namespaced
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Note: `genomeFile()` is a top-level def in `funannotate.nf`, inaccessible from included subworkflows. The subworkflow inlines the equivalent logic. Moving `genomeFile()` to `lib/` is tracked separately.
Test plan
🤖 Generated with Claude Code