Skip to content

feat(#1-3): meta-map contract + INPUT_CHECK + repo skeleton - #17

Merged
hyphaltip merged 1 commit into
mainfrom
refactor/meta-map
Jun 29, 2026
Merged

feat(#1-3): meta-map contract + INPUT_CHECK + repo skeleton#17
hyphaltip merged 1 commit into
mainfrom
refactor/meta-map

Conversation

@hyphaltip

@hyphaltip hyphaltip commented Jun 29, 2026

Copy link
Copy Markdown
Member

Summary

Phases 0–1 of the DSL2 modularization plan (REFACTORING_PLAN.md): replace the fragile positional 10-tuple data contract with the standard meta-map, introduce INPUT_CHECK to
deduplicate samplesheet parsing, and establish the modules/local/ / subworkflows/local/ directory skeleton.

Closes #1 (meta-map contract), #2 (INPUT_CHECK subworkflow), #3 (repo skeleton)


Issue #1 — atomic meta-map conversion (799897a)

Every process was wired with a positional 10-tuple that required touching every call site to add a field. Replaced with the standard DSL2 contract throughout:

Before: tuple(out, asmid, species, strain, locustag, busco, header_length, transl_table, gz, taxonid)
After: tuple val(meta), val/path(genome)
meta = [id, asmid, species, strain, locustag, busco, transl_table, taxonid]

  • All 20 inline processes converted — shim pattern: input:/output: rewritten, alias block (def out = meta.id; def asmid = meta.asmid; …) at the top of each script: so existing
    shell interpolations are untouched.
  • All ~40 workflow channel ops updated: filters, maps, groupTuple, combine, join.
  • params.header_length (default 24) replaces the per-sample tuple field.
  • Key non-obvious fix: clean_genome_ch uses it[0] index access instead of fixed-arity destructuring because .combine(ifEmpty([])) produces a variable-length tuple (2 elements when
    the sentinel is an empty list, 3+ when populated). A fixed { meta, _gz, _sentinel -> } closure fails on the 2-element case.

Issue #2 — INPUT_CHECK shared subworkflow (53deaef)

Extracted the samplesheet ingestion logic (duplicated between funannotate.nf and earlgrey_mask.nf) into subworkflows/local/input_check.nf. Emits two channels:

  • samples — val(meta): taxon / asmid / suppress / n_test filters applied, no genome-existence check (used for postpredict, earlgrey select path)
  • genomes — tuple(val(meta), path(gz)): additionally filtered to samples whose raw genome file exists (the primary jobs channel in funannotate)

earlgrey_mask.nf's skip_select_reps=true path now calls INPUT_CHECK and gains taxon/asmid/suppress/n_test filtering it previously lacked entirely.

Issue #3 — repo skeleton (53deaef)
Directory layout established:
modules/local/ ← one process per file (convention)
subworkflows/local/ ← shared subworkflows
workflows/ ← future workflow entrypoints

  • modules/asm_stats.nf → modules/local/asm_stats.nf (git mv)
  • Three annotation processes extracted from the funannotate.nf monolith into individual module files: antismash_run.nf, interproscan_run.nf, signalp_run.nf
  • Stale modules/annotation_tools.nf deleted (old 10-tuple versions, superseded by the above)

Test plan

  • nextflow run . -profile test -stub-run — 15 completed, 0 failed (full pipeline graph: GENOME_CLEAN_BATCH → MASKREPEAT×2 → FUNANNOTATE_PREDICT×2 →
    ANTISMASH/INTERPROSCAN/SIGNALP×2 → FUNANNOTATE_ANNOTATE×2)
  • nextflow run . -profile test -stub-run --run_sra_fetch true — RNA-seq subgraph validation (dual-stub)
  • Real-data HPCC run — confirms grouping picks the right representative and reads join to the right strain

🤖 Generated with Claude Code

https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa

… subworkflows/local/)

Issue #2 — INPUT_CHECK shared subworkflow
  Create subworkflows/local/input_check.nf encapsulating all samplesheet
  ingestion: splitCsv, taxon/asmid/suppress/n_test filters, SampleUtils.makeMeta,
  genome path resolution.  Two emits:
    samples — val(meta)         filters applied, no genome-existence check
    genomes — tuple(meta, gz)   additionally gz.exists() filtered
  funannotate.nf: drop the 70-line inline suppressSet/taxonFilter/asmidFilter/
  jobs-channel block; replace with INPUT_CHECK() + INPUT_CHECK.out.genomes.
  postpredict now derives from INPUT_CHECK.out.samples (no duplicate splitCsv).
  earlgrey_mask.nf: skip_select_reps=true path now calls INPUT_CHECK and maps
  meta → reps CSV, gaining taxon/asmid/suppress/n_test filtering it previously
  lacked entirely.

Issue #3 — repo skeleton
  Create directory layout: modules/local/, subworkflows/local/, workflows/
  git mv modules/asm_stats.nf → modules/local/asm_stats.nf
  Extract three annotation processes from funannotate.nf inline:
    modules/local/antismash_run.nf     (ANTISMASH_RUN)
    modules/local/interproscan_run.nf  (INTERPROSCAN_RUN)
    modules/local/signalp_run.nf       (SIGNALP_RUN)
  Delete stale modules/annotation_tools.nf (old 10-tuple versions, superseded
  by the meta-map extraction into modules/local/).
  Update all include paths in funannotate.nf and earlgrey_mask.nf.

Stub-run gate: 15 completed, 0 failed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbWwmm9v6dJn13sgAJn3Sa
@hyphaltip
hyphaltip merged commit 5912711 into main Jun 29, 2026
2 checks passed
@hyphaltip
hyphaltip deleted the refactor/meta-map branch June 29, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant