Add shuffle_after_epoch_seed argument to file-based readers. by JanuszL · Pull Request #6236 · NVIDIA/DALI

JanuszL · 2026-02-27T09:03:36Z

Adds an optional shuffle_after_epoch_seed (int32) argument to all
readers that support shuffle_after_epoch: readers.file, readers.numpy,
readers.fits, readers.coco, and readers.nemo_asr.
Previously the per-epoch shuffle was always seeded with a fixed
constant, making the dataset order identical across training runs.
The new argument lets users supply a custom base seed so that
different training runs produce statistically independent orderings,
while still guaranteeing a consistent global permutation across shards.
When omitted, the old fixed seed is used, preserving backward compatibility.
Tests added in test_shuffling.py and test_numpy.py.

Category:

New feature (non-breaking change which adds functionality)
Relates to #5827

Description:

Adds an optional shuffle_after_epoch_seed (int64) argument to all
readers that support shuffle_after_epoch: readers.file, readers.numpy,
readers.fits, readers.coco, and readers.nemo_asr.
Previously the per-epoch shuffle was always seeded with a fixed
constant, making the dataset order identical across training runs.
The new argument lets users supply a custom base seed so that
different training runs produce statistically independent orderings,
while still guaranteeing a consistent global permutation across shards.
When omitted, the old fixed seed is used, preserving backward compatibility.
Tests added in test_shuffling.py and test_numpy.py.

Additional information:

Affected modules and functionalities:

readers.file, readers.numpy, readers.fits, readers.coco, and readers.nemo_asr

Key points relevant for the review:

Tests:

Checklist

Documentation

DALI team only

Requirements

Implements new requirements
Affects existing requirements
N/A

REQ IDs: N/A

JIRA TASK: N/A

JanuszL · 2026-02-27T09:03:44Z

!build

dali-automaton · 2026-02-27T09:05:25Z

CI MESSAGE: [44956153]: BUILD FAILED

greptile-apps · 2026-02-27T09:06:48Z

Greptile Summary

This PR adds configurable per-epoch shuffle seeding to file-based readers, addressing the limitation where all training runs used identical shuffle patterns. The implementation consistently adds an optional shuffle_after_epoch_seed (int32_t) argument across five reader types: file, numpy, fits, coco, and nemo_asr.

Key changes:

Replaced hardcoded kDaliDataloaderSeed with configurable shuffle_after_epoch_seed that defaults to the same constant for backward compatibility
Added proper validation warning when seed is specified but shuffle_after_epoch is disabled
Comprehensive documentation explains multi-GPU training considerations and proper usage
Thorough test coverage validates reproducibility, different seed behavior, and backward compatibility

Assessment:
The implementation is clean, consistent across all affected readers, and properly addresses previous review feedback regarding int32_t type usage. The changes maintain backward compatibility while enabling the requested functionality for statistically independent training runs.

Confidence Score: 5/5

This PR is safe to merge with no identified issues
Implementation is consistent across all readers, previous review concerns have been addressed (int32_t type), comprehensive tests validate all scenarios, and backward compatibility is preserved
No files require special attention

Important Files Changed

Filename	Overview
dali/operators/reader/loader/file_loader.h	Adds `shuffle_after_epoch_seed` argument support with proper default handling and warning for misuse
dali/operators/reader/loader/file_label_loader.h	Mirrors file_loader.h changes, adds shuffle seed support with consistent implementation
dali/operators/reader/loader/nemo_asr_loader.h	Adds shuffle seed with lambda initialization pattern, includes warning for misuse
dali/test/python/reader/test_shuffling.py	Adds comprehensive tests for seed reproducibility, different seeds, and backward compatibility
dali/test/python/reader/test_numpy.py	Adds numpy-specific shuffle seed tests covering all critical scenarios

_{Last reviewed commit: 4445fea}

greptile-apps

_{11 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

dali/operators/reader/loader/file_loader.h

dali/operators/reader/file_reader_op.cc

* Adds an optional `shuffle_after_epoch_seed` (int32) argument to all readers that support `shuffle_after_epoch`: readers.file, readers.numpy, readers.fits, readers.coco, and readers.nemo_asr. * Previously the per-epoch shuffle was always seeded with a fixed constant, making the dataset order identical across training runs. The new argument lets users supply a custom base seed so that different training runs produce statistically independent orderings, while still guaranteeing a consistent global permutation across shards. * When omitted, the old fixed seed is used, preserving backward compatibility. * Tests added in test_shuffling.py and test_numpy.py. Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

JanuszL · 2026-02-27T09:35:47Z

!build

dali-automaton · 2026-02-27T09:40:32Z

CI MESSAGE: [44958231]: BUILD STARTED

dali-automaton · 2026-02-27T11:37:13Z

CI MESSAGE: [44958231]: BUILD PASSED

szkarpinski · 2026-02-27T11:55:10Z

dali/operators/reader/loader/file_loader.h

        current_index_(0),
        current_epoch_(0) {
+    int32_t seed_arg = kDaliDataloaderSeed;
+    spec.TryGetArgument(seed_arg, "shuffle_after_epoch_seed");


Open question: should we raise a warning, if shuffle_after_epoch_seed is set when shuffle_after_epoch=False?

Good idea. Fixed

szkarpinski · 2026-02-27T12:02:01Z

dali/operators/reader/nemo_asr_reader_op.cc

+
+.. note::
+    When using multiple DALI pipelines (e.g., for multi-GPU training), all pipeline
+    instances should use the same ``shuffle_after_epoch_seed`` to ensure a consistent


Nitpick: I believe that with the paramlinks you can use single quote and it'll link to the parameter

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

JanuszL · 2026-02-27T12:31:11Z

!build

dali-automaton · 2026-02-27T12:35:31Z

CI MESSAGE: [44966443]: BUILD STARTED

dali-automaton · 2026-02-27T15:02:22Z

CI MESSAGE: [44966443]: BUILD PASSED

greptile-apps bot reviewed Feb 27, 2026

View reviewed changes

dali/operators/reader/loader/file_loader.h Show resolved Hide resolved

JanuszL force-pushed the add_global_shufling_seed branch from c2311b0 to 7c24cd6 Compare February 27, 2026 09:23

greptile-apps bot reviewed Feb 27, 2026

View reviewed changes

dali/operators/reader/file_reader_op.cc Outdated Show resolved Hide resolved

JanuszL force-pushed the add_global_shufling_seed branch from 7c24cd6 to c616299 Compare February 27, 2026 09:34

dali-automaton assigned szkarpinski and mdabek-nvidia Feb 27, 2026

szkarpinski reviewed Feb 27, 2026

View reviewed changes

szkarpinski approved these changes Feb 27, 2026

View reviewed changes

JanuszL force-pushed the add_global_shufling_seed branch from 2f8b0c0 to 5bcb802 Compare February 27, 2026 12:30

Code review fixes

4445fea

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

JanuszL force-pushed the add_global_shufling_seed branch from 5bcb802 to 4445fea Compare February 27, 2026 12:30

mdabek-nvidia approved these changes Mar 5, 2026

View reviewed changes

JanuszL merged commit ef8bd5c into NVIDIA:main Mar 5, 2026
7 checks passed

JanuszL deleted the add_global_shufling_seed branch March 5, 2026 08:16

JanuszL mentioned this pull request Mar 5, 2026

The shuffle_after_epoch parameter in fn.readers.numpy is independent of the seed. #5827

Open

Conversation

JanuszL commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Category:

Description:

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

Checklist

Documentation

DALI team only

Requirements

Uh oh!

JanuszL commented Feb 27, 2026

Uh oh!

dali-automaton commented Feb 27, 2026

Uh oh!

greptile-apps bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

greptile-apps bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JanuszL commented Feb 27, 2026

Uh oh!

dali-automaton commented Feb 27, 2026

Uh oh!

dali-automaton commented Feb 27, 2026

Uh oh!

szkarpinski Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

JanuszL Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

szkarpinski Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

JanuszL Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

JanuszL commented Feb 27, 2026

Uh oh!

dali-automaton commented Feb 27, 2026

Uh oh!

dali-automaton commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JanuszL commented Feb 27, 2026 •

edited

Loading

greptile-apps bot commented Feb 27, 2026 •

edited

Loading

greptile-apps bot left a comment •

edited

Loading