fix: dtype alignment for attention mask and add bf16 support by NULL204 · Pull Request #14 · EutropicAI/cccv

NULL204 · 2026-06-16T16:51:38Z

This pull request adds support for bfloat16 (bf16) precision throughout the model loading and inference pipeline, providing users with an alternative to fp16 for half-precision inference. It also ensures proper dtype handling in attention modules and data conversions, improving numerical stability and compatibility with various hardware and models.

Precision and dtype support:

Added a bf16 (bfloat16) option to model loading interfaces (from_pretrained, from_config, and CCBaseModel), which takes precedence over fp16 and is used throughout the codebase for half-precision inference. This includes updating documentation and parameter lists to reflect the new option. [1] [2] [3] [4] [5] [6] [7] [8] [9]
Updated model initialization and inference code to consistently use the correct half-precision dtype (torch.bfloat16 or torch.float16), with fallback to fp32 if half-precision is not supported.
Modified image and tensor conversion logic in inference methods to cast outputs back to float32 before converting to NumPy arrays, ensuring compatibility since NumPy does not support bfloat16. [1] [2] [3] [4] [5]

Attention module stability:

Ensured that relative position bias and attention masks are cast to the same dtype as attention tensors in several transformer-based architectures, preventing dtype mismatches and potential computation errors. [1] [2] [3] [4]

DAT/HAT/SwinIR window attention added the float32 attention mask and relative position bias directly onto a half-precision attn tensor, which type-promoted the result back to float32 and then failed at the attn @ v matmul with "expected scalar type Half but found Float" (same under bf16). Cast both to attn.dtype before adding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add a bf16 flag alongside fp16 to the model base class and AutoModel (from_pretrained / from_config). Weights and inputs are cast through a shared half_dtype, with bf16 taking precedence over fp16. bf16 has the same dynamic range as fp32, so it avoids the fp16 overflow that produces NaN/black output on transformer SR models (e.g. DAT, SwinIR) while keeping fp16's memory savings. Also cast model output back to float32 before converting to numpy, since numpy has no bfloat16 dtype. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adds bfloat16 (bf16) support alongside fp16 for inference, aiming to reduce overflow/NaN issues on some SR/transformer models while ensuring outputs can be converted to NumPy safely.

Changes:

Add bf16 option to model construction APIs and plumb it through to base model setup.
Standardize half-precision casting via self.half_dtype and update inference input/output conversions accordingly.
Ensure attention bias/mask tensors match attention dtype to avoid dtype-mismatch errors in half precision.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`cccv/model/base_model.py`	Introduces `bf16` flag, `half_dtype`, and applies half-precision casting to model weights.
`cccv/auto/model.py`	Exposes `bf16` in factory functions and forwards it to model constructors.
`cccv/model/sr_base_model.py`	Casts inputs to `half_dtype` and converts outputs to float32 before NumPy.
`cccv/model/vsr_base_model.py`	Updates video SR inference to use `half_dtype` and float32 conversion before NumPy.
`cccv/model/vfi/rife_model.py`	Updates VFI inference to use `half_dtype` and float32 conversion before NumPy.
`cccv/model/vfi/drba_model.py`	Updates VFI inference to use `half_dtype` and float32 conversion before NumPy.
`cccv/arch/sr/swinir_arch.py`	Casts attention bias/mask to `attn.dtype` for half-precision compatibility.
`cccv/arch/sr/hat_arch.py`	Casts attention bias/mask to `attn.dtype` for half-precision compatibility.
`cccv/arch/sr/dat_arch.py`	Casts attention bias/mask to `attn.dtype` for half-precision compatibility.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

gemini-code-assist

Code Review

This pull request introduces support for bf16 (bfloat16) precision across various models and architectures, updating model initialization, inference pipelines, and attention mechanisms. Feedback on the changes suggests adding a check using torch.cuda.is_bf16_supported() during initialization to prevent runtime crashes on older GPUs that do not support bfloat16, falling back to fp16 if necessary.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

codecov · 2026-06-17T02:37:12Z

Codecov Report

❌ Patch coverage is 73.14815% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.91%. Comparing base (79725f4) to head (35cda6c).

Files with missing lines	Patch %	Lines
cccv/model/base_model.py	79.72%	15 Missing ⚠️
cccv/model/vsr_base_model.py	40.00%	6 Missing ⚠️
cccv/model/vfi/drba_model.py	50.00%	4 Missing ⚠️
cccv/model/vfi/rife_model.py	50.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #14      +/-   ##
==========================================
- Coverage   92.76%   91.91%   -0.86%     
==========================================
  Files          40       40              
  Lines        1216     1298      +82     
==========================================
+ Hits         1128     1193      +65     
- Misses         88      105      +17

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Tohrusky · 2026-06-29T10:48:32Z

need a unit test for BF16 @NULL204

Cover the bf16 code path end to end (AutoModel.from_config(bf16=True) -> weight/input cast -> inference -> numpy output) with an SSIM + output-size assertion, mirroring test_sr_fp16. Runs on CPU in CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

CPU bfloat16 replication padding (used by tile inference) isn't implemented in the torch build CI runs on, so test_sr_bf16 errored there. test_sr_fp16 already guards CPU half-precision with skipif(not torch_2_4); apply the same guard so both half-precision tests run only on the pinned dev torch (GPU), not CI's CPU build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Justin62628

Inference of bf16 on older GPUs not tested. Add fallbacks of disabling bf16 with a test inference (e.g. a 540p inference before actual run, if this inference round fails, disable bf16 as fallback).
Transformer Arch modification not tested. Add DAT, HAT, SwinIR unit test.

Tohrusky · 2026-07-07T13:52:23Z

please click resolve conversation after you've done

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Tohrusky · 2026-07-09T07:37:47Z

        tile_pad: int = 8,
        pad_img: Optional[Tuple[int, int]] = None,
+        bf16_preflight: bool = True,
+        bf16_preflight_size: Tuple[int, int] = (540, 960),


I think use 64*64 can fasten fallback

Tohrusky · 2026-07-09T07:38:43Z

    assert compare_image_size(img1, img2, cfg.scale)


+@pytest.mark.skipif(not torch_2_4, reason="Skip test if PyTorch version is not 2.4")


https://github.com/EutropicAI/cccv/blob/main/tests/util.py#L34

not CI

NULL204 and others added 2 commits June 17, 2026 00:44

Copilot AI review requested due to automatic review settings June 16, 2026 16:51

Copilot AI reviewed Jun 16, 2026

View reviewed changes

Comment thread cccv/model/base_model.py

Comment thread cccv/model/vsr_base_model.py Outdated

Comment thread cccv/model/vsr_base_model.py Outdated

Comment thread cccv/model/vfi/rife_model.py Outdated

Comment thread cccv/model/vfi/drba_model.py Outdated

gemini-code-assist Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread cccv/model/base_model.py

style: wrap overlong warning line to satisfy ruff format

cc9b4a2

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

NULL204 and others added 2 commits June 29, 2026 19:21

Tohrusky changed the title ~~Fix dtype alignment for attention mask and add bf16 support~~ fix: dtype alignment for attention mask and add bf16 support Jun 29, 2026

Tohrusky requested a review from Justin62628 June 29, 2026 13:22

Justin62628 suggested changes Jul 2, 2026

View reviewed changes

feat: add bf16 preflight support and related methods across models

8b6a567

Justin62628 approved these changes Jul 8, 2026

View reviewed changes

NULL204 and others added 3 commits July 8, 2026 22:44

style: apply pre-commit formatting

fa5caeb

test: stabilize ssim matlab assertion

585fc88

Apply suggestion from @gemini-code-assist[bot]

7c63619

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

NULL204 force-pushed the main branch from ca3bbff to 7c63619 Compare July 8, 2026 14:46

fix: refine bf16 fallback and output conversion

4693cbf

Tohrusky requested a review from Copilot July 8, 2026 17:14

Copilot started reviewing on behalf of Tohrusky July 8, 2026 17:15 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread cccv/model/base_model.py

Comment thread tests/test_util.py

NULL204 force-pushed the main branch from df4f9dd to 4693cbf Compare July 9, 2026 07:12

Tohrusky self-requested a review July 9, 2026 07:45

Tohrusky reviewed Jul 9, 2026

View reviewed changes

fix: address bf16 fallback review comments

35cda6c

Tohrusky merged commit 29d14fa into EutropicAI:main Jul 9, 2026
9 of 11 checks passed

		assert compare_image_size(img1, img2, cfg.scale)


		@pytest.mark.skipif(not torch_2_4, reason="Skip test if PyTorch version is not 2.4")

Uh oh!

Conversation

NULL204 commented Jun 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

codecov Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Tohrusky commented Jun 29, 2026

Uh oh!

Justin62628 left a comment

Choose a reason for hiding this comment

Uh oh!

Tohrusky commented Jul 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Tohrusky Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

Tohrusky Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Jun 17, 2026 •

edited

Loading