Added PerFrameTransferHelper for streaming and async readback. by JackOfBlades232 · Pull Request #86 · AlexandrShcherbakov/etna

JackOfBlades232 · 2026-01-17T22:53:05Z

Added a new class etna::PerFrameTransferHelper for handling upload and readback inside the application loop.

Has a sync API for uploads and an async API for uploads and readbacks. Operates with one staging buffer for both, so through a combination of splitting the interface into multiple "classes" and runtime checks ensures all readback is processed before upload. Handles asynchronous operations by giving the user a state struct and having the user call methods on this struct.

Usage example:

// Somewhere 
etna::PerFrameTransferHelper th{etna::PerFrameTransferHelper::CreateInfo{
  .totalStagingSize = 4096 * 4096 * 4, .wc = &wc}};

// init calls for async operations states can be called any time
auto bufferUpload = th.initUploadBufferAsync(buf1, 0, bufData1);
auto bufferReadback = th.initReadbackBufferAsync(bufStorageDest2, buf2, 0);
auto imageUpload = th.initUploadImageAsync(img1, 0, 0, imgData1);

// actual sync/progress calls must be done every frame inside of processing
// scope, for that the API is split into several classes to enforce order.

if (auto frame = th.beginFrame())
{
  if (auto rb = frame.beginReadback())
  {
    if (!rb.hasSpaceThisFrame())
      ; // Out of staging space, all upload/progress calls will do nothing
    if (rb.progressBufferReadbackAsync(cmd_buf, bufferReadback))
      ; // Readback is done
  }

  if (auto up = frame.beginUpload())
  {
    if (up.progressBufferUploadAsync(cmd_buf, bufferUpload))
      ; // Upload will be completed this frame
    if (up.progressImageUploadAsync(cmd_buf, imageUpload))
      ; // Upload will be completed this frame
    if (up.uploadBufferSync(cmd_buf, buf3, 0, bufData3))
      ; // There was space in staging and upload will be done
    if (up.uploadImageSync(cmd_buf, img2, 0, 0, imgData2))
      ; // There was space in staging and upload will be done

    for (auto &texUpload : texUploads)
    {
      if (!up.hasSpaceThisFrame())
        break;
      if (texUpload.done())
        continue;
      if (up.progressImageUploadAsync(cmd_buf, texUpload))
        ; // Upload will be completed this frame
    }
  }
}

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 13. Check the log or trigger a new build to see more.

github-actions · 2026-01-17T22:55:48Z

+  uint32_t offset;
+  std::span<std::byte const> src;
+
+  bool done() const { return transferHelper && src.size() == 0; }


warning: implicit conversion 'const PerFrameTransferHelper *' -> 'bool' [readability-implicit-bool-conversion]

Suggested change

bool done() const { return transferHelper && src.size() == 0; }

bool done() const { return (transferHelper != nullptr) && src.size() == 0; }

github-actions · 2026-01-17T22:55:48Z

+  uint32_t offset;
+  std::span<std::byte const> src;
+
+  bool done() const { return transferHelper && src.size() == 0; }


warning: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty]

Suggested change

bool done() const { return transferHelper && src.size() == 0; }

bool done() const { return transferHelper && src.empty(); }

Additional context

/usr/include/c++/13/span:262: method 'span'::empty() defined here

empty() const noexcept ^

github-actions · 2026-01-17T22:55:48Z

+  size_t nextIssueSlot;
+  std::span<std::byte> dst;
+
+  bool done() const { return transferHelper && dst.size() == 0; }


warning: implicit conversion 'const PerFrameTransferHelper *' -> 'bool' [readability-implicit-bool-conversion]

Suggested change

bool done() const { return transferHelper && dst.size() == 0; }

bool done() const { return (transferHelper != nullptr) && dst.size() == 0; }

github-actions · 2026-01-17T22:55:48Z

+  size_t nextIssueSlot;
+  std::span<std::byte> dst;
+
+  bool done() const { return transferHelper && dst.size() == 0; }


warning: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty]

Suggested change

bool done() const { return transferHelper && dst.size() == 0; }

bool done() const { return transferHelper && dst.empty(); }

Additional context

/usr/include/c++/13/span:262: method 'span'::empty() defined here

empty() const noexcept ^

github-actions · 2026-01-17T22:55:48Z

+  vk::Offset3D offset;
+  std::span<std::byte const> src;
+
+  bool done() const { return transferHelper && src.size() == 0; }


warning: implicit conversion 'const PerFrameTransferHelper *' -> 'bool' [readability-implicit-bool-conversion]

Suggested change

bool done() const { return transferHelper && src.size() == 0; }

bool done() const { return (transferHelper != nullptr) && src.size() == 0; }

github-actions · 2026-01-17T22:55:48Z

+  vk::Offset3D offset;
+  std::span<std::byte const> src;
+
+  bool done() const { return transferHelper && src.size() == 0; }


warning: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty]

Suggested change

bool done() const { return transferHelper && src.size() == 0; }

bool done() const { return transferHelper && src.empty(); }

Additional context

/usr/include/c++/13/span:262: method 'span'::empty() defined here

empty() const noexcept ^

github-actions · 2026-01-17T22:55:48Z

+
+    void finish();
+
+    operator bool() const { return self != nullptr; }


warning: 'operator bool' must be marked explicit to avoid unintentional implicit conversions [google-explicit-constructor]

Suggested change

operator bool() const { return self != nullptr; }

explicit operator bool() const { return self != nullptr; }

github-actions · 2026-01-17T22:55:48Z

+
+    void finish();
+
+    operator bool() const { return self != nullptr; }


warning: 'operator bool' must be marked explicit to avoid unintentional implicit conversions [google-explicit-constructor]

Suggested change

operator bool() const { return self != nullptr; }

explicit operator bool() const { return self != nullptr; }

github-actions · 2026-01-17T22:55:48Z

+
+    void finish();
+
+    operator bool() const { return self != nullptr; }


warning: 'operator bool' must be marked explicit to avoid unintentional implicit conversions [google-explicit-constructor]

Suggested change

operator bool() const { return self != nullptr; }

explicit operator bool() const { return self != nullptr; }

github-actions · 2026-01-17T22:55:48Z

+
+void PerFrameTransferHelper::UploadProcessor::finish()
+{
+  if (!self)


warning: implicit conversion 'PerFrameTransferHelper *' -> 'bool' [readability-implicit-bool-conversion]

Suggested change

if (!self)

if (self == nullptr)

github-actions

clang-tidy made some suggestions

github-actions · 2026-01-18T12:21:54Z

+
+void PerFrameTransferHelper::FrameProcessor::finish()
+{
+  if (!self)


warning: implicit conversion 'PerFrameTransferHelper *' -> 'bool' [readability-implicit-bool-conversion]

Suggested change

if (!self)

if (self == nullptr)

github-actions · 2026-01-18T12:21:54Z

+    }
+  }
+
+  if (state.dst.size() == 0)


warning: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty]

Suggested change

if (state.dst.size() == 0)

if (state.dst.empty())

Additional context

/usr/include/c++/13/span:262: method 'span'::empty() defined here

empty() const noexcept ^

github-actions · 2026-01-18T12:30:34Z