Skip to content

feat(io): add io_uring backend#2389

Open
LHT129 wants to merge 1 commit into
antgroup:mainfrom
LHT129:feat/io-uring-support
Open

feat(io): add io_uring backend#2389
LHT129 wants to merge 1 commit into
antgroup:mainfrom
LHT129:feat/io-uring-support

Conversation

@LHT129

@LHT129 LHT129 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

What

This PR adds an optional io_uring IO backend for Linux systems with liburing available.

Changes include:

  • Add UringIO, UringIOContext, and UringIOParameter.
  • Add CMake detection for liburing via ENABLE_LIBURING, HAVE_LIBURING, and NO_LIBURING.
  • Wire uring_io into IO parameter parsing, IO headers, flatten/bucket/graph datacell factories, and RabitQ split IO type validation.
  • Fall back from uring_io to buffer_io with a one-time warning when liburing is unavailable.
  • Add UringIO and UringIOParameter unit tests.

Both filtered test runs passed with 4 test cases and 16805 assertions.

Closes #1533

Copilot AI review requested due to automatic review settings July 2, 2026 05:12
@LHT129 LHT129 added kind/feature Brand-new functionality or capabilities 引入全新的功能、新特性或新能力 version/0.18 labels Jul 2, 2026
@mergify

mergify Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 3 merge protections satisfied — ready to merge.

Show 3 satisfied protections

🟢 Require kind label

  • label~=^kind/

🟢 Require version label

  • label~=^version/

🟢 Require linked issue for feature/bug PRs

  • body~=(?im)(?:^|[\s\-\*])(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s+(?:#\d+|[\w.\-]+/[\w.\-]+#\d+|https?://github\.com/[\w.\-]+/[\w.\-]+/issues/\d+)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for the io_uring asynchronous I/O framework (uring_io) as an alternative I/O backend. It adds CMake configuration options to detect and link liburing, integrates the new I/O type into various datacell factories, and implements the UringIO class along with its parameters and unit tests. The feedback highlights several critical areas for improvement: a potential file descriptor and file leak in the UringIO constructor if initialization fails, a performance bottleneck caused by mixing O_DIRECT reads with buffered writes (requiring frequent fsync calls), error-prone manual resource management in MultiReadImpl that should leverage RAII, and the use of preprocessor macros for fallback types which should be replaced with clean C++ type aliases.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/io/uring_io/uring_io.cpp
Comment thread src/io/uring_io/uring_io.cpp
Comment thread src/io/uring_io.cpp Outdated
Comment thread src/io/uring_io/uring_io.h
Comment thread src/io/uring_io/uring_io_parameter.h

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an optional io_uring-based I/O backend for Linux builds, integrating it into IO parameter parsing and the major datacell factories, with CMake-based detection/linking of liburing and fallback behavior when unavailable.

Changes:

  • Introduces UringIO, UringIOContext, and UringIOParameter (plus unit tests) under HAVE_LIBURING.
  • Extends IO type parsing/dispatch and datacell factories to recognize "uring_io", with fallback-to-buffer behavior when liburing is missing.
  • Adds a build option (ENABLE_LIBURING) and CMake detection/linking for liburing.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
src/io/uring_io.h Declares UringIO and provides a no-liburing alias fallback.
src/io/uring_io.cpp Implements UringIO operations including multi-read using io_uring.
src/io/uring_io_test.cpp Adds unit tests for UringIO behavior.
src/io/uring_io_parameter.h Defines UringIOParameter with fallback behavior when liburing is absent.
src/io/uring_io_parameter.cpp Implements JSON (de)serialization for UringIOParameter.
src/io/uring_io_parameter_test.cpp Adds a unit test for UringIOParameter JSON round-tripping.
src/io/uring_io_context.h Adds an io_uring context wrapper and pool type.
src/io/io_parameter.cpp Adds "uring_io" parsing with one-time fallback warning when liburing is unavailable.
src/io/io_headers.h Wires uring_io.h into the common IO header set.
src/io/CMakeLists.txt Adds new sources and links liburing when available.
src/inner_string_params.h Adds the "uring_io" IO type string constant.
src/datacell/rabitq_split_datacell.h Allows "uring_io" as a known IO type for validation.
src/datacell/graph_interface.cpp Enables graph datacell instantiation with UringIO.
src/datacell/flatten_interface.cpp Enables flatten datacell instantiation with UringIO.
src/datacell/bucket_interface.cpp Enables bucket datacell instantiation with UringIO.
cmake/VSAGSystemDeps.cmake Detects liburing and defines HAVE_LIBURING/NO_LIBURING.
cmake/VSAGOptions.cmake Adds ENABLE_LIBURING option.

Comment thread src/io/uring_io.cpp
Comment thread src/io/uring_io.cpp Outdated
Comment thread src/io/uring_io.cpp
Comment thread src/io/uring_io.cpp Outdated
Comment thread src/io/uring_io_parameter.h Outdated
Comment thread src/io/uring_io_test.cpp
Comment thread src/io/uring_io_test.cpp
Comment thread src/io/uring_io_test.cpp
Comment thread src/io/uring_io_parameter_test.cpp
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 3b1b211 to da3ff4d Compare July 2, 2026 05:39
@LHT129 LHT129 self-assigned this Jul 2, 2026
Copilot AI review requested due to automatic review settings July 2, 2026 06:21
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from da3ff4d to 8ca7877 Compare July 2, 2026 06:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Comment thread src/io/uring_io.cpp
Comment thread src/io/uring_io.cpp Outdated
Comment thread src/io/uring_io.cpp
@LHT129 LHT129 force-pushed the feat/io-uring-support branch 2 times, most recently from 640f09c to acfe848 Compare July 2, 2026 07:07
Copilot AI review requested due to automatic review settings July 2, 2026 07:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Comment thread src/io/uring_io.cpp Outdated
Comment thread src/io/uring_io.cpp
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from acfe848 to c857bb3 Compare July 2, 2026 07:28
Copilot AI review requested due to automatic review settings July 2, 2026 11:29
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from c857bb3 to 273d1d1 Compare July 2, 2026 11:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Comment thread src/io/uring_io.cpp
Comment thread src/io/uring_io/uring_io_context.h
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 273d1d1 to 0481c6b Compare July 3, 2026 03:07
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 0481c6b to fa96ad3 Compare July 6, 2026 03:05
Copilot AI review requested due to automatic review settings July 6, 2026 03:05
@vsag-bot

vsag-bot commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

/label S-waiting-on-review
/waiting-on reviewer
/request-review @jiaweizone
/request-review @wxyucs
/request-review @inabao

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.

Comment thread src/io/uring_io/uring_io.cpp
Comment thread src/io/uring_io/uring_io.cpp
Comment thread cmake/VSAGSystemDeps.cmake
Comment thread cmake/VSAGSystemDeps.cmake
Comment thread src/io/CMakeLists.txt
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from fa96ad3 to 299ed9a Compare July 6, 2026 03:20
Copilot AI review requested due to automatic review settings July 6, 2026 09:18
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 299ed9a to 6d21339 Compare July 6, 2026 09:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

Comment thread src/io/uring_io/uring_io.cpp
Comment thread src/io/uring_io/uring_io.cpp Outdated
Comment thread src/io/uring_io/uring_io.cpp
Comment thread cmake/VSAGSystemDeps.cmake
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 6d21339 to 9c13cc1 Compare July 6, 2026 09:31
Copilot AI review requested due to automatic review settings July 7, 2026 03:32
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 9c13cc1 to 0afa062 Compare July 7, 2026 03:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Comment thread src/io/uring_io/uring_io.cpp
Comment thread src/io/uring_io/uring_io.cpp Outdated
Comment thread src/io/uring_io/uring_io_parameter.cpp Outdated
Add an optional UringIO backend with liburing detection, IO parameter parsing, datacell factory wiring, and unit coverage for both liburing and fallback paths.

Signed-off-by: LHT129 <tianlan.lht@antgroup.com>
@LHT129 LHT129 force-pushed the feat/io-uring-support branch from 0afa062 to 59e89a0 Compare July 7, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Brand-new functionality or capabilities 引入全新的功能、新特性或新能力 size/XL version/1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support iouring for vsag

3 participants