Skip to content

reorg: rewrite agfs with rust, and named with ragfs, keep License#1220

Closed
markwhen wants to merge 8 commits intovolcengine:mainfrom
MaojiaSheng:restore-pr-1209
Closed

reorg: rewrite agfs with rust, and named with ragfs, keep License#1220
markwhen wants to merge 8 commits intovolcengine:mainfrom
MaojiaSheng:restore-pr-1209

Conversation

@markwhen
Copy link
Copy Markdown
Contributor

@markwhen markwhen commented Apr 4, 2026

Summary

  • restore the accidentally reset changes from the original closed PR Rewrite agfs with rust and only keep simple local mode #1209 onto a dedicated branch
  • rewrite AGFS as Rust-based ragfs, including the new Rust core, plugins, server, and Python binding packaging changes
  • update the Python integration and build pipeline to load and configure the bundled Rust-backed implementation

Test plan

  • cargo check -p ragfs
  • cargo check -p ragfs-python
  • python -m pytest tests/agfs

🤖 Generated with Claude Code

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


openviking seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 4, 2026

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

1209 - Partially compliant

Compliant requirements:

  • Rewrote AGFS as Rust-based ragfs
  • Added Rust core, plugins, server, Python binding packaging
  • Updated Python integration and build pipeline

Non-compliant requirements:

  • Kept more than simple local mode (added S3FS, SQLFS, QueueFS, etc.)
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🏅 Score: 78
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 Multiple PR themes

Sub-PR theme: Add Python binding selection for RAGFS_IMPL

Relevant files:

  • openviking/pyagfs/init.py
  • openviking/utils/agfs_utils.py
  • openviking_cli/utils/config/agfs_config.py

Sub-PR theme: Add maturin build for ragfs-python

Relevant files:

  • setup.py

Sub-PR theme: Add Rust ragfs core and plugins

Relevant files:

  • crates/ragfs/**/*
  • crates/ragfs-python/**/*

⚡ Recommended focus areas for review

Test Assertion Mismatch

test_queuefs_enqueue_dequeue and test_queuefs_peek expect raw message bytes but dequeue/peek return JSON {"id":"...","data":"..."}. This will cause test failures.

#[tokio::test]
async fn test_queuefs_enqueue_dequeue() {
    let fs = QueueFileSystem::new();

    // Create a queue first
    fs.mkdir("/test", 0o755).await.unwrap();

    // Enqueue messages
    let data1 = b"message 1";
    let data2 = b"message 2";

    fs.write("/test/enqueue", data1, 0, WriteFlag::None)
        .await
        .unwrap();
    fs.write("/test/enqueue", data2, 0, WriteFlag::None)
        .await
        .unwrap();

    // Dequeue messages
    let result1 = fs.read("/test/dequeue", 0, 0).await.unwrap();
    assert_eq!(result1, data1);

    let result2 = fs.read("/test/dequeue", 0, 0).await.unwrap();
    assert_eq!(result2, data2);

    // Queue should be empty
    let result = fs.read("/test/dequeue", 0, 0).await;
    assert!(result.is_err());
}

#[tokio::test]
async fn test_queuefs_peek() {
    let fs = QueueFileSystem::new();

    // Create a queue first
    fs.mkdir("/test", 0o755).await.unwrap();

    // Enqueue a message
    let data = b"test message";
    fs.write("/test/enqueue", data, 0, WriteFlag::None)
        .await
        .unwrap();

    // Peek should return the message without removing it
    let result1 = fs.read("/test/peek", 0, 0).await.unwrap();
    assert_eq!(result1, data);

    let result2 = fs.read("/test/peek", 0, 0).await.unwrap();
    assert_eq!(result2, data);

    // Dequeue should still work
    let result3 = fs.read("/test/dequeue", 0, 0).await.unwrap();
    assert_eq!(result3, data);
}
Incorrect Test Assertion

test_queuefs_read_dir expects 5 control files but the implementation returns 6 (missing "ack").

let entries = fs.read_dir("/test").await.unwrap();
assert_eq!(entries.len(), 5);
Redundant Code

effective_impl and env_impl are computed but not used; get_binding_client already handles RAGFS_IMPL env var precedence.

config_impl = getattr(agfs_config, "impl", "auto")
env_impl = os.environ.get("RAGFS_IMPL", "").lower() or None
effective_impl = env_impl or config_impl or "auto"

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 4, 2026

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix error handling in path_exists method

Fix the path_exists method to properly handle database errors instead of using
unwrap_or(0), which silently swallows real errors and could incorrectly return false
on failure.

crates/ragfs/src/plugins/sqlfs/backend.rs [154-164]

 fn path_exists(&self, path: &str) -> Result<bool> {
     let conn = self.conn.lock().map_err(|e| Error::internal(e.to_string()))?;
     let mut stmt = conn
         .prepare_cached("SELECT COUNT(*) FROM files WHERE path = ?1")
         .map_err(|e| Error::internal(format!("prepare error: {}", e)))?;
 
     let count: i64 = stmt
         .query_row(params![path], |row| row.get(0))
-        .unwrap_or(0);
+        .map_err(|e| Error::internal(format!("query error: {}", e)))?;
 
     Ok(count > 0)
 }
Suggestion importance[1-10]: 8

__

Why: The original code uses unwrap_or(0) which silently swallows database errors, potentially leading to incorrect false results. The fix properly propagates errors, improving correctness.

Medium
Use async filesystem APIs in LocalFS

Replace blocking std::fs calls with async tokio::fs equivalents in the
LocalFileSystem async trait methods. Blocking I/O in async functions can degrade
runtime performance by blocking executor threads.

crates/ragfs/src/plugins/localfs/mod.rs [66-87]

 #[async_trait]
 impl FileSystem for LocalFileSystem {
     async fn create(&self, path: &str) -> Result<()> {
         let local_path = self.resolve_path(path);
 
         // Check if file already exists
-        if local_path.exists() {
+        if tokio::fs::metadata(&local_path).await.is_ok() {
             return Err(Error::AlreadyExists(path.to_string()));
         }
 
         // Check if parent directory exists
         if let Some(parent) = local_path.parent() {
-            if !parent.exists() {
+            if tokio::fs::metadata(parent).await.is_err() {
                 return Err(Error::NotFound(parent.to_string_lossy().to_string()));
             }
         }
 
         // Create empty file
-        fs::File::create(&local_path)
+        tokio::fs::File::create(&local_path)
+            .await
             .map_err(|e| Error::plugin(format!("failed to create file: {}", e)))?;
 
         Ok(())
     }
+    // Update other methods similarly to use tokio::fs async APIs
     ...
 }
Suggestion importance[1-10]: 6

__

Why: Replacing blocking std::fs calls with async tokio::fs equivalents in async trait methods is a valid performance improvement, as blocking I/O can degrade executor performance.

Low

@MaojiaSheng MaojiaSheng closed this Apr 4, 2026
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Apr 4, 2026
@MaojiaSheng MaojiaSheng deleted the restore-pr-1209 branch April 13, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants