Xydra.Core

A high-performance, multi-threaded download engine for .NET

Features • Installation • Quick Start • API Reference • Architecture • License

---

Overview

Xydra.Core is a modern, high-performance download engine library for .NET applications. Built with async/await patterns and designed for maximum throughput, it provides multi-threaded downloading with automatic resume support, chunk-based transfers, and comprehensive progress tracking.

Originally inspired by Rust's fdm-core, Xydra.Core brings enterprise-grade download capabilities to the .NET ecosystem with a clean, event-driven API.

Features

Core Capabilities

• 🚀 Multi-threaded Downloads - Split large files into chunks and download them in parallel (up to 64 connections per server)
• ⏸️ Pause & Resume - Full resume support with automatic state persistence
• 📊 Real-time Progress - Track speed, ETA, and per-chunk progress
• 🔄 Automatic Retry - Built-in retry logic for failed chunks with exponential backoff
• 🔗 Link Expiration Detection - Automatically detects expired/invalid download links (403, 401, 410 errors)
• 💾 State Persistence - Download state saved to disk for crash recovery
• 📁 Smart File Naming - Extracts filenames from Content-Disposition headers and URLs
• 🗂️ File Type Detection - Automatic categorization with icons for 50+ file types

Technical Highlights

• Sparse File Support - Pre-allocates disk space efficiently on Windows (NTFS)
• Async/Await - Fully asynchronous API with cancellation token support
• Thread-Safe - Concurrent collections and proper synchronization
• Memory Efficient - Streaming downloads with configurable buffer sizes (1MB default)
• ETag Validation - Ensures file integrity during resume operations

Requirements

• .NET 9.0 or later
• Windows (for sparse file support) or any .NET-supported platform

Installation

NuGet Package (Coming Soon)

dotnet add package Xydra.Core

Project Reference

<ProjectReference Include="..\Xydra.Core\Xydra.Core.csproj" />

Manual Build

cd Xydra.Core
dotnet build -c Release

Quick Start

Basic Download

using Xydra.Core;

// Create download manager
using var manager = new DownloadManager("./data");

// Add and start a download
var download = await manager.AddDownloadAsync(
    url: "https://example.com/file.zip",
    savePath: @"C:\Downloads",
    connections: 8,
    startImmediately: true
);

Console.WriteLine($"Downloading: {download.FileName}");
Console.WriteLine($"Size: {download.TotalBytes / 1024 / 1024} MB");

With Progress Tracking

using Xydra.Core;

using var manager = new DownloadManager("./data");

// Subscribe to events
manager.DownloadProgressChanged += item =>
{
    Console.WriteLine($"Progress: {item.Progress:F1}% | Speed: {item.Speed / 1024 / 1024:F2} MB/s");
};

manager.DownloadStatusChanged += item =>
{
    Console.WriteLine($"Status: {item.Status}");
};

// Start download
var download = await manager.AddDownloadAsync("https://example.com/large-file.iso");

Using the Engine Directly

using Xydra.Core;

using var engine = new DownloadEngine();

// Get file info without downloading
var info = await engine.GetFileInfoAsync("https://example.com/file.zip");
Console.WriteLine($"File: {info.FileName}");
Console.WriteLine($"Size: {info.Size} bytes");
Console.WriteLine($"Resume Support: {info.SupportsResume}");

// Create download item
var item = new DownloadItem
{
    Url = "https://example.com/file.zip",
    FileName = info.FileName,
    SavePath = @"C:\Downloads",
    TotalBytes = info.Size,
    SupportsResume = info.SupportsResume,
    Connections = 8
};

// Subscribe to events
engine.DownloadProgressChanged += d => Console.WriteLine($"{d.Progress:F1}%");
engine.DownloadCompleted += d => Console.WriteLine("Done!");

// Start download
await engine.StartDownloadAsync(item);

Pause and Resume

// Pause a download
manager.PauseDownload(download.Id);

// Resume later
await manager.StartDownloadAsync(download.Id);

Handle Expired Links

manager.DownloadStatusChanged += async item =>
{
    if (item.Status == DownloadStatus.LinkExpired)
    {
        Console.WriteLine("Link expired! Please provide a new URL.");
        
        // Update with new URL
        string newUrl = GetNewUrlFromUser();
        bool success = await manager.UpdateLinkAsync(item.Id, newUrl);
        
        if (success)
        {
            await manager.StartDownloadAsync(item.Id);
        }
    }
};

API Reference

DownloadEngine

The core download engine that handles HTTP requests and file operations.

Constructor

public DownloadEngine()

Creates a new download engine with:

• 64 max connections per server
• Infinite timeout (per-request handling)
• Automatic redirect following

Methods

Method	Description
GetFileInfoAsync(string url)	Gets file information (size, name, resume support) without downloading
StartDownloadAsync(DownloadItem item)	Starts or resumes a download
PauseDownload(string downloadId)	Pauses an active download
CancelDownload(string downloadId)	Cancels an active download
IsDownloadActive(string downloadId)	Checks if a download is currently active

Events

Event	Description
DownloadProgressChanged	Fired periodically with progress updates (~5 times/second)
DownloadStatusChanged	Fired when download status changes
DownloadCompleted	Fired when download completes successfully
DownloadFailed	Fired when download fails with exception details

DownloadManager

High-level manager that handles queue, persistence, and coordinates with the engine.

Constructor

public DownloadManager(string dataPath)

• dataPath: Directory for storing download state (downloads.json)

Properties

Property	Type	Description
Downloads	IReadOnlyList<DownloadItem>	List of all downloads
DefaultSavePath	string	Default save directory
DefaultConnections	int	Default connection count (8)

Methods

Method	Description
AddDownloadAsync(...)	Adds a new download and optionally starts it
StartDownloadAsync(string id)	Starts or resumes a download
PauseDownload(string id)	Pauses a download
CancelDownload(string id)	Cancels a download
RemoveDownload(string id, bool deleteFile)	Removes download from queue
UpdateLinkAsync(string id, string newUrl)	Updates URL for expired links
GetDownload(string id)	Gets download by ID
ClearCompleted()	Removes all completed downloads
SaveDownloads()	Manually saves download state

DownloadItem

Represents a download with all metadata and progress.

Properties

Property	Type	Description
Id	string	Unique identifier (GUID)
Url	string	Download URL
FileName	string	Target filename
SavePath	string	Save directory
TotalBytes	long	Total file size
DownloadedBytes	long	Bytes downloaded
Status	DownloadStatus	Current status
Connections	int	Number of parallel connections
Progress	double	Progress percentage (0-100)
Speed	double	Current speed (bytes/sec)
EstimatedTimeRemaining	TimeSpan?	ETA
SupportsResume	bool	Server supports byte ranges
ETag	string?	File ETag for validation
Chunks	List<ChunkProgress>	Per-chunk progress
FileType	FileTypeInfo	Detected file type info
CreatedAt	DateTime	When download was created
CompletedAt	DateTime?	When download completed
ErrorMessage	string?	Error details if failed

DownloadStatus

public enum DownloadStatus
{
    Queued,      // Waiting to start
    Downloading, // Currently downloading
    Paused,      // Paused by user
    Completed,   // Successfully completed
    Failed,      // Failed with error
    Cancelled,   // Cancelled by user
    LinkExpired  // URL expired or access denied
}

DownloadFileInfo

Returned by GetFileInfoAsync().

public class DownloadFileInfo
{
    public long Size { get; set; }
    public string FileName { get; set; }
    public bool SupportsResume { get; set; }
    public string? ETag { get; set; }
}

FileTypeDetector

Static utility for detecting file types.

var fileType = FileTypeDetector.Detect("movie.mp4");
// fileType.Category = "Video"
// fileType.Icon = "🎬"
// fileType.MimeType = "video/mp4"

Supported Categories:

• Video (mp4, mkv, avi, mov, webm, etc.)
• Audio (mp3, wav, flac, aac, ogg, etc.)
• Image (jpg, png, gif, webp, svg, etc.)
• Document (pdf, doc, docx, xls, xlsx, ppt, pptx)
• Archive (zip, rar, 7z, tar, gz)
• Executable (exe, msi, dmg, deb, rpm)
• Code (js, ts, py, rs, go, java, c, cpp, cs)
• Web (html, css, json, xml)
• And more...

Architecture

Download Strategies

Xydra.Core automatically selects the optimal download strategy:

Single-Threaded Mode

Used when:

• Server doesn't support byte ranges
• File size < 2MB
• Connections set to 1

[=====================================] 100%
         Single Stream

Multi-Threaded Mode

Used when:

• Server supports Accept-Ranges: bytes
• File size > 2MB
• Connections > 1

Chunk 1: [========] 100%
Chunk 2: [======  ]  75%
Chunk 3: [====    ]  50%
Chunk 4: [==      ]  25%
         ↓
    Merged File

File Structure

During download, Xydra.Core creates:

Downloads/
├── filename.ext           # Final file (or partial for multi-threaded)
├── filename.ext.fdmtemp   # Temp file (single-threaded mode)
└── filename.ext.fdmstate  # State file (JSON, multi-threaded mode)

State File Format

{
  "TotalSize": 104857600,
  "ETag": "\"abc123\"",
  "Chunks": [
    { "Start": 0, "End": 26214399, "Downloaded": 26214400 },
    { "Start": 26214400, "End": 52428799, "Downloaded": 15000000 },
    { "Start": 52428800, "End": 78643199, "Downloaded": 0 },
    { "Start": 78643200, "End": 104857599, "Downloaded": 0 }
  ]
}

Configuration

Connection Count

The default is 8 connections. Optimal values depend on:

Scenario	Recommended
Fast server, high bandwidth	8-16
Slow server	2-4
Many simultaneous downloads	2-4 per download
Single large file	8-16

var download = await manager.AddDownloadAsync(url, connections: 16);

Buffer Size

The internal buffer is 1MB by default. This is optimal for most scenarios and cannot be changed via the public API.

Error Handling

Link Expiration

Xydra.Core detects expired links via HTTP status codes:

• 401 Unauthorized
• 403 Forbidden
• 404 Not Found
• 410 Gone

if (item.Status == DownloadStatus.LinkExpired)
{
    // Prompt user for new URL
    await manager.UpdateLinkAsync(item.Id, newUrl);
}

Network Errors

Failed chunks are automatically retried up to 3 times with the download resuming from the last successful position.

Thread Safety

All public methods are thread-safe:

• DownloadManager uses locks for collection access
• DownloadEngine uses ConcurrentDictionary for active downloads
• Progress updates use Interlocked operations
• File writes use explicit locking

Performance Tips

1. Use appropriate connection count - More isn't always better
2. Let the engine choose - Don't force single-threaded mode
3. Handle events efficiently - Don't block in event handlers
4. Dispose properly - Use using statements or call Dispose()

License

MIT License - See LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.