Feature/refactor event system

CHANGELOG.md

@@ -0,0 +1,162 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.3.0] - 2024
+
+### ⚠️ BREAKING CHANGES
+
+This release introduces a complete redesign of the event system with a unified, consistent pattern. **This is a breaking change** that requires migration for existing code.
+
+#### Event System Redesign
+
+**Old Pattern (v0.2.x)**:
+- 19 specific event types (`WorkflowStarted`, `AgentProcessing`, `ToolCallStarted`, etc.)
+- Inconsistent naming and lifecycle patterns
+- Limited extensibility
+
+**New Pattern (v0.3.0)**:
+- Unified `Scope × Type × Status` model
+- 6 event scopes: `Workflow`, `WorkflowStep`, `Agent`, `LlmRequest`, `Tool`, `System`
+- 5 standard lifecycle types: `Started`, `Progress`, `Completed`, `Failed`, `Canceled`
+- 5 component statuses: `Pending`, `Running`, `Completed`, `Failed`, `Canceled`
+
+### Added
+
+- **Event Scopes** (`EventScope` enum):
+  - `Workflow` - Entire workflow execution
+  - `WorkflowStep` - Individual workflow steps
+  - `Agent` - Agent execution
+  - `LlmRequest` - LLM/ChatClient requests (renamed from "Agent LLM")
+  - `Tool` - Tool execution
+  - `System` - System-level events
+
+- **Unified Event Types** (`EventType` enum):
+  - `Started` - Component begins execution
+  - `Progress` - Component reports progress/streaming data
+  - `Completed` - Component finished successfully
+  - `Failed` - Component encountered error
+  - **`Canceled`** - Component was canceled/interrupted (new!)
+
+- **Component Status Tracking** (`ComponentStatus` enum):
+  - Explicit status field on all events
+  - Tracks component state after event
+
+- **Component ID Validation**:
+  - Enforced format standards per scope
+  - `Workflow`: `workflow_name`
+  - `WorkflowStep`: `workflow_name:step:N`
+  - `Agent`: `agent_name`
+  - `LlmRequest`: `agent_name:llm:N`
+  - `Tool`: `tool_name`
+  - `System`: `system:subsystem`
+
+- **Helper Methods** on `EventStream`:
+  - `agent_started()`, `agent_completed()`, `agent_failed()`
+  - `llm_started()`, `llm_progress()`, `llm_completed()`, `llm_failed()`
+  - `tool_started()`, `tool_progress()`, `tool_completed()`, `tool_failed()`
+  - `workflow_started()`, `workflow_completed()`, `workflow_failed()`
+  - `step_started()`, `step_completed()`, `step_failed()`
+
+- **Event Fields**:
+  - `component_id` - Standardized component identifier
+  - `scope` - Event scope (which component type)
+  - `status` - Current component status
+  - `message` - Optional human-readable message
+
+### Changed
+
+- **Event struct**:
+  - Added: `scope`, `event_type`, `component_id`, `status`, `message`
+  - Renamed: `event_type` field is now the lifecycle stage, not the specific event
+
+- **EventStream::append() signature**:
+  ```rust
+  // Old
+  fn append(&self, event_type: EventType, workflow_id: String, data: JsonValue)
+
+  // New
+  fn append(
+      &self, 
+      scope: EventScope, 
+      event_type: EventType,
+      component_id: String,
+      status: ComponentStatus,
+      workflow_id: String,
+      message: Option<String>,
+      data: JsonValue
+  )
+  ```
+
+- **Tool loop detection** now emits `System::Progress` events instead of `AgentToolLoopDetected`
+
+- Version bumped from `0.1.0` → `0.3.0` (skip 0.2.0 due to breaking nature)
+
+### Removed
+
+- **Old EventType variants**:
+  - `WorkflowStarted`, `WorkflowStepStarted`, `WorkflowStepCompleted`, `WorkflowCompleted`, `WorkflowFailed`
+  - `AgentInitialized`, `AgentProcessing`, `AgentCompleted`, `AgentFailed`
+  - `AgentLlmRequestStarted`, `AgentLlmStreamChunk`, `AgentLlmRequestCompleted`, `AgentLlmRequestFailed`
+  - `ToolCallStarted`, `ToolCallCompleted`, `ToolCallFailed`, `AgentToolLoopDetected`
+  - `SystemError`, `StateSaved`
+
+### Migration Guide
+
+See [docs/MIGRATION_0.2_TO_0.3.md](docs/MIGRATION_0.2_TO_0.3.md) for detailed migration instructions.
+
+**Quick migration examples**:
+
+```rust
+// Old: Pattern matching on specific event types
+match event.event_type {
+    EventType::WorkflowStarted => { ... }
+    EventType::ToolCallCompleted => { ... }
+}
+
+// New: Pattern matching on scope + type
+match (event.scope, event.event_type) {
+    (EventScope::Workflow, EventType::Started) => { ... }
+    (EventScope::Tool, EventType::Completed) => { ... }
+}
+
+// Old: Manual event emission
+stream.append(
+    EventType::ToolCallStarted,
+    "wf_1",
+    json!({"tool": "my_tool"})
+);
+
+// New: Use helper methods
+stream.tool_started(
+    "my_tool",
+    "wf_1".to_string(),
+    json!({})
+);
+```
+
+### Technical Details
+
+- All events now emit asynchronously via spawned tasks (from v0.2.x)
+- Event validation ensures component IDs follow standardized formats
+- Helper methods provide type-safe, ergonomic API for common event patterns
+- Raw `append()` method still available for custom event scenarios
+
+### Tests
+
+- 97 tests passing (63 lib + 7 chat_history + 12 error + 3 integration + 12 load)
+- All clippy warnings resolved
+- All doctests passing
+
+---
+
+## [0.2.0] - 2024 (Skipped)
+
+Skipped to avoid confusion with pre-release versions.
+
+## [0.1.0] - 2024
+
+Initial release with basic agent runtime, workflow system, and event streaming.

Cargo.toml

@@ -4,7 +4,7 @@ members = [
 ]
 
 [workspace.package]
-version = "0.1.0"
+version = "0.3.0"
 edition = "2021"
 authors = ["Travis Sharp <travis@kuipersys.com>"]
 license = "MIT OR Apache-2.0"
@@ -19,7 +19,7 @@ all = "warn"
 # Root package - the main MCP library
 [package]
 name = "agent-runtime"
-version = "0.1.0"
+version = "0.3.0"
 edition = "2021"
 authors = ["Travis Sharp <travis@kuipersys.com>"]
 license = "MIT OR Apache-2.0"

README.md

@@ -25,12 +25,12 @@ A production-ready Rust framework for building AI agent workflows with native an
 - **Nested workflows** - SubWorkflows for complex orchestration
 - **Mermaid export** - Visualize workflows as diagrams
 
-### 📡 Event System
-- **Real-time events** - Complete visibility into execution
-- **Fine-grained tracking** - Workflow, agent, LLM, and tool events
-- **Streaming chunks** - Live LLM token streaming via events
+### 📡 Event System (v0.3.0 - Unified)
+- **Unified event model** - Consistent `Scope × Type × Status` pattern across all components
+- **Complete lifecycle tracking** - Started → Progress → Completed/Failed for workflows, agents, LLM requests, tools
+- **Real-time streaming** - Live LLM token streaming via Progress events
 - **Multi-subscriber** - Multiple event listeners per workflow
-- **Event bubbling** - Events propagate from tools → agents → workflows
+- **Type-safe component IDs** - Enforced formats with validation
 
 ### ⚙️ Configuration
 - **YAML and TOML support** - Human-readable config files
@@ -39,11 +39,11 @@ A production-ready Rust framework for building AI agent workflows with native an
 - **Per-agent settings** - System prompts, tools, LLM clients, loop prevention
 
 ### 🔒 Production Ready
-- **61 comprehensive tests** - All core functionality tested
-- **Tool loop prevention** - Prevents LLM from calling same tool repeatedly
-- **Microsecond timing** - Precise performance metrics
-- **Structured logging** - FileLogger with timestamped output
-- **Error handling** - Detailed error types with context
+- **97 comprehensive tests** - All core functionality tested
+- **Tool loop prevention** - Prevents LLM from calling same tool repeatedly with System::Progress events
+- **Microsecond timing** - Precise performance metrics via event data
+- **Async event emission** - Non-blocking event streaming with tokio::spawn
+- **Error handling** - Detailed error types with context and human-readable messages
 
 ## Quick Start
 
@@ -111,23 +111,36 @@ let workflow = Workflow::new("analysis")
 let result = workflow.execute(initial_input, &mut event_rx).await?;
 ```
 
-### Event Streaming
+### Event Streaming (v0.3.0)
 ```rust
+use agent_runtime::{EventScope, EventType};
+
 let (tx, mut rx) = mpsc::channel(100);
 
 // Subscribe to events
 tokio::spawn(async move {
     while let Some(event) = rx.recv().await {
-        match event.event_type {
-            EventType::AgentLlmStreamChunk => {
-                print!("{}", event.data.get("chunk").unwrap());
+        match (event.scope, event.event_type) {
+            // Stream LLM responses in real-time
+            (EventScope::LlmRequest, EventType::Progress) => {
+                if let Some(chunk) = event.data["chunk"].as_str() {
+                    print!("{}", chunk);
+                }
             }
-            EventType::ToolCallCompleted => {
-                println!("Tool {} returned: {}", 
-                    event.data["tool_name"],
+            // Track tool executions
+            (EventScope::Tool, EventType::Completed) => {
+                println!("✓ Tool {} returned: {}", 
+                    event.component_id,
                     event.data["result"]
                 );
             }
+            // Handle failures
+            (_, EventType::Failed) => {
+                eprintln!("❌ {}: {}",
+                    event.component_id,
+                    event.message.unwrap_or_default()
+                );
+            }
             _ => {}
         }
     }
@@ -171,23 +184,42 @@ let config = RuntimeConfig::from_file("agent-runtime.yaml")?;
 - **`config`** - YAML/TOML configuration loading
 - **`tool_loop_detection`** - Intelligent duplicate tool call prevention
 
-### Event Types
-- **Workflow**: Started, StepStarted, StepCompleted, StepFailed, Completed, Failed
-- **Agent**: Started, Completed, Failed, LlmStreamChunk
-- **LLM**: RequestSent, ResponseReceived, StreamChunkReceived
-- **Tool**: ToolCallStarted, ToolCallCompleted, ToolCallFailed, AgentToolLoopDetected
+### Event System (v0.3.0)
+**Unified Scope × Type × Status Pattern:**
+- **Scopes**: Workflow, WorkflowStep, Agent, LlmRequest, Tool, System
+- **Types**: Started, Progress, Completed, Failed, Canceled
+- **Status**: Pending, Running, Completed, Failed, Canceled
+
+**Key Events:**
+- `Workflow::Started/Completed/Failed` - Overall workflow execution
+- `WorkflowStep::Started/Completed/Failed` - Individual step tracking
+- `Agent::Started/Completed/Failed` - Agent processing lifecycle
+- `LlmRequest::Started/Progress/Completed/Failed` - Real-time LLM streaming
+- `Tool::Started/Progress/Completed/Failed` - Tool execution tracking
+- `System::Progress` - Runtime behaviors (e.g., tool loop detection)
+
+**Component ID Formats:**
+- Workflow: `workflow_name`
+- WorkflowStep: `workflow:step:N`
+- Agent: `agent_name`
+- LlmRequest: `agent:llm:N`
+- Tool: `tool_name` or `tool_name:N`
+- System: `system:subsystem`
 
 ### Tool Loop Prevention
 Prevents LLMs from calling the same tool with identical arguments repeatedly:
 - **Automatic detection** - Tracks tool calls and arguments using MD5 hashing
+- **System events** - Emits `System::Progress` event with `system:tool_loop_detection` component ID
 - **Configurable messages** - Custom messages with `{tool_name}` and `{previous_result}` placeholders
-- **Event emission** - `AgentToolLoopDetected` event for observability
 - **Enabled by default** - Can be disabled per-agent if needed
 
 ## Examples
 
 Run any demo:
 ```bash
+# Event System
+cargo run --bin async_events_demo    # NEW! Async event streaming demo with visible sequence
+
 # Workflows
 cargo run --bin workflow_demo          # 3-agent workflow with LLM
 cargo run --bin hello_workflow         # Simple sequential workflow
@@ -212,23 +244,52 @@ cargo run --bin complex_viz            # Complex workflow diagram
 
 ## Documentation
 
+- **[Event Streaming Guide](docs/EVENT_STREAMING.md)** - Complete event system documentation (v0.3.0)
+- **[Migration Guide](docs/MIGRATION_0.2_TO_0.3.md)** - Upgrading from v0.2.x to v0.3.0
+- **[Changelog](CHANGELOG.md)** - Release notes for v0.3.0
 - **[Specification](docs/spec.md)** - Complete system design
 - **[Tool Calling](docs/TOOL_CALLING.md)** - Native tool usage
 - **[MCP Integration](docs/MCP_INTEGRATION.md)** - External MCP tools
-- **[Event Streaming](docs/EVENT_STREAMING.md)** - Event system guide
 - **[LLM Module](docs/LLM_MODULE.md)** - LLM provider integration
 - **[Workflow Composition](docs/WORKFLOW_COMPOSITION.md)** - Building workflows
 - **[Testing](docs/TESTING.md)** - Test suite documentation
 
 ## Testing
 
 ```bash
-cargo test              # All 61 tests
+cargo test              # All 97 tests
 cargo test --lib        # Library tests only
 cargo test agent        # Agent tests
 cargo test tool         # Tool tests
+cargo test event        # Event system tests
 cargo clippy            # Linting
+cargo fmt --all         # Format code
+```
+
+## What's New in v0.3.0
+
+**🎉 Unified Event System** - Complete rewrite for consistency and extensibility
+
+- **Breaking Changes**: New event structure with `EventScope`, `ComponentStatus`, unified `EventType`
+- **Helper Methods**: 19 ergonomic helper methods for common event patterns
+- **Component IDs**: Enforced formats with validation for type safety
+- **Async Events**: Non-blocking event emission via `tokio::spawn()`
+- **Migration Guide**: See [docs/MIGRATION_0.2_TO_0.3.md](docs/MIGRATION_0.2_TO_0.3.md)
+
+**Upgrading from v0.2.x?**
+```rust
+// Old (v0.2.x)
+match event.event_type {
+    EventType::AgentLlmStreamChunk => { ... }
+}
+
+// New (v0.3.0)
+match (event.scope, event.event_type) {
+    (EventScope::LlmRequest, EventType::Progress) => { ... }
+}
 ```
 
+See [CHANGELOG.md](CHANGELOG.md) for complete details.
+
 ## License
 Dual-licensed under MIT or Apache-2.0 at your option.

benches/agent_benchmarks.rs

@@ -1,5 +1,5 @@
 use agent_runtime::prelude::*;
-use agent_runtime::{Agent, AgentConfig, AgentInput, Event, EventType, NativeTool, ToolRegistry};
+use agent_runtime::{Agent, AgentConfig, AgentInput, Event, NativeTool, ToolRegistry};
 use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion};
 use serde_json::json;
 use std::sync::Arc;
@@ -185,10 +185,15 @@ fn bench_event_emission(c: &mut Criterion) {
             // Emit event
             let event = Event::new(
                 0,
-                EventType::AgentProcessing,
+                agent_runtime::EventScope::Agent,
+                agent_runtime::EventType::Started,
+                "bench_agent".to_string(),
+                agent_runtime::ComponentStatus::Running,
                 "bench_workflow".to_string(),
-                json!({"agent": "bench_agent", "step": 0}),
-            );
+                None,
+                json!({"step": 0}),
+            )
+            .unwrap();
 
             let _ = tx.send(black_box(event));