Date: 2023-06-10 Participants: 5 users Goal: Analyze coding patterns and trajectories
- Used OpenAI GPT-4 for code analysis
- Collected code snapshots every 5 minutes
- Analyzed complexity, style, and patterns
| User ID | Snapshots | Avg. Complexity | Notable Patterns |
|---|---|---|---|
| 20d03f52 | 15 | Medium | Frequent refactoring |
| 31e8f7a9 | 12 | Low | Linear progression |
| 47c9b2d5 | 18 | High | Exploratory coding |
| 58a6f4e1 | 10 | Medium | Test-driven approach |
| 69d7c3b8 | 14 | Medium | Incremental changes |
- Users with test-driven approaches showed more consistent complexity metrics
- Exploratory coders had higher variance in complexity
- Refactoring patterns correlated with improved code quality over time
- Implement more detailed analysis of function-level changes
- Add visualization of code evolution over time
- Collect more data on user intent through surveys
Date: 2023-06-20 Participants: Same 5 users Goal: Test ToM-based code understanding
- Enhanced analyzer with ToM capabilities
- Added intent prediction module
- Collected user feedback on predictions
| User ID | Intent Accuracy | Helpful Suggestions | User Rating |
|---|---|---|---|
| 20d03f52 | 78% | 8/10 | 4.2/5 |
| 31e8f7a9 | 65% | 6/10 | 3.8/5 |
| 47c9b2d5 | 82% | 9/10 | 4.5/5 |
| 58a6f4e1 | 75% | 7/10 | 4.0/5 |
| 69d7c3b8 | 70% | 7/10 | 3.9/5 |
- ToM model was most accurate for users with consistent coding patterns
- Suggestions were rated more helpful when aligned with user's actual intent
- Users reported feeling "understood" when intent predictions were accurate
- Refine ToM model with more training data
- Implement real-time suggestions based on predicted intent
- Explore personalization of the model to individual coding styles