micrograd-cpp

A modern C++20 implementation of micrograd, a tiny autograd engine and neural network library. This project is a C++ port of Andrej Karpathy's micrograd, leveraging modern C++ features like Concepts, CRTP, and Policy-based design for high performance and type safety.

Architecture

The project is built on a clean, policy-based design that leverages C++ templates to achieve flexibility without runtime overhead.

Key Components

• Value<T>: The fundamental building block of the autograd engine. It wraps a scalar value and tracks its gradient and the operation that produced it, forming a dynamic computational graph.
• Module<Derived, T>: A CRTP base class that provides a common interface for all neural network components (Neuron, Layer, MLP). It handles parameter access and gradient zeroing.
• Optimizer<Derived, T>: A CRTP base class for optimization algorithms (like SGD).
• Policies:
  • Activation: ReLU, Tanh, Linear.
  • Initialization: He, RandomUniform.

Results

Below is a visualization of the decision boundary learned by a 2-layer MLP on the "Two Moons" dataset.

Features

• Scalar Autograd Engine: Implements a computational graph with automatic backpropagation.
• Neural Network Library:
  • Support for Neuron, Layer, and MLP modules.
  • Policy-based Design: Choose activations (ReLU, Tanh, Linear) and initialization strategies (He, RandomUniform) at compile time.
  • Modern C++ Architecture: Uses CRTP (Curiously Recurring Template Pattern) instead of virtual functions for zero-overhead polymorphism.
• Generic Training: A flexible train() function that works with any Module and Optimizer.
• End-to-End Pipeline: Includes Python scripts for dataset generation and decision boundary visualization.
• Dockerized Environment: Fully portable build environment using Docker.
• Testing: Comprehensive unit tests powered by doctest.

Getting Started

The project uses a Docker-based workflow to ensure a consistent build environment (GCC 12+).

Prerequisites

• Docker
• Make

Build and Run Instructions

1. Clone the Repository
   Clone the project along with its submodules:

   git clone --recursive https://github.com/kornelhowil/micrograd-cpp.git
   cd micrograd-cpp

2. Prepare the Docker Environment
   Build the base image containing all necessary dependencies (GCC, CMake, python3, Matplotlib):

   make build-docker-deps-image

3. Compile the Project
   Build the engine, neural network library, example application, and tests:

   make build

4. Run the Full Pipeline
   This command automatically:

   1. Generates the "Two Moons" dataset using Python.
   2. Trains the C++ MLP model on this data.
   3. Visualizes the decision boundary and saves it to decision_boundary.png.

   make run

5. Run Tests
   Execute the unit test suite:

   make test

Project Structure

• micrograd-cpp/: Core library headers.
  • engine.hpp: The autograd engine (Value class).
  • nn.hpp: Neural network components (Neuron, Layer, MLP) and optimizers.
• scripts/: Python scripts for data generation and visualization.
• tests/: Unit tests for engine logic and neural network layers.
• main.cpp: Entry point demonstrating library usage (Two Moons classification).
• libs/: External dependencies.

Example Usage

#include "nn.hpp"
using namespace micrograd;

int main() {
    // 1. Load Data (example using std vectors)
    std::vector<std::vector<double>> X = {{2.0, 3.0, -1.0}, {3.0, -1.0, 0.5}};
    std::vector<std::vector<double>> y = {{1.0}, {-1.0}};

    // 2. Initialize Model
    // 3 inputs, two hidden layers of 4 neurons each (ReLU), 1 output (Tanh)
    MLP<double, ReLUActivation<double>, TanhActivation<double>, HeInit<double>> model(3, {4, 4, 1});
    
    // 3. Setup Optimizer
    SGD<decltype(model), double> optimizer(model, 0.01);

    // 4. Train
    train(model, optimizer, X, y, 100);

    // 5. Predict
    auto score = model({2.0, 3.0, -1.0});
    std::cout << "Prediction: " << score[0]->data << "\n";
    
    return 0;
}