Proximity Forest: Time Series Classification Benchmark

This repository contains the replication and visualization of benchmark results for the Proximity Forest algorithm, tested against the standard UCR Time Series Archive, as well as a Python implementation testing the model on a diverse subset of datasets.

Project Overview

Time series classification is a challenging problem in machine learning. While algorithms like HIVE-COTE have shown high accuracy, they often lack scalability. Proximity Forest addresses this by using an ensemble of decision trees that split data based on various distance metrics (such as DTW, ERP, LCSS), making it both highly accurate and computationally scalable.

This project focuses on two main objectives:

1. Model Evaluation & Visualization: Generating Critical Difference Diagrams to statistically compare Proximity Forest against other state-of-the-art classifiers (like DTW, EE, BOSS, ST, and COTE) using average ranks.
2. Implementation & Testing: Implementing the algorithm in Python and training it across diverse domains (Medical, Seismic, and Motion) to evaluate its real-world applicability.

References

This project is based on the research presented in the following paper:

Lucas, B., Shifaz, A., Pelletier, C., O’Neill, L., Zaidi, N., Goethals, B., ... & Webb, G. I. (2019). Proximity Forest: an effective and scalable distance-based classifier for time series. Data Mining and Knowledge Discovery, 33(3), 607-635. Original GitHub Repository by the Authors

Repository Structure

Proximity_tree_Model/
├── data/
│   └── UCR Archive 2018.zip                  # The standard benchmark dataset
│   └── ProximityForest_r1.csv                # Evaluation metrics (run 1) - see note below
│   └── ProximityForest_r2.csv                # Evaluation metrics (run 2) - see note below
│   └── ProximityForest_r5.csv                # Evaluation metrics (run 5) - see note below
├── src/
│   └── train_multiple_datasets.py            # Python model training script
├── proximity_forest_benchmark_analysis.ipynb # Core analysis and plotting
├── docs/
│   └── Proximity_tree_DWDM_ProjectFile.pdf   # Detailed Project Report
├── results/                                  # Generated plots and evaluation metrics
├── requirements.txt                          # Python dependencies
└── README.md                                 # Project documentation

Note on Data Files: The ProximityForest_r*.csv files contain the evaluation metrics required to run the visualization notebook. These are typically generated by running the original Proximity Forest Java implementation against the UCR archive, or they can be downloaded from standard benchmark repositories (like timeseriesclassification.com). If these files are not present locally, the notebook serves as a read-only view of the generated statistical graphs.

Setup & Installation

1. Clone the repository:

   git clone <your-repository-url>
   cd Proximity_tree_Model

2. Install the required dependencies:

   pip install -r requirements.txt

Training the Model

To demonstrate the versatility of the Proximity Forest algorithm, a Python script is provided to train and evaluate the model across multiple diverse UCR datasets:

• ECG200: Medical (Electrocardiogram data)
• Earthquakes: Seismic data
• GunPoint: Motion capture data

Run the training script via the command line:

python src/train_multiple_datasets.py

This script will automatically load the specified datasets, perform training using a constrained number of trees for efficiency, and output the performance metrics (Accuracy and Classification Report) into the results/model_evaluation.txt file.

Note on Data Cleaning: The script utilizes standard UCR benchmark datasets, which are strictly pre-processed and z-normalized to allow objective comparison between algorithms. In a non-benchmark production setting, preliminary steps such as missing value imputation and noise smoothing would be required prior to inference.

Generating Benchmark Visualizations

To view the benchmark results and regenerate the plots:

1. Ensure you are in the project root directory.
2. Launch Jupyter Notebook:

   jupyter notebook proximity_forest_benchmark_analysis.ipynb

3. Execute the cells to process the evaluation metrics. The output Critical Difference Diagrams will be saved to the results/ folder.

Documentation

For a comprehensive review of the methodology, algorithms utilized, and extended analytical results, please consult the formal project report located at docs/Proximity_tree_DWDM_ProjectFile.pdf.