Feature/clusterdynamics ml saxs

docs/api/overview.md

@@ -14,6 +14,10 @@ classes that are most likely to be imported directly.
 
 - `saxshell.cluster.workflow.ClusterWorkflow`
 
+### Cluster dynamics
+
+- `saxshell.clusterdynamics.workflow.ClusterDynamicsWorkflow`
+
 ### Bond analysis
 
 - `saxshell.bondanalysis.workflow.BondAnalysisWorkflow`

docs/development/repo-structure.md

@@ -8,6 +8,7 @@ This page is a short orientation guide for contributors.
 src/saxshell/
   bondanalysis/
   cluster/
+  clusterdynamics/
   fullrmc/
   mdtrajectory/
   saxs/
@@ -32,6 +33,11 @@ Residue-aware XYZ-to-PDB conversion, reference-library helpers, and UI code.
 
 Cluster extraction workflows and UI.
 
+### `src/saxshell/clusterdynamics`
+
+Time-binned cluster-distribution analysis, lifetime/rate summaries, dataset
+save/load helpers, and the matching UI.
+
 ### `src/saxshell/bondanalysis`
 
 Bond-pair and angle-analysis workflows and UI.

docs/getting-started/installation.md

@@ -48,6 +48,7 @@ The current package exposes these top-level tools:
 - `saxshell`
 - `mdtrajectory`
 - `clusters`
+- `clusterdynamics`
 - `bondanalysis`
 - `xyz2pdb`
 - `saxs`

docs/index.md

@@ -34,10 +34,11 @@ The current repo supports an end-to-end path that usually looks like this:
 1. Inspect and split trajectories with `mdtrajectory`.
 2. Optionally convert XYZ frames to residue-aware PDB files with `xyz2pdb`.
 3. Extract clusters with `clusters`.
-4. Measure bond and angle distributions with `bondanalysis`.
-5. Build a SAXS project with `saxs`.
-6. Refine the project in **SAXS Prefit** and, if needed, run **pyDREAM**.
-7. Use the resulting distributions and selected structures in downstream tools
+4. Analyze time-dependent cluster populations with `clusterdynamics`.
+5. Measure bond and angle distributions with `bondanalysis`.
+6. Build a SAXS project with `saxs`.
+7. Refine the project in **SAXS Prefit** and, if needed, run **pyDREAM**.
+8. Use the resulting distributions and selected structures in downstream tools
    such as `fullrmc`.
 
 ## Documentation map

docs/javascripts/mathjax.js

@@ -0,0 +1,15 @@
+window.MathJax = {
+  tex: {
+    inlineMath: [["\\(", "\\)"]],
+    displayMath: [["\\[", "\\]"]],
+    processEscapes: true,
+    processEnvironments: true,
+  },
+  options: {
+    skipHtmlTags: ["script", "noscript", "style", "textarea", "pre"],
+  },
+};
+
+document$.subscribe(() => {
+  MathJax.typesetPromise?.();
+});

docs/user-guide/cluster-dynamics.md

@@ -0,0 +1,175 @@
+# Cluster Dynamics
+
+`clusterdynamics` is the time-resolved companion to `clusters`. It analyzes the
+same extracted XYZ or PDB frame folders from `mdtrajectory`, reuses the same
+cluster-definition and pair-cutoff logic, and converts the per-frame cluster
+counts into:
+
+- a time-binned cluster-distribution heatmap
+- an optional lower subplot from a CP2K `.ener` file
+- a sortable lifetime table by stoichiometry label
+- reloadable JSON/CSV datasets for later plotting
+
+## Inputs
+
+The application expects:
+
+- an extracted frame folder produced by `mdtrajectory`
+- optional CP2K `.ener` data for the lower overlay subplot
+- the same atom-type definitions, pair cutoffs, shell options, and PBC/search
+  settings you would use in `clusters`
+
+The left pane includes all of the cluster-rule controls from the cluster
+extraction UI, plus the time-axis controls and dataset save/load actions.
+
+## Time Axis Rules
+
+The time axis is resolved in this order:
+
+1. `mdtrajectory_export.json` if the frame folder came from a recent
+   `mdtrajectory` export. This is the most reliable source because it stores
+   the original frame indices and frame times written during export.
+2. `frame_<index>.xyz` or `frame_<index>.pdb` filenames multiplied by the
+   user-specified frame timestep.
+3. A sequential fallback that starts from the folder/start-time field.
+
+The frame timestep field defaults to `0.5 fs`.
+
+The heatmap binning now uses an integer `frames / colormap timestep` control.
+The UI shows the derived `colormap timestep used (fs)` field so the effective
+heatmap timestep is always an exact multiple of the frame timestep.
+
+`mdtrajectory` cutoff exports now use folder names such as `splitxyz_f847fs`.
+The `f847fs` part records the cutoff time in femtoseconds and is auto-filled
+into the folder/start-time field when it is detected.
+
+### Important example
+
+If the folder is `splitxyz_f847fs`, the frame timestep is `0.5 fs`, and the
+first file is `frame_1866.xyz`, the resolved frame time is:
+
+```text
+1866 x 0.5 fs = 933 fs
+```
+
+In that case the preview warns that the folder cutoff/start time (`847 fs`) and
+the first resolved extracted-frame time (`933 fs`) are different. The heatmap
+and lifetime calculations follow the resolved frame times, not the folder label.
+
+## Typical UI Workflow
+
+1. Load the extracted XYZ/PDB frames folder.
+2. Optionally load a CP2K `.ener` file.
+3. Confirm the active SAXSShell project if you want saved datasets to default
+   into `exported_results/data/clusterdynamics`.
+4. Enter the cluster definitions, pair cutoffs, shell options, and PBC/search
+   settings.
+5. Confirm the frame timestep, frames per colormap timestep, derived
+   colormap timestep, and analysis start/stop window.
+6. Run the analysis.
+7. Adjust the heatmap display mode, time units, colormap, quantile limits, and
+   optional overlay interactively.
+8. Use the **Saved Results** panel to save the current result or reopen a
+   previously saved dataset without rerunning the frame analysis.
+
+If the tool is launched from the main SAXS UI, it inherits the active project
+directory automatically.
+
+## Saved Outputs
+
+The save action writes a JSON dataset plus companion CSV files beside it:
+
+- `*_cluster_distribution.csv`
+- `*_lifetime.csv`
+- `*_energy.csv` when an energy overlay is present
+
+The JSON file is the reloadable artifact. It stores the plotted matrices,
+summary tables, time-axis metadata, and optional energy data needed to reopen
+the analysis. In the UI, these controls live in the **Saved Results** panel so
+they are separate from the **Run Analysis** controls.
+
+## Heatmap Data
+
+Each heatmap row is a stoichiometry label, and the full y-axis spans the labels
+observed across all time bins in the current analysis window.
+
+The display mode can be switched interactively between:
+
+- raw counts per bin
+- fraction of all clusters in the bin
+- mean count per sampled frame in the bin
+
+The color scaling uses quantile limits instead of fixed min/max values so large
+outliers do not flatten the rest of the heatmap.
+
+## Association, Dissociation, and Lifetimes
+
+The lifetime table is computed from the per-label count series over consecutive
+sampled frames.
+
+- Association events: every positive increase in the count for a label between
+  two adjacent sampled frames.
+- Dissociation events: every negative decrease in that count between adjacent
+  sampled frames.
+- Association rate: `association_events / observation_window_ps`
+- Dissociation rate: `dissociation_events / observation_window_ps`
+
+This is a count-based kinetic summary. It does not claim atom-by-atom identity
+tracking across frames. Instead, it reports how the occupancy of each
+stoichiometry label changes between samples.
+
+### Lifetime definition
+
+- Completed lifetime: a cluster instance that appears after the first sampled
+  frame and disappears before the end of the observation window.
+- Window-truncated lifetime: a cluster instance that was already present in the
+  first sampled frame or was still present in the last sampled frame.
+
+Mean and standard deviation lifetimes are computed from completed lifetimes
+only. The window-truncated count is reported separately so you can see how much
+of the series is clipped by the analysis boundaries.
+
+## Lifetime Table Columns
+
+- `Label`: stoichiometry label, such as `Pb2I`
+- `Size`: total number of atoms represented by the stoichiometry label
+- `Mean lifetime (fs)`: average duration of completed lifetimes for that label
+- `Std lifetime (fs)`: standard deviation of completed lifetimes
+- `Completed`: number of completed lifetimes used in the mean/std calculation
+- `Window-truncated`: number of lifetimes clipped by the start or end of the
+  sampled window
+- `Assoc. rate (1/ps)`: positive count changes per picosecond over the selected
+  analysis window
+- `Dissoc. rate (1/ps)`: negative count changes per picosecond over the
+  selected analysis window
+- `Occupancy (%)`: fraction of sampled frames in which at least one cluster of
+  that label was present
+- `Mean count/frame`: average number of clusters with that label per sampled
+  frame
+
+Sort the `Lifetime` tab by the `Size` column if you want the older “lifetime by
+size” view without a separate tab.
+
+## Cluster-Distribution CSV Columns
+
+The saved `*_cluster_distribution.csv` file stores one row per
+`label x time-bin` combination with:
+
+- the label and cluster size
+- bin index, start, stop, and center in femtoseconds
+- raw count in the bin
+- fraction in the bin
+- mean count per sampled frame in the bin
+- number of sampled frames in that bin
+- total clusters in that bin
+
+## When Warnings Appear
+
+The preview and summary can include warnings when:
+
+- the folder/start-time tag such as `_f847fs` was not found
+- `mdtrajectory_export.json` is missing or incomplete
+- the folder/start-time tag and the resolved frame times disagree
+
+These warnings are informational. The analysis still runs, but the UI makes the
+time basis explicit so the heatmap and lifetime interpretation stay transparent.

docs/user-guide/cluster-extraction.md

@@ -8,8 +8,10 @@ In this repository, that bridge spans more than one tool.
 1. Use `mdtrajectory` to inspect a trajectory and export frames.
 2. Optionally use `xyz2pdb` if you need molecule-aware PDB frames.
 3. Use `clusters` to extract stoichiometry-sorted cluster files.
-4. Use `bondanalysis` to measure bond or angle distributions on those clusters.
-5. Feed the resulting cluster folder into the SAXS project.
+4. Use `clusterdynamics` to build time-dependent cluster-distribution heatmaps
+   and lifetime tables from the extracted frames.
+5. Use `bondanalysis` to measure bond or angle distributions on those clusters.
+6. Feed the resulting cluster folder into the SAXS project.
 
 ## `mdtrajectory`
 
@@ -19,6 +21,8 @@ This tool is responsible for:
 - optionally reading CP2K `.ener` files
 - suggesting a cutoff
 - exporting selected frames into a sibling folder
+- writing `mdtrajectory_export.json` metadata beside the exported frames so
+  downstream tools can recover the original frame indices and times
 
 Example:
 
@@ -28,6 +32,10 @@ mdtrajectory suggest-cutoff traj.xyz --energy-file traj.ener --temp-target-k 300
 mdtrajectory export traj.xyz --energy-file traj.ener --use-suggested-cutoff --temp-target-k 300 --window 3
 ```
 
+When a cutoff is applied, the default folder name now uses the form
+`splitxyz_f847fs` or `splitpdb_f847fs`, where the `f847fs` portion records the
+cutoff time in femtoseconds.
+
 ## `xyz2pdb`
 
 Use this only when residue identity matters downstream.
@@ -54,6 +62,23 @@ The cluster workflow supports both UI and CLI usage. Its CLI exposes separate
 The CLI help text explicitly calls out faster neighbor search modes such as
 `kdtree` and `vectorized`.
 
+## `clusterdynamics`
+
+This application consumes the extracted XYZ or PDB frames from `mdtrajectory`
+and applies the same cluster definitions and pair-cutoff rules used by
+`clusters`, but bins the results over time instead of writing one
+stoichiometry-folder export.
+
+Key outputs:
+
+- time-binned cluster-distribution heatmaps
+- optional CP2K `.ener` overlays aligned to the same time axis
+- a sortable lifetime table by stoichiometry label
+- saved JSON/CSV datasets that can be reopened later for plotting
+
+See [Cluster Dynamics](cluster-dynamics.md) for the full workflow, timing
+rules, and the definitions of the lifetime/rate columns.
+
 ## `bondanalysis`
 
 Bond analysis is downstream of cluster extraction. Use it to derive bond-pair

docs/user-guide/gui-overview.md

@@ -19,6 +19,12 @@ PDB conversion before clustering.
 
 Use this when you need to build cluster-network exports from extracted frames.
 
+### `clusterdynamics`
+
+Use this when you need time-binned cluster-distribution heatmaps, optional
+energy overlays, and lifetime / association / dissociation summaries from an
+extracted XYZ or PDB frame folder.
+
 ### `bondanalysis`
 
 Use this when you need bond-pair and angle-distribution measurements on the
@@ -73,3 +79,7 @@ Several newer SAXS UI surfaces follow the same patterns:
 
 TODO: add screenshots once the docs site has a stable asset pipeline and the
 UI labels settle after the current SAXS workflow changes.
+
+??? note "Artwork Attribution"
+The SAXSShell application icon used in the SAXS UI is based on artwork
+generated with ChatGPT (OpenAI).