Plan Radiative Design

Smoke Radiative Analysis Subpackage — Design Document

Date: 2026-03-30 Status: Implementation complete on ceres branch — not yet merged to main Branch: ceres

Status Summary

All PlumeSentinelAI CERES smoke radiative analysis functionality has been ported to DAVINCI as a standalone davinci_monet/radiative/ subpackage. The Sep 2020 West Coast fire event reproduces all 14 original figures from the config-driven pipeline.

Integration status: This lives on the ceres branch. It has not been merged to main — pending decision on how this side project fits into the main DAVINCI project.

What's Done

Component	Status	Notes
RT module (rt.py)	Complete	Ported as-is, 18 tests passing
Pydantic config schema	Complete	14 plot types, SiteConfig, peak_date, Merra2RadConfig
Processing module	Complete	Anomalies, regridding, smoke AOD, surface dimming
CERES loader (local)	Complete	SYN1deg daily files, domain subsetting
CERES loader (OPeNDAP)	Implemented, untested	EBAF + SYN1deg DAP2 binary fetch — needs network
MERRA-2 aerosol loader	Complete	Daily mean, regrid to CERES grid
MERRA-2 radiation loader	Complete	tavg1_2d_rad_Nx, SWGNT/SWGNTCLN/ALBEDO
AERONET loader	Complete	CSV, lat/lon alias handling, domain/site filter
Progressive RT levels	Complete	L0–L4 model hierarchy using rt.py
14 plot renderers	Complete	All match original PlumeSentinelAI figures
Runner + CLI	Complete	davinci-monet radiative run config.yaml
Unit tests	51 passing	Config, processing, loaders, plots, runner, RT
Sep 2020 validation	Complete	14 plots generated, visually verified against originals

What's Remaining

Item	Priority	Notes
OPeNDAP integration tests	Medium	Deferred — needs real network connection
Australian fires 2019-2020	Next	New config + data acquisition
FIREX-AQ analysis	Planned	New config + data acquisition
Tilde expansion in glob paths	Low	Workaround: use absolute paths in machine-specific configs
Merge strategy to main	Decision needed	See below

Open Question: Integration with Main Project

The ceres branch adds a self-contained davinci_monet/radiative/ subpackage that does NOT use the standard model-vs-obs pipeline. Options:

1. Merge to main as-is — it's self-contained and doesn't interfere with existing code
2. Keep on a long-lived feature branch — merge only when the side project matures
3. Extract to a separate package — if it diverges too far from DAVINCI's scope

The only touchpoint with main is cli/app.py (adds the radiative subcommand).

---

Problem

The PlumeSentinelAI repository contains a set of research scripts for analyzing wildfire smoke radiative effects using CERES, MERRA-2, and AERONET data. These scripts are hardcoded to a single event (Sep 2020 West Coast fires) and lack the config-driven, modular structure needed to extend to other events (2019-2020 Australian fires, FIREX-AQ). We want to move this functionality into DAVINCI as a standalone subpackage that uses DAVINCI's infrastructure (config validation, styling, CLI) without forcing it into the model-vs-obs evaluation pipeline.

Scope

In scope:

• CERES data loading (EBAF monthly, SYN1deg daily) from local files and OPeNDAP
• MERRA-2 aerosol and radiation loading
• AERONET site data loading with domain/site filtering
• Anomaly computation (event minus background baseline)
• Smoke AOD derivation (configurable species)
• Radiative transfer calculations (delta-Eddington two-stream, surface dimming)
• Progressive RT model comparison (5 levels: Beer-Lambert through spectral δ-Eddington)
• 14 plot types driven by YAML config
• CLI subcommand

Out of scope:

• CERES as a generic observation reader in DAVINCI's pairing pipeline
• OPeNDAP integration tests (deferred — hotspot network)
• Global climatology workflow (future phase)

Target Events

1. Sep 2020 West Coast fires — complete, all 14 plots validated
2. 2019-2020 Australian fire season — planned
3. FIREX-AQ — planned

Architecture

Package Structure (as implemented)

davinci_monet/radiative/
├── __init__.py                # Public API (RadiativeConfig, run_radiative_analysis)
├── config.py                  # Pydantic schemas (EventConfig, CeresConfig, Merra2Config,
│                              #   Merra2RadConfig, SiteConfig, AeronetConfig,
│                              #   SurfaceImpactConfig, RadiativeConfig)
├── rt.py                      # δ-Eddington two-stream, Rayleigh, Angstrom, solar geometry
├── processing.py              # Anomalies, regridding, smoke AOD, surface dimming
├── processing_rt_levels.py    # 5 progressive RT model levels (L0–L4) + compute_rt_levels()
├── loaders/
│   ├── __init__.py
│   ├── ceres.py               # CERES local + OPeNDAP (EBAF, SYN1deg DAP2 binary)
│   ├── merra2.py              # MERRA-2 aerosol (tavg1_2d_aer_Nx)
│   ├── merra2_rad.py          # MERRA-2 radiation (tavg1_2d_rad_Nx)
│   └── aeronet.py             # AERONET CSV with lat/lon alias handling
├── plots/
│   ├── __init__.py
│   ├── event_fields.py        # 4-panel TOA maps
│   ├── anomaly_maps.py        # 4-panel anomaly maps
│   ├── surface_flux.py        # 4-panel CERES surface SW/LW
│   ├── scatter.py             # 2-panel AOD vs SW with colorbar
│   ├── daily_correlation.py   # Per-day correlation bar chart
│   ├── spatial_comparison.py  # MERRA-2 smoke AOD vs CERES ΔSW maps
│   ├── site_timeseries.py     # 6-site smoke AOD + ΔSW + AERONET
│   ├── surface_impact.py      # 3-panel surface dimming maps
│   ├── surface_dimming_timeseries.py  # MERRA-2 RT vs semi-empirical at sites
│   ├── method_comparison.py   # RT vs semi-empirical scatter + efficiency
│   ├── rt_efficiency.py       # Progressive model radiative efficiency curves
│   ├── rt_scatter.py          # 5-panel scatter (L0–L4 vs MERRA-2 RT)
│   ├── rt_timeseries.py       # 6-site model levels vs MERRA-2 RT
│   └── rt_spatial.py          # 3-panel spatial RT comparison
└── runner.py                  # Orchestrates load → process → plot, CLI entry point

Relationship to DAVINCI

The radiative subpackage is self-contained — it does not use DAVINCI's pairing pipeline, observation registry, or stats engine. It shares:

• Config infrastructure: Pydantic validation
• Style system: apply_ncar_style(), NCAR_COLORS
• CLI: davinci-monet radiative subcommand (registered in cli/app.py)
• Logging: davinci_monet.logging

It does NOT register with observation_registry or model_registry.

Plot Types (14 total)

CERES Event Analysis (3)

Plot ID	Description
toa_event_fields	4-panel map: AOD, TOA SW, TOA net, cloud fraction
anomaly_maps	4-panel event-minus-background anomalies
surface_flux	4-panel CERES surface SW/LW down + anomalies

CERES × MERRA-2 Comparison (4)

Plot ID	Description
sw_vs_aod_scatter	2-panel scatter: total + smoke AOD vs CERES SW
daily_correlation	Per-day Pearson r bar chart
spatial_comparison	Side-by-side MERRA-2 smoke AOD vs CERES ΔSW
site_timeseries	6-site dual-axis: smoke AOD + ΔSW + AERONET

Surface Impact (3)

Plot ID	Description
surface_impact	3-panel maps: smoke AOD, MERRA-2 RT, semi-empirical
surface_dimming_timeseries	6-site MERRA-2 RT vs semi-empirical
method_comparison	Scatter + radiative efficiency comparison

Progressive RT Model Comparison (4)

Plot ID	Description
rt_efficiency	Radiative efficiency curves for 5 model levels
rt_scatter	5-panel scatter: each level vs MERRA-2 RT
rt_timeseries	6-site time-series for all levels
rt_spatial	3-panel spatial: MERRA-2 RT, best model, residual

CLI

davinci-monet radiative run configs/west-coast-2020.yaml
davinci-monet radiative fetch-ceres --product syn1deg --start 2020-09-05 --end 2020-09-15 --output ~/Data/ceres/

Tests

51 tests across 6 test files, all passing:

• test_radiative_rt.py — 18 RT module tests (energy conservation, physical limits, spectral, solar geometry)
• test_radiative_config.py — 10 config validation tests
• test_radiative_processing.py — 11 processing function tests
• test_radiative_loaders.py — 5 loader tests (synthetic data)
• test_radiative_plots.py — 5 plot smoke tests
• test_radiative_runner.py — 2 runner integration tests

Full DAVINCI suite: 1081 passed, 1 skipped, 0 failures.

Commits on ceres branch

04007d6 feat(radiative): add progressive RT model comparison (5 levels vs MERRA-2 RT)
1b1a252 feat(radiative): reproduce all 10 original PlumeSentinelAI figures
03a1107 fix(radiative): handle lat/lon column aliases in AERONET loader
002bbd1 fix(radiative): initialize sites on AERONET load failure
d02b5d0 style(radiative): finalize public API, black/isort formatting
e39a3a6 feat(radiative): add runner and CLI integration
f6d5ce8 feat(radiative): add all plot renderers
c5aa13d feat(radiative): add CERES, MERRA-2, and AERONET data loaders
5780494 feat(radiative): add processing module
22b5af1 feat(radiative): add Pydantic configuration schema
66c4a89 feat(radiative): port RT module and tests from PlumeSentinelAI

Source Material

Scripts ported from ~/EarthSystem/PlumeSentinelAI/scripts/:

• fetch_ceres_ebaf.py → loaders/ceres.py
• fetch_ceres_syn1deg_daily.py → loaders/ceres.py
• plot_ceres_syn1deg_event.py → event_fields.py + anomaly_maps.py + surface_flux.py
• plot_ceres_sw_vs_merra2_aod.py → scatter.py + daily_correlation.py + spatial_comparison.py + site_timeseries.py
• plot_surface_sw_impact.py → surface_impact.py + surface_dimming_timeseries.py + method_comparison.py
• plot_semiempirical_levels.py → processing_rt_levels.py + rt_efficiency.py + rt_scatter.py + rt_timeseries.py + rt_spatial.py
• rt.py → rt.py (as-is)
• test_rt.py → tests/test_radiative_rt.py