Skip to content

Bring ESMX_Data enhancements back into ESMF#547

Open
theurich wants to merge 35 commits into
developfrom
feature/esmx-data
Open

Bring ESMX_Data enhancements back into ESMF#547
theurich wants to merge 35 commits into
developfrom
feature/esmx-data

Conversation

@theurich

@theurich theurich commented May 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This PR is close to a full rewrite of the ESMX_Data implementation. The bulk of this implementation was developed under the NRL NEPTUNE and ESPC projects, where we use ESMX to build, run, and validate components and applications. Key aspects of this work include:

  • Support for multiple geom definitions
  • Each field to specify the associated geom, can be different for different fields
  • Grid definitions to support periodic options (for global grids), as well as support local area grids
  • Better diagnostic output options for fields, with options to diagnose, issue warnings or errors
  • Use of the standard NUOPC Diagnostic attribute to control field data output to NetCDF.
  • More generic options for field data initialization and advance, with support for use of import field data in simple arithmetic expressions for feedback
  • Support switching between Mediator and Model timekeeping mode. This affects time stamping of export fields. It affects where in the run sequence the ESMX_Data instance can/should be placed.

This PR supersedes #483, and closes #521.

The README.md associated with this PR provides a useful high-level overview of the features provided by the proposed ESMX_Data implementation.

The associated nuopc-app-prototypes PR is esmf-org/nuopc-app-prototypes#16

See #551 for UI enhancements identified during review with core team. This PR resolves #551.

theurich added 14 commits May 8, 2026 09:33
@theurich
Replace ESMX_Data implementation with NRL version of Navy-ESPC
3a8b481 
espc-ecosystem hash 5a24d65.
@theurich
Make safe for case without export fields.
8769187
@theurich
Make Advance() routine completely multi-instance safe.
e0b6032
@theurich
Refactor the data processing routines into their own module.
d53f71b
@theurich
Implement special variable handling _step and _coordX. Also align
ccda1f4 
dataInit better with dataAdvance.
@theurich
Fix a typo and move any output to stdout under the dataDiagnose logic.
c3ccd3d
@theurich
Implement support for unary elemental conversion and mathematical
78b9712 
functions.
@theurich
Add the "Dynamic arithmetic expressions" section.
85363e8
@theurich
Restructure how validation is sepcified.
e9d52fd
@theurich
Update documentation to match latest changes with respect to field data
583db1b 
validation.
@theurich
Explicit cast for integer functions to real.
30f71bc
@theurich
Remove dataInit from importFields, which was actually never used.
fd3b1eb
@theurich
Some more documentation polish.
c11bca4
@theurich
More relevant usage examples.
90d5c33
@theurich theurich self-assigned this May 12, 2026
@theurich theurich mentioned this pull request May 13, 2026
Open
@theurich theurich marked this pull request as ready for review May 13, 2026 00:23
@theurich theurich added this to the v09.00.00 milestone May 13, 2026
@theurich theurich added ESMX non backward compatible Modification breaks backward compatibility (e.g. API changes) who: NRL Originates from NRL labels May 13, 2026
@theurich theurich mentioned this pull request May 20, 2026
Open
@theurich

theurich commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

See #551 for UI enhancements identified during review with core team.

theurich added 15 commits June 18, 2026 09:28
@theurich
Add 'Ufrm' to grid options for clarity and consistency with ESMF.
25f23f2 
Now: grid1PeriDimUfrm and gridNoPeriDimUfrm
@theurich
Robustly error out if the geometry referenced by a field is unknown.
915d066 
Capture all cases, including where geometry was not even specified,
@theurich
Update to grid1PeriDimUfrm and gridNoPeriDimUfrm.
5ffe30e
@theurich
Support staggerLocList for geometries. Fields to select staggerLoc with
8194f2c 
@ syntax if multiple options provided.
@theurich
Implement the 'outputs' key and base output handling on it instead of
e6310ef 
Diagnostic attribute.
@theurich
Add the 'separateFieldFiles' logical, optional argument to NUOPC_Write()
46f80ca 
for State and FieldBundle, to support writing all fields to single file
or separate files.
@theurich
Implement 'separateFieldFiles' and 'separateTimeFiles' support.
19bd788
@theurich
Adjust to the introduction of optional argument 'separateFieldFiles' in
c56c9e4 
NUOPC_Write().
@theurich
Switch to default separateFieldFiles=.true. for backward compatibility
e442885 
by default.
@theurich
Change dataValidate: diagnose -> print. Also action options now
44ddce8 
"ignore", "warning", and "error" (default).
@theurich
Use the 'dataInit' on import and export fields to correctly interact
119fa53 
with NUOPC data-dependencies-during-initialize protocol.
@theurich
Merge branch 'develop' into feature/esmx-data
d22d315
@theurich
Correctly initialize neededCurrent and better document relevant code.
a984b2c
@theurich
Add 'dataInit' for importFields. Cover doc on interaction with
6471d99 
data-dependency protocol during initialization. Other doc improvements.
@theurich
Support process() for field variables in importState named identical to
2387f93 
predfined mathematical functions.
@danrosen25 danrosen25 self-assigned this Jul 1, 2026
@theurich theurich requested a review from oehmke July 1, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@uturuncoglu uturuncoglu Awaiting requested review from uturuncoglu
@danrosen25 danrosen25 Awaiting requested review from danrosen25
@oehmke oehmke Awaiting requested review from oehmke

Assignees

@theurich theurich

@danrosen25 danrosen25

Labels

ESMX non backward compatible Modification breaks backward compatibility (e.g. API changes) who: NRL Originates from NRL

Projects

None yet

Milestone

v09.00.00

2 participants

@theurich @danrosen25