The UPP Jenkins CI builds are failing

[clyden-noaa]

Jenkins UPP CI Builds are failing
Intel Linux Build fails in setup

@AlexanderRichert-NOAA Your name came up as having some experience troubleshooting these types of errors. Is this something you can take a look at?

[AlexanderRichert-NOAA]

I'm not familiar with the Jenkins CI for UPP. If you can point me to some logs etc. I can try to take a guess as to what might be going on.

[clyden-noaa]

@AlexanderRichert-NOAA Please let me know if you need more information...thanks

2026-02-10T17:45:51.5021993Z Cloning into 'spack'...
2026-02-10T17:46:16.3457566Z ==> Error: [Errno 2] No such file or directory: '/home/runner/work/UPP/UPP/spack/etc/spack/defaults/base/${SPACK_STACK_DIR}/spack-ext/lib/jcsda-emc/spack-stack/stack/cmd'
2026-02-10T17:46:16.3812209Z ##[error]Process completed with exit code 3.

https://github.com/AlysonStahl-NOAA/UPP/actions/runs/21875945664/job/63144740003

[AlexanderRichert-NOAA]

Thanks. I strongly suspect that this is the result of recent changes to the JCSDA/spack-spack fork of Spack, where the SPACK_STACK_DIR variable is now baked into a handful of configuration paths, so things fail when the variable isn't defined (because it's being used outside the spack-stack framework).

Tagging @climbfuji here-- Prior to the Spack v1-related changes, we had our Spack fork relatively pristine and swappable with main Spack. Do you think it would be much work to get things back to that state?

In any case, @clyden-noaa you may wish to try getting Spack from github.com/spack/spack instead of JCSDA and see if that fixes it. Since you're not using spack-stack, you're not likely to get any advantage from using the JCSDA fork.

[climbfuji]

@AlexanderRichert-NOAA In our spack fork, there's a script that we need to run to get the unit tests of Spack itself to pass. I think this basically does what you need (and it can be changed as long as it doesn't break the unit tests):

https://github.com/JCSDA/spack/blob/spack-stack-dev/REMOVE_SPACK_STACK_MODS_FOR_CI.sh