[natalie-perlin]

Use of generative AI

Please refer to our temporary spack-stack AI policy for further information.

• I confirm that this contribution was submitted by a real person. If AI tools or agents were used in its preparation, I have personally reviewed and take responsibility for all submitted content, code, and proposed changes. This contribution is not autonomously generated by AI.

Description

Problem: spack bootstrap mirror writes metadata/sources and metadata/binaries at fixed paths. Rerunning it for a newer Spack (e.g., when doing spack-stack v2.2.0 install or spack-stack-nvhpc-dev) and attempting to use mirrors overwrites the metadata (including clingo.json, gnupg.json, patchelf.json) that installs from older spack-stack versions rely on. Running it directly into an existing mirror also fails with Destination path .../bootstrap_cache/blobs already exists.

Fixes the following error, if run from the new version of spack-stack built under /contrib/spack-stack/:

 spack bootstrap mirror --binary-packages ${BOOT} 2>&1 | tee log.bootstrap-mirror
==> Adding "clingo-bootstrap@spack+python platform=linux target=x86_64" and dependencies to the mirror at /contrib/spack-stack/bootstrap-mirror/bootstrap_cache
==> Compilers have been configured automatically from PATH inspection
==> Adding "gnupg@2.3: platform=linux target=x86_64" and dependencies to the mirror at /contrib/spack-stack/bootstrap-mirror/bootstrap_cache
==> Adding "patchelf@0.13.1: platform=linux target=x86_64" and dependencies to the mirror at /contrib/spack-stack/bootstrap-mirror/bootstrap_cache
==> Adding "gnuconfig" and dependencies to the mirror at /contrib/spack-stack/bootstrap-mirror/bootstrap_cache
==> Error: Destination path '/contrib/spack-stack/bootstrap-mirror/bootstrap_cache/blobs' already exists
==> Adding binary packages from "https://github.com/spack/spack-bootstrap-mirrors/releases/download/v2.2/bootstrap-buildcache.tar.gz" to the mirror at /contrib/spack-stack/bootstrap-mirror/bootstrap_cache

Change (util/epic/batch_install.sh):

• Build a metadata key from Spack's bootstrap release: -. The tag is from BINARY_TARBALL in spack/lib/spack/spack/cmd/bootstrap.py, e.g. v2.2; the hash is the first 8 characters of the sha256 of the three github-actions-v2/*.json files.
• Keep generating the mirror in a temp directory. Merge bootstrap_cache/ into the shared cache, which is safe because its files are stored by checksum and by name. Copy the metadata to metadata// instead of metadata/.
  Rewrite url: in the copied metadata.yaml files to the absolute bootstrap_cache path, since the relative ../../bootstrap_cache no longer resolves one level deeper.
• Register local-sources and local-binaries from metadata//{sources,binaries}.

Dependencies

It would be good to merge it into the release/2.2 version, so it could be used to install (or test-installed) the spack-stack-2.2.0 on EPIC-supported platforms with the batch_install.sh script.

If other batch_install.sh scripts exist, they may also adapt this approach.

Issues addressed

Add any issues here that this PR closes or is related to. Use "Resolves" or "Closes" to automatically close issues when the PR is merged. Using "Working towards" or "Related to" for other cases.

Applications affected

List all known applications (UFS WM, JEDI, SRW, etc.) intentionally or unintentionally affected by this PR.

Systems affected

EPIC-supported platforms : Ursa, Gaea-c6, Hercule, Orion, Cloud platforms

Testing

• CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
  • GitHub actions CI tests pass
  • GitHub actions CI tests do not pass (provide explanation)
  • GitHub actions CI tests skipped (provide explanation if necessary)
• New tests added: List and describe any new tests added to GitHub actions
  • ...
• Additional testing: Add information on any additional tests conducted
  • ...

Checklist

• This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
• These changes have been tested on the affected systems and applications.
• All dependency PRs/issues have been resolved and this PR can be merged.
• All necessary updates to the documentation (spack-stack wiki) will be made when this PR is merged

[climbfuji]

I have not seen this problem before. But if this is a problem (and I can see why) and this solution works, then that's great. May steal it from you later if the need arises. Also added @mathomp4 for his awareness.

[mathomp4]

Interesting. I'm not sure I've seen this (yet) on my NASA machines.