Update ROCm to version 7.0.0

[fwyzard]

For the changes since ROCm 6.4.3, see the ROCm 7.0.0 release notes.

[cmsbuild]

A new Pull Request was created by @fwyzard for branch IB/CMSSW_16_0_X/master.

@akritkbehera, @cmsbuild, @iarspider, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

[cmsbuild]

cms-bot internal usage

[fwyzard]

enable gpu

[fwyzard]

test parameters:

• full_cmssw = true
• enable = gpu

[fwyzard]

please test

[cmsbuild]

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1c646d/48188/summary.html
COMMIT: f34cbf9
CMSSW: CMSSW_16_0_X_2025-09-16-2300/el8_amd64_gcc12
Additional Tests: GPU,AMD_MI300X,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10084/48188/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @Dr15Jones Moved Pool based tests from FWCore to IOPool cmssw#48908
• @makortel Remove SwitchProducer and ConditionalTask from the framework cmssw#48816
• @mmusich Further fixes for HGCal validation at HLT cmssw#48221
• @cms-sw [RFC] Add tool to automatically update most of ROCm subpackages' versions #10080
• @cms-sw clhep: replace stdlib.h with cstdlib #10083

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1c646d/48188/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1c646d/48188/git-merge-result

Build

I found compilation error when building:

Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneRadixSortROCmAsync/libalpakaTestOneRadixSortROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneToManyAssocROCmAsync/libalpakaTestOneToManyAssocROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestPrefixScanROCmAsync/libalpakaTestPrefixScanROCmAsync_rocm.a to productstore area:
cp: cannot stat 'tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestPrefixScanROCmAsync/libalpakaTestPrefixScanROCmAsync_rocm.a': No such file or directory
>> Deleted: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestPrefixScanROCmAsync/libalpakaTestPrefixScanROCmAsync_rocm.a
gmake: *** [config/SCRAM/GMake/Makefile.rules:1920: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestPrefixScanROCmAsync/libalpakaTestPrefixScanROCmAsync_rocm.a] Error 1
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestRadixSortROCmAsync/libalpakaTestRadixSortROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestSimpleVectorROCmAsync/libalpakaTestSimpleVectorROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaInterface/test/alpakaWorkDivisionROCmAsync/libalpakaWorkDivisionROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaMath/test/alpakaTestDeltaPhiROCmAsync/libalpakaTestDeltaPhiROCmAsync_rocm.a to productstore area:
>> Entering Package RecoTracker/MkFitCore


[fwyzard]

ROCm 7.0 does not implement warpSize as constexpr 🤦🏻

See e.g. ROCm/ROCm#4121 (comment) .

[fwyzard]

please test with cms-sw/cmssw#48947

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1c646d/48208/summary.html
COMMIT: f34cbf9
CMSSW: CMSSW_16_0_X_2025-09-18-1100/el8_amd64_gcc12
Additional Tests: GPU,AMD_MI300X,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10084/48208/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @smorovic [DAQ] skip directory creation in read-only discovery mode cmssw#48944
• @linacre DemonstratorTools: make TMUX slice staggering optional, and disable it in GTTFileWriter cmssw#48771

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1c646d/48208/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1c646d/48208/git-merge-result

Comparison Summary

Summary:

• You potentially removed 156 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3978581
• DQMHistoTests: Total failures: 23
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3978538
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 210 log files, 180 edm output root files, 49 DQM output files
• TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 53486
• DQMHistoTests: Total failures: 9489
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 43997
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 24 log files, 30 edm output root files, 7 DQM output files
• TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

• You potentially removed 26 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 53486
• DQMHistoTests: Total failures: 7285
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 46201
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 24 log files, 30 edm output root files, 7 DQM output files
• TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

• You potentially removed 21 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 53486
• DQMHistoTests: Total failures: 7334
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 46152
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 24 log files, 30 edm output root files, 7 DQM output files
• TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

• You potentially removed 24 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 53486
• DQMHistoTests: Total failures: 8459
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 45027
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 24 log files, 30 edm output root files, 7 DQM output files
• TriggerResults: no differences found

[fwyzard]

hold

The tests look good, but we should run some benchmarks before merging the PR.

[cmsbuild]

Pull request has been put on hold by @fwyzard
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[fwyzard]

Superseded by ROCm 7.0.1 bugfix release.