Fix issue in makeHGCalValidationPlots.py

[tcuisset]

PR description:

The HGCAL validation script was not working anymore (following a bug introduced in #48565). The labels in HGCALValidator were changed from InputTag to string but the python scripts were not updated.
Also fixed the CaloParticle plots.

PR validation

Before : running makeHGCalValidationPlots.py outputs TypeError: unsupported operand type(s) for +: 'string' and 'str'.
With this PR it succeeds

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49775/47410

[cmsbuild]

A new Pull Request was created by @tcuisset for master.

It involves the following packages:

• Validation/HGCalValidation (dqm)

@cmsbuild, @ctarricone, @gabrielmscampos, @nothingface0, @rseidita can you please review it and eventually sign? Thanks.
@apsallid, @bsunanda, @cseez, @denizsun, @hatakeyamak, @lecriste, @lgray, @pfs, @rovere, @salimcerci, @sethzenz, @vandreev11 this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[nothingface0]

Thank you @tcuisset! Would it make sense to introduce a new CMSSW test to make sure that makeHGCalValidationPlots.py does not break in future PRs? I assume that since no test broke in #48565, one does not already exist.

[tcuisset]

I believe no test exists for this script. However I don't know how to make such a test. It would need a DQM file from a suitable matrix workflow and then run the script on it and check the output and logs.

[bfonta]

@tcuisset
As kindly provided by @mmusich, I can point you to a Twiki and an existing example.
We are also planning to write similar tests for other validation scripts.

[nothingface0]

@tcuisset As kindly provided by @mmusich, I can point you to a Twiki and an existing example. We are also planning to write similar scripts for other validation scripts.

Thanks @bfonta !

An extra piece of information, is the PRE_TEST flag that can be used in BuildFile.xml of the tests, in case a previous test is required to produce the input file (examples here) (I don't see this mentioned in the Twiki)

[mmusich]

Would it make sense to introduce a new CMSSW test to make sure that makeHGCalValidationPlots.py does not break in future PRs?

I wholeheartedly second the request to add a suitable unit test.

It would need a DQM file from a suitable matrix workflow and then run the script on it and check the output and logs.

I wonder if that's really needed. Assuming the structure of the DQM file doesn't change that often, could one use a harvested file pre-cached on eos ?
For instance are these distributed on it?

[tcuisset]

It would need a DQM file from a suitable matrix workflow and then run the script on it and check the output and logs.

I wonder if that's really needed. Assuming the structure of the DQM file doesn't change that often, could one use a harvested file pre-cached on eos ? For instance are these distributed on it?

I could not find the harvested files on EOS. I found the DQMIO files (eg /eos/cms/store/relval/CMSSW_16_0_0_pre3/RelValTTbar_14TeV/DQMIO/PU_150X_mcRun4_realistic_v1_STD_Run4D121_PU-v1/2580000/F009FF04-44A5-45BC-BE52-1D5AFE664BFE.root) however, which I guess could be harvested in the test. But I don't really know how to get a "latest" relval either (it might break at each geometry/GT change).

[mmusich]

I could not find the harvested files on EOS.

the question was for @cms-sw/dqm-l2

But I don't really know how to get a "latest" relval either (it might break at each geometry/GT change).

I don't think that's a concern, harvesting should be fairly robust against that kind of changes.

[nothingface0]

I could not find the harvested files on EOS.

the question was for @cms-sw/dqm-l2

Could you clarify which ones are needed, so that I know what to look for?

[mmusich]

@nothingface0

Could you clarify which ones are needed, so that I know what to look for?

the question is

For instance are these distributed on it?

(as in #49775 (comment))

[nothingface0]

For instance are these distributed on it?

(as in #49775 (comment))

Thanks, I got confused. Yes, they can be found (zipped) here:

/eos/cms/store/group/comm_dqm/DQMGUI_Backup/data/relval/RelVal/CMSSW_16_x_x

E.g.

$ unzip -l DQMGUI_Backup/data/relval/RelVal/CMSSW_16_x_x/DQM_RelVal_CMSSW_16_0_0_pre4_S0029.zip
Archive:  DQMGUI_Backup/data/relval/RelVal/CMSSW_16_x_x/DQM_RelVal_CMSSW_16_0_0_pre4_S0029.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
250899907  01-13-2026 02:03   DQM_V0001_R000000001__RelValBdToKstarEleEle_14TeV__CMSSW_16_0_0_pre4-PU_150X_mcRun4_realistic_v1_STD_BPH81_Run4D121_PU-v1__DQMIO.root
206519624  01-13-2026 02:05   DQM_V0001_R000000001__RelValB0ToKstarMuMu_14TeV__CMSSW_16_0_0_pre4-150X_mcRun4_realistic_v1_STD_BPH81_RegeneratedGS_Run4D121_noPU-v1__DQMIO.root
207339310  01-13-2026 02:34   DQM_V0001_R000000001__RelValBuMixing_14__CMSSW_16_0_0_pre4-150X_mcRun4_realistic_v1_STD_BPH81_RegeneratedGS_Run4D121_noPU-v1__DQMIO.root
252081150  01-13-2026 02:34   DQM_V0001_R000000001__RelValQCD_Pt20toInfMuEnrichPt15_14__CMSSW_16_0_0_pre4-PU_150X_mcRun4_realistic_v1_STD_Run4D110_PU-v2__DQMIO.root
259637830  01-13-2026 02:53   DQM_V0001_R000000001__RelValPsi2SToJPsiPiPi_14TeV__CMSSW_16_0_0_pre4-PU_150X_mcRun4_realistic_v1_STD_BPH81_Run4D121_PU-v1__DQMIO.root
201550536  01-13-2026 03:07   DQM_V0001_R000000001__RelValBsToPhiPhi_KKKK_14TeV__CMSSW_16_0_0_pre4-150X_mcRun4_realistic_v1_STD_BPH81_RegeneratedGS_Run4D121_noPU-v1__DQMIO.root
261234325  01-13-2026 03:08   DQM_V0001_R000000001__RelValBsToMuMu_14TeV__CMSSW_16_0_0_pre4-PU_150X_mcRun4_realistic_v1_STD_BPH81_Run4D121_PU-v1__DQMIO.root
260943235  01-13-2026 03:13   DQM_V0001_R000000001__RelValBsToJpsiPhi_mumuKK_14TeV__CMSSW_16_0_0_pre4-PU_150X_mcRun4_realistic_v1_STD_BPH81_Run4D121_PU-v1__DQMIO.root
---------                     -------
1900205917                     8 files

[tcuisset]

I've added a test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49775/47657

[cmsbuild]

Pull request #49775 was updated. @cmsbuild, @ctarricone, @gabrielmscampos, @nothingface0, @rseidita can you please check and sign again.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49775/47658

[cmsbuild]

Pull request #49775 was updated. @cmsbuild, @ctarricone, @gabrielmscampos, @nothingface0, @rseidita can you please check and sign again.

[mmusich]

please test

[cmsbuild]

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d957b4/50848/summary.html
COMMIT: 2ab7b76
CMSSW: CMSSW_16_1_X_2026-01-23-1100/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/49775/50848/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 1 lines from the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 52
• DQMHistoTests: Total histograms compared: 4025536
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4025513
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 51 files compared)
• Checked 222 log files, 193 edm output root files, 52 DQM output files
• TriggerResults: no differences found

[nothingface0]

+dqm

Thanks @tcuisset for adding the test!

There's probably going to be a conflict in #49932 after this one is merged, but I guess it can be handled on their side.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

[mandrenguyen]

+1