Update Pt(111) surface thermo library with NOx chemistry and consistent DFT references#716
Conversation
Regression Testing ResultsWARNING:root:Initial mole fractions do not sum to one; normalizing. Detailed regression test results.Regression test aromatics:Reference: Execution time (DD:HH:MM:SS): 00:00:00:54 aromatics Passed Core Comparison ✅Original model has 15 species. aromatics Failed Edge Comparison ❌Original model has 106 species. Non-identical thermo! ❌
Identical thermo comments: Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: Aromatics Comparison✅ All Observables varied by less than 0.500 on average between old model and new model in all conditions! aromatics Passed Observable Testing ✅Regression test liquid_oxidation:Reference: Execution time (DD:HH:MM:SS): 00:00:01:59 liquid_oxidation Passed Core Comparison ✅Original model has 37 species. liquid_oxidation Failed Edge Comparison ❌Original model has 214 species. DetailsObservables Test Case: liquid_oxidation Comparison✅ All Observables varied by less than 0.100 on average between old model and new model in all conditions! liquid_oxidation Passed Observable Testing ✅Regression test nitrogen:Reference: Execution time (DD:HH:MM:SS): 00:00:01:03 nitrogen Passed Core Comparison ✅Original model has 41 species. nitrogen Failed Edge Comparison ❌Original model has 133 species. Non-identical thermo! ❌
thermo: Thermo group additivity estimation: group(O2s-CdN3d) + group(N3d-OCd) + group(Cd-HN3dO) + ring(oxirene) + radical(CdJ-NdO) Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: NC Comparison✅ All Observables varied by less than 0.200 on average between old model and new model in all conditions! nitrogen Passed Observable Testing ✅Regression test oxidation:Reference: Execution time (DD:HH:MM:SS): 00:00:01:45 oxidation Passed Core Comparison ✅Original model has 59 species. oxidation Passed Edge Comparison ✅Original model has 230 species. DetailsObservables Test Case: Oxidation Comparison✅ All Observables varied by less than 0.500 on average between old model and new model in all conditions! oxidation Passed Observable Testing ✅Regression test sulfur:Reference: Execution time (DD:HH:MM:SS): 00:00:00:40 sulfur Passed Core Comparison ✅Original model has 27 species. sulfur Failed Edge Comparison ❌Original model has 89 species. DetailsObservables Test Case: SO2 ComparisonThe following observables did not match: ❌ Observable species O=S=O varied by more than 0.100 on average between old model SO2(15) and new model SO2(15) in condition 1.
sulfur Failed Observable Testing ❌Regression test superminimal:Reference: Execution time (DD:HH:MM:SS): 00:00:00:25 superminimal Passed Core Comparison ✅Original model has 13 species. superminimal Passed Edge Comparison ✅Original model has 18 species. Regression test RMS_constantVIdealGasReactor_superminimal:Reference: Execution time (DD:HH:MM:SS): 00:00:02:18 RMS_constantVIdealGasReactor_superminimal Passed Core Comparison ✅Original model has 13 species. RMS_constantVIdealGasReactor_superminimal Passed Edge Comparison ✅Original model has 13 species. DetailsObservables Test Case: RMS_constantVIdealGasReactor_superminimal Comparison✅ All Observables varied by less than 0.100 on average between old model and new model in all conditions! RMS_constantVIdealGasReactor_superminimal Passed Observable Testing ✅Regression test RMS_CSTR_liquid_oxidation:Reference: Execution time (DD:HH:MM:SS): 00:00:30:23 RMS_CSTR_liquid_oxidation Failed Core Comparison ❌Original model has 35 species. RMS_CSTR_liquid_oxidation Failed Edge Comparison ❌Original model has 107 species. DetailsObservables Test Case: RMS_CSTR_liquid_oxidation Comparison✅ All Observables varied by less than 0.100 on average between old model and new model in all conditions! RMS_CSTR_liquid_oxidation Passed Observable Testing ✅beep boop this comment was written by a bot 🤖 |
bjkreitz
left a comment
bjkreitz
left a comment
There was a problem hiding this comment.
Thanks, Kirk! I performed a comparison of the old and new thermodatabases. Here are the results:
As you mentioned, most of these changes affect the nitrogen containing adsorbates. I can confirm that the enthalpy only changes significantly for XOC(OH)O, CO2X, XOC(O)XO due to the reason that you mentioned above. What do others think about correcting these species again @cfgoldsmith @rwest @mjohnson541 ? It is well known that BEEF overbinds species with a OCO backbone and correction factors have been routinely applied. Without these corrections, it is difficult to build good mechanisms that include CO2. All other enthalpy changes are within 5 kJ/mol except for XOH, but you found a more stable structure (Matt found this too with Pynta). But it is hacky ...
Entropy and heat capacities follow the same trend and mostly N-containing species deviate more significantly from the database. However, there were a couple of other adsorbates that had a large change in the entropy. These adsorbates were XCCH3 and XOXO. Can you investigate if everything is okay for these species? Not sure which value is correct, but it is good to double check.
|
I also observed the same issue with HOX with CH3OX is that also resolved here? I trained an SIDT delta model a while ago for correcting Hf from BEEF-vdW to a calibrated mix of Pt111 Silbaugh and Campbell paper and active tables data: I'm not intimately familiar with the corrections, so I'm not sure if it solves this problem as is, but I feel like these delta frameworks are the most effective, interpretable, and updateable way to consistently apply these kinds of corrections...particularly ones we talk about in terms of biases in method treatment of specific molecular substructures. Can this correction be described in terms of trainable data? The delta can be things other than Hf. |
|
@bjkreitz I actually recalculated XOXO for something Maryke is working on, and XCCH3 I used the energy, ZPE, and frequencies from the methane oxidation structure google sheet. @mjohnson541 I used the HOX (or now called XOH) that you found that was lower in energy, but have not used a CH3O structure that you found, I am using one that Bjarne found. As far as corrections go, this is just the thermolibrary PR, #717 is handling the adsorption correction tree separately. Bjarne and I are discussing modifying 3 library entries for which we know that the beef value is overbinding some structures. Currently if you look at the file: thermo_PR_data/data/Pt111_ads_data.py I have a chem graph stored for each structure (and all the relevant DFT data) in a Python dictionary. If I were to take this file format and also attach "group" corrections for each species, would that be sufficient for you to train pysidt? |
|
@kirkbadger18 I've sent you the lowest energy structure Pynta found for CH3OX, it was about 2 kcal/mol lower than the library value. These delta model frameworks aren't mapping structure to adsorption energy corrections, but rather structure to corrections between a lower accuracy data source and a higher accuracy data source. I recently did this for MLIPs to DFT here (https://chemrxiv.org/doi/full/10.26434/chemrxiv.15000328/v1). Although, this here is of course a far more delicate operation than that. |
|
@mjohnson541 Okay so in the future if we had better DFT data for some species, or had more experimental data, or hand pick functionals for some species and different functionals for other species as "best", then delta learned how to get from some standard calculator to "best". Not correcting as in adsorption corrections but learning a way to correct from some lower-level method to the "best" method. |
|
Matt's point is valid and we actually thought about using the error cancellation method to boost accuracy. The SIDT would be an alternative approach, although the results in the figure you shared do not look that great (or maybe I'm missing something). I think that it would make sense to have 2 thermo libraries. One would be the BEEF-vdW DFT data library (with maybe the corrections for CO2) and then one that contains "more accurate" enthalpies of formation from various sources or the SIDT corrected. The normal user just wants to use the most accurate data, whereas for our study, consistency is typically more important, especially for the UQ stuff that we and Sevy are doing. |
|
@kirkbadger18 could you double check your XCCH3 data and the NASA calculation again? If you used the same data, there shouldn't be any difference. I will also double check my old files. |
|
@bjkreitz This is the entry I am using: {
"adsorbate_name": "XCCH3",
"atomic_composition": {
"H": 3, "C": 2, "N": 0, "O": 0, "Pt": 1
},
"dft_energy": [
-378183.471539935, "eV"
],
"zpe": [
1.114, "eV"
],
"frequencies": [
68.1, 130.4, 183.2, 375, 426.6, 463.9, 980, 981.6, 1060, 1367.1, 1432.8, 1433.6, 2975.1, 3045.3, 3046.8, "cm-1"
],
"sites_occupied": 1,
"connectivity": "1 X u0 p0 c0 {3,T}\n2 C u0 p0 c0 {3,S} {4,S} {5,S} {6,S}\n3 C u0 p0 c0 {1,T} {2,S}\n4 H u0 p0 c0 {2,S}\n5 H u0 p0 c0 {2,S}\n6 H u0 p0 c0 {2,S}\n"
}, |
|
@bjkreitz That is full leave-one-out/extrapolation errors not fitting/training error, but histograms are always tricky to look at. More concretely one I just retrained now takes BEEF-vdW with 0.42 eV MAE leave-one-out relative to experiment/active tables=> corrected BEEF-vdW with 0.16 eV MAE leave-one-out. Anyway, it seemed like you did a lot of these species with CBH already, is there a reason there hasn't been an associated thermo library until now? What's stopping us right now from consistently using CBH corrections on Pt111 data? |
|
Darn, my lowest vibrational frequency for CCH3 was missing a 6 (My lowest mode was 8.1 instead of 68.1). Must have happened when copying over the data from the spreadsheet. Your values are correct. |
|
@mjohnson541 I have recomputed the XOCH3 energy and frequencies with your structure and have updated this PR with the resulting NASA7 polynomials. |
Regression Testing ResultsWARNING:root:Initial mole fractions do not sum to one; normalizing. Detailed regression test results.Regression test aromatics:Reference: Execution time (DD:HH:MM:SS): 00:00:00:54 aromatics Passed Core Comparison ✅Original model has 15 species. aromatics Failed Edge Comparison ❌Original model has 106 species. Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: Aromatics Comparison✅ All Observables varied by less than 0.500 on average between old model and new model in all conditions! aromatics Passed Observable Testing ✅Regression test liquid_oxidation:Reference: Execution time (DD:HH:MM:SS): 00:00:01:59 liquid_oxidation Passed Core Comparison ✅Original model has 37 species. liquid_oxidation Failed Edge Comparison ❌Original model has 214 species. Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: liquid_oxidation Comparison✅ All Observables varied by less than 0.100 on average between old model and new model in all conditions! liquid_oxidation Passed Observable Testing ✅Regression test nitrogen:Reference: Execution time (DD:HH:MM:SS): 00:00:01:03 nitrogen Failed Core Comparison ❌Original model has 41 species. Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: nitrogen Failed Edge Comparison ❌Original model has 133 species. Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: NC Comparison✅ All Observables varied by less than 0.200 on average between old model and new model in all conditions! nitrogen Passed Observable Testing ✅Regression test oxidation:Reference: Execution time (DD:HH:MM:SS): 00:00:01:45 oxidation Failed Core Comparison ❌Original model has 59 species. Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: oxidation Failed Edge Comparison ❌Original model has 230 species. Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: Oxidation ComparisonThe following observables did not match: ❌ Observable species [OH] varied by more than 0.500 on average between old model OH(5) and new model OH(5) in condition 1.
oxidation Failed Observable Testing ❌Regression test sulfur:Reference: Execution time (DD:HH:MM:SS): 00:00:00:40 sulfur Passed Core Comparison ✅Original model has 27 species. sulfur Failed Edge Comparison ❌Original model has 89 species. Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: Non-identical kinetics! ❌
kinetics: DetailsObservables Test Case: SO2 ComparisonThe following observables did not match: ❌ Observable species O=S=O varied by more than 0.100 on average between old model SO2(15) and new model SO2(15) in condition 1.
sulfur Failed Observable Testing ❌Regression test superminimal:Reference: Execution time (DD:HH:MM:SS): 00:00:00:25 superminimal Passed Core Comparison ✅Original model has 13 species. superminimal Passed Edge Comparison ✅Original model has 18 species. Regression test RMS_constantVIdealGasReactor_superminimal:Reference: Execution time (DD:HH:MM:SS): 00:00:02:18 RMS_constantVIdealGasReactor_superminimal Passed Core Comparison ✅Original model has 13 species. RMS_constantVIdealGasReactor_superminimal Passed Edge Comparison ✅Original model has 13 species. DetailsObservables Test Case: RMS_constantVIdealGasReactor_superminimal Comparison✅ All Observables varied by less than 0.100 on average between old model and new model in all conditions! RMS_constantVIdealGasReactor_superminimal Passed Observable Testing ✅Regression test RMS_CSTR_liquid_oxidation:Reference: Execution time (DD:HH:MM:SS): 00:00:30:23 RMS_CSTR_liquid_oxidation Failed Core Comparison ❌Original model has 35 species. RMS_CSTR_liquid_oxidation Failed Edge Comparison ❌Original model has 107 species. DetailsObservables Test Case: RMS_CSTR_liquid_oxidation Comparison✅ All Observables varied by less than 0.100 on average between old model and new model in all conditions! RMS_CSTR_liquid_oxidation Passed Observable Testing ✅beep boop this comment was written by a bot 🤖 |
3 participants
I have run new DFT calculations on nitrogen containing adsorbates as well as reran DFT calculation on existing nitrogen containing adsorbates in this library. This was done with a set of DFT settings consistent with the settings used on the hydrocarbons. I compiled the new DFT results along with @bjkreitz's results into a python readable file, and used the new workflow for computing adsorbate thermochemistry as found here.
All of the data files used along with the script I used to generate the new library can be found here:
thermo_PR_data.zip
After looking at the diff, I see that the NASA7 polynomials changed for nitrogen containing adsorbates as expected and remained mostly the same for the hydrocarbon species. The two species that change noticeably are: CO2X and CO3X. This is due to a correction @bjkreitz made in #587. I will leave it as an open question to reviewers if we should use the consistent BEEF-vdW dataset even though it over binds these species, or if we should make an exception for them.