-
Notifications
You must be signed in to change notification settings - Fork 0
feat: split molecular liquids into RDF + density benchmarks #149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
lwalew
merged 14 commits into
feat/npt-sims-solvent
from
feat/split-molecular-liquids-density
Jul 9, 2026
Merged
Changes from 4 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
4777de2
feat: split molecular liquids RDF and density into separate benchmarks
lwalew 511261b
refactor: type Benchmark.run_mode as RunMode, drop _as_run_mode helper
lwalew da3481d
fix: use Leon's density scoring params (2% threshold, alpha=0.1)
lwalew ce664a6
chore: drop verbose comment on density scoring constants
lwalew d374e80
refactor: look up solvent structures by name in the UI, drop order co…
lwalew 49cee31
feat: address comments
lwalew 338866e
feat: remove file divider
lwalew ed09f71
feat: update `data_dir` docstring
lwalew 1b10eaf
feat: more private functions
lwalew a8c9141
docs: add references
lwalew 9953bca
docs: add CHANGELOG entry for molecular liquids split; simplify solve…
lwalew 95a8be7
docs: shorten release
lwalew 8b139aa
docs: add references for water density
lwehrhan 34a4e94
feat: update reference solvent densities
lwalew File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
20 changes: 20 additions & 0 deletions
20
docs/source/api_reference/molecular_liquids/solvent_density.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| .. _solvent_density_api: | ||
|
|
||
| Solvent Density | ||
| =============== | ||
|
|
||
| .. module:: mlipaudit.benchmarks.solvent_density.solvent_density | ||
|
|
||
| .. autoclass:: SolventDensityBenchmark | ||
|
|
||
| .. automethod:: __init__ | ||
|
|
||
| .. automethod:: run_model | ||
|
|
||
| .. automethod:: analyze | ||
|
|
||
| .. autoclass:: SolventDensityResult | ||
|
|
||
| .. autoclass:: SolventDensityStructureResult | ||
|
|
||
| .. autoclass:: SolventDensityModelOutput |
File renamed without changes.
18 changes: 18 additions & 0 deletions
18
docs/source/api_reference/molecular_liquids/water_density.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| .. _water_density_api: | ||
|
|
||
| Water Density | ||
| ============= | ||
|
|
||
| .. module:: mlipaudit.benchmarks.water_density.water_density | ||
|
|
||
| .. autoclass:: WaterDensityBenchmark | ||
|
|
||
| .. automethod:: __init__ | ||
|
|
||
| .. automethod:: run_model | ||
|
|
||
| .. automethod:: analyze | ||
|
|
||
| .. autoclass:: WaterDensityResult | ||
|
|
||
| .. autoclass:: WaterDensityModelOutput |
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| .. _density: | ||
|
|
||
| Density | ||
| ======= | ||
|
|
||
| Purpose | ||
| ------- | ||
|
|
||
| This benchmark assesses the ability of machine-learned interatomic potentials (**MLIP**) to | ||
| reproduce the **equilibrium density** of molecular liquids. Density is a fundamental | ||
| thermodynamic property: a model that predicts accurate local structure (see the | ||
| :ref:`radial_distribution` benchmark) may still get the density badly wrong if the simulation | ||
| box expands or collapses. Reproducing the correct density is therefore a complementary and | ||
| necessary check on the physical realism of a liquid-phase simulation. | ||
|
|
||
| Description | ||
| ----------- | ||
|
|
||
| The benchmark runs the same **MD** simulation as the :ref:`radial_distribution` benchmark: an | ||
| **NPT** simulation using the **MLIP** model for **500,000 steps**, leveraging the | ||
| `jax-md <https://github.com/google/jax-md>`_ engine from the | ||
| `mlip <https://github.com/instadeepai/mlip>`_ library. Water is run at **295.15 K** and **1 atm**, | ||
| while all other solvents are run at **293.15 K** and **1 atm**. Because the RDF and density | ||
| benchmarks of a system share their input systems and simulation output, the simulation is only | ||
| run once when both benchmarks are run together. | ||
|
|
||
| The density of each frame is computed from the (fluctuating) simulation cell volume: | ||
|
|
||
| .. math:: | ||
|
|
||
| \rho = \frac{N_\text{mol} \, M}{N_A \, V} | ||
|
|
||
| where :math:`N_\text{mol}` is the number of molecules in the box, :math:`M` is the molecular | ||
| weight, :math:`N_A` is Avogadro's number and :math:`V` is the cell volume. The **equilibrium | ||
| density** is taken as the average density over the final four fifths of the trajectory (the | ||
| first fifth is discarded as equilibration). | ||
|
|
||
| Dataset | ||
| ------- | ||
|
|
||
| The benchmark uses the same equilibrated input boxes as the :ref:`radial_distribution` benchmark | ||
| (a 500-molecule TIP3P water box, and methanol / acetonitrile / CCl4 boxes built with the GAFF | ||
| force field in OpenMM). | ||
|
|
||
| Reference densities are the experimental values at the simulation conditions: water | ||
| :math:`0.9978\ \text{g/cm}^3`, CCl4 :math:`1.594\ \text{g/cm}^3`, methanol | ||
| :math:`0.791\ \text{g/cm}^3` and acetonitrile :math:`0.786\ \text{g/cm}^3`. | ||
|
lwehrhan marked this conversation as resolved.
Outdated
|
||
|
|
||
| Interpretation | ||
| -------------- | ||
|
|
||
| Performance is quantified by the **density deviation**, the absolute difference between the | ||
| equilibrium density and the experimental reference. The deviation should be **as low as | ||
| possible**. The score is derived from the *relative* deviation (deviation divided by the | ||
| reference density) so that it is comparable across liquids of very different densities: a | ||
| relative deviation within roughly **2%** scores close to 1, decaying gently beyond that. (The | ||
| ideal per-solvent target is ultimately set by the isothermal compressibility of the liquid.) A | ||
| large deviation typically indicates that the box has expanded or collapsed during the | ||
| simulation, and the density time series can be inspected on the results page to diagnose this. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -93,6 +93,10 @@ class Benchmark(ABC): | |||||||
| If present, a user or the CLI can make use of this information to reuse | ||||||||
| cached model outputs from another benchmark carrying the same ID instead of | ||||||||
| rerunning simulations or inference. | ||||||||
| data_name: An optional name of the input data directory (and HuggingFace | ||||||||
| archive) to use instead of `name`. This lets several benchmarks share the | ||||||||
| same input data without duplicating it. Defaults to None, in which case | ||||||||
| `name` is used. | ||||||||
| """ | ||||||||
|
|
||||||||
| name: str = "" | ||||||||
|
|
@@ -105,6 +109,8 @@ class Benchmark(ABC): | |||||||
|
|
||||||||
| reusable_output_id: tuple[str, ...] | None = None | ||||||||
|
|
||||||||
| data_name: str | None = None | ||||||||
|
|
||||||||
| def __init__( | ||||||||
| self, | ||||||||
| force_field: ForceField | ASECalculator, | ||||||||
|
|
@@ -134,9 +140,9 @@ def __init__( | |||||||
| required elements. | ||||||||
| ValueError: If force field type is not compatible. | ||||||||
| """ | ||||||||
| self.run_mode = run_mode | ||||||||
| if not isinstance(self.run_mode, RunMode): | ||||||||
| self.run_mode = RunMode(run_mode) | ||||||||
| self.run_mode: RunMode = ( | ||||||||
| run_mode if isinstance(run_mode, RunMode) else RunMode(run_mode) | ||||||||
| ) | ||||||||
|
|
||||||||
| self.force_field = force_field | ||||||||
|
|
||||||||
|
|
@@ -225,17 +231,27 @@ def check_can_run_model(cls, force_field: ForceField) -> bool: | |||||||
|
|
||||||||
| return True | ||||||||
|
|
||||||||
| @property | ||||||||
| def data_dir(self) -> Path: | ||||||||
| """The local directory holding this benchmark's input data. | ||||||||
|
|
||||||||
| Uses `data_name` when set, otherwise `name`, so that benchmarks can share | ||||||||
| input data (e.g. an RDF and a density benchmark running the same system). | ||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nitpick: generalise
Suggested change
|
||||||||
| """ | ||||||||
| return self.data_input_dir / (self.data_name or self.name) | ||||||||
|
|
||||||||
| def _download_data(self) -> None: | ||||||||
| """Download the data from the data input directory if not already exists.""" | ||||||||
| already_exists = (self.data_input_dir / self.name).exists() | ||||||||
| data_name = self.data_name or self.name | ||||||||
| already_exists = (self.data_input_dir / data_name).exists() | ||||||||
| if not already_exists: | ||||||||
| hf_hub_download( | ||||||||
| repo_id="InstaDeepAI/MLIPAudit-data", | ||||||||
| filename=f"{self.name}.zip", | ||||||||
| filename=f"{data_name}.zip", | ||||||||
| local_dir=self.data_input_dir, | ||||||||
| repo_type="dataset", | ||||||||
| ) | ||||||||
| with zipfile.ZipFile(self.data_input_dir / f"{self.name}.zip", "r") as z: | ||||||||
| with zipfile.ZipFile(self.data_input_dir / f"{data_name}.zip", "r") as z: | ||||||||
| z.extractall(self.data_input_dir) | ||||||||
|
|
||||||||
| @abstractmethod | ||||||||
|
|
||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First use of "RDF" here. I guess above we should have