OpenDA: update forcing with netcdf4

[vers-w]

OpenDA has upgraded it's netCDF library from netcdf3 to netcdf4.

When running Wflow from OpenDA with netcdf4, the Wflow forcing file is now locked. This can be resolved by setting the environment variable HDF5_USE_FILE_LOCKING=FALSE. With this setting OpenDA can update the forcing in the forcing file. However, the update is not picked up by Wflow, this will only work if the forcing file is closed and opened again by Wfow.

Possible solutions:

1. Reopen the file from Wfow after each OpenDA update. Disadvantage is that this will make Wflow slower and probably needs an OpenDA (or other application that updates file based dynamic input) specific implementation.

2. Update the forcing through BMI. Probably most clean solution. Wflow then needs to skip the update of forcing variable(s).

[visr]

How did you conclude that Wflow doesn't pick up HDF5_USE_FILE_LOCKING=FALSE? It should work without any special treatment in Wflow or NCDatasets, since we use the netcdf and hdf shared libraries. We probable have to make sure then environment variable is set before they are loaded though.

For instance this worked: meggart/NetCDF.jl#127 (comment)

[vers-w]

How did you conclude that Wflow doesn't pick up HDF5_USE_FILE_LOCKING=FALSE? It should work without any special treatment in Wflow or NCDatasets, since we use the netcdf and hdf shared libraries. We probable have to make sure then environment variable is set before they are loaded though.

I think this setting is picked up correctly by both Wflow and OpenDA based on testing by Erik Pelgrim (the file is not locked anymore). The issue is that the updated forcing data by OpenDA (written to the forcing file) is not picked up by Wflow directly (note that initialization has already occurred and Wflow did also open the forcing file). For this to work Wflow needs to re-open the file.

[visr]

Oh I see, yes that is tricky, I am not sure what we can guarantee about re-reading just modified files. Also see #1052.

[vers-w]

You're not sure what we can guarantee about re-reading just modified files because we support multiple forcing files? Or because it makes the model slower?

Other option (2) is to update the forcing through BMI.