Add preprocessing functions#205
Conversation
Also fix issue by respecifying the netcdf backend
This makes all optional arguments consistent in needing to be passed by keyword. I think it also makes it clearer.
MarionBWeinzierl
left a comment
There was a problem hiding this comment.
Just had a quick look and some comments -- @surbhigoel77 , could you have a more thorough look through this PR?
Return cf.Field objects instead of size-1 lists. Also allow size-1 lists to be passed as inputs to functions that expect single fields.
|
In fc12ab3 I have also added a correction for what I suspect is an error in This seems to resolve the issue in #156 when combined with updates to TSTORMS. This will be done in another PR. |
| list[cf.Field] | ||
| The list of fields read from the input files. | ||
| """ | ||
| fields = {field.nc_get_variable(): field for field in read_files(input_files)} |
There was a problem hiding this comment.
Maybe we can add a check before we build the dict that raises ValueError error if there is a missing variable name
There was a problem hiding this comment.
Perhaps I'm misunderstanding but it is already checking if any output variable names are missing in the loop below.
I don't think it would be easy to do it before building the dict because it is only after reading the input files that we can know which variables are available.
| # Negate the curl due to a suspected error in cf.curl_xy for spherical polar coords | ||
| # (In the first term the gradient is taken wrt latitude, not theta) | ||
| # (The second term is not negated) | ||
| curl.data = -curl.data |
There was a problem hiding this comment.
Not quite clear on why we are negating the curl?
There was a problem hiding this comment.
I found this formula of curl :
curl(u, v) = (1/r) * dv/dλ − (1/r·cosφ) * d(u·cosφ)/dφ
As per your comment, only first term gradient needs negation due to the opposite direction of latitude and theta. Why would we negate the entire curl?
There was a problem hiding this comment.
cf.curl_xy uses the spherical polar form of the equation:
So the first term is negative because the derivative is taken in the code wrt latitude rather than
For the second term, I mistakenly thought it had been added rather than subtracted. The actual issue is that
So overall it is calculating the negative of the curl, hence the need for the negation.
I will open an issue on cf-python regarding this.
MarionBWeinzierl
left a comment
There was a problem hiding this comment.
I am happy from my part, but leave it to @surbhigoel77 to accept this PR after she has done another review
Adds preprocessing functions for all of the topics in the 'data preprocessing' page of the docs. Intended for use with the batching (#194). Generally the functions are designed to accept either file paths or
cf.Fieldinputs and return single or lists ofcf.Field. Each function can also write an output file by providing theoutput_fileargument.src/tctrack/preprocessing.pyesmpyis installed for regridding. Update the docs to reflect that this is now a "proper dependency".Closes #189