Add basic means to force promote variables to parallel regions#3431
Add basic means to force promote variables to parallel regions#3431MetBenjaminWent wants to merge 29 commits into
Conversation
|
Thanks @MetBenjaminWent , I agree we need this (the ability to specify force_private for region transformations now that the region directive supports it and by extension the MaximalOMPRegionTrans). To be ready to merge we will need:
|
Thanks Sergi, happy to have a plug away with these, I'm glad it's in the ballpark! I'm still getting a little used to the |
|
Few comments on what I think we should also have in this PR since this is something we want to address automatically with improvements in the current work packages.
From a code perspective this is a bit more challenging - MaximalRegionTrans is not a great transformation to support this option as we may have MaximalRegionTrans inherited children (e.g. the ProfileRegionTrans in the nemo examples) that do transformations that don't support this option. Instead it should only be on MaximalOMPParallelRegionTrans. The challenge is how to affect the call to instead of the current apply and this can generalise it to any child, and the @sergisiso Does this seem reasonable to you? The The one minor thing is that technically Note that in general if an option is defined on the superclass you don't need to (and usually shouldn't) define it on the child class, instead use |
Thanks for the insight, happy to open an issue which captures this a little deeper, especially where #598 doesn't quite cover it. |
This is why I was mentioning to propagate kwargs, wouldn't it be enough to do |
No, since |
|
Is it because if kwargs reach the root class without being processed there is an error? e.g. we can not have ignored kwargs? |
Kinda - we made a decision we wanted to validate input options to make sure that provided options were correct, expected and type checked, so if you did e.g. |
|
Can we do |
|
@MetBenjaminWent Maybe we can split the PR into 2, keep this to add force_private to OMPParallelTrans (it needs to be the OMPParallelTrans and not the ParallelRegionTrans because only the OMP has the explicitly_private_symbols set) and add some tests for it. Then me and @LonelyCat124 can discuss in a separate PR how to best propagate the kwargs to it. What do you think? |
The issue is more when we call |
|
Now that I've caught up on the context, and had a read through each of the 4x classes which inherit A dict is certainly one way through and probably the safest. |
|
I think I also need to pipe
It doesn't seem to be recognising that the option in the |
|
@MetBenjaminWent This is the problem that @LonelyCat124 was warning about, essentially kwargs are passed to both, the self transformation in |
@LonelyCat124 I am not proposing to avoid this validation, I am saying something like Of course this needs to be done also in the validate so it probably could be a method that splits into both subsets of kwargs, but this is the idea. PS: In fact it can be a utility method provided in the base Transformation class to be used by any metatransformation, accepting a general kwarg and n-transformations classes and returning n-subsets of kwargs (but we can do that in a separate PR) |
|
I've had a thought which works around it, and partly answers how we control whats passed through If we add the new function to We can also do some checks in |
I think if you make a metatransformation it would work anyway (i.e. if you have I don't like the and let the self.validate raise the failures (as it can give errors for more than one invalid argument). It also raises the question of (admittedly bad implementations) but if option names are shadowed we might get unexpected non-failing results. This shouldn't happen but its something we should be careful of from a software design standpoint I think. I have a slight reservation with just automatically making all of the options of |
Ah, we had a naming confusion here, I was using metatransformations to mean transformations that call other transformations, not that inherit from multiple transformations. I haven't though about apply/validate MRO (but I don't think this PR should go that far)
What about: This would only remove self_kwards IFF they are in _trans and not self, so self.validate_options still works as expected for invalid options.
I don't think that's true, it does full testing of all options some through (the trans.validate is missing but I think we need it there otherwise we validate a potentially different thing) |
@sergisiso, I think relative to work remaining in this PR I just need to look at some new tests now, and this is otherwise ready I believe. |
|
Can to check if it works if you revert your changes to |
…Syclone into force_private_parallel
|
@sergisiso and @LonelyCat124, I think this PR is ready to go, I've added a new test to check that these are added. I'm not sure where my privileges are with the repo, it looks like I cannot add reviewers, and the checks are stuck pending still? Thanks! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3431 +/- ##
=======================================
Coverage 99.96% 99.96%
=======================================
Files 391 391
Lines 54668 54690 +22
=======================================
+ Hits 54649 54671 +22
Misses 19 19 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Re tests, I'm not yet sure how to cover one of the exceptions in: Codecov I suspect through, after some playing around, the transformation would have already exited by this time, which means it might not be needed at all? This one though Codecov, I should be able to do something in the parallel region testing |
I can't see how the first one could raise a TypeError unless i'm mising something. |
Ah cool, so if it always returns none, I'll just remove the try! |
|
This should be ready for review. As noted, I don't think I can test what codecov is upset about. |
|
@MetBenjaminWent I mocked up an idea on how to test the uncovered code, let me know if you have any issues with it as I haven't tested it properly since I didn't setup an env with your branch yet: |
LonelyCat124
left a comment
There was a problem hiding this comment.
Functionality looks good now, but there are code style/formatting issues. I've asked Sergi one other code-style question, which I'll update this PR if we have consensus on but otherwise its just the fixes here.
| @@ -40,6 +40,7 @@ | |||
| # J. Dendy, Met Office | |||
There was a problem hiding this comment.
Add yourself to the Author lists on the files modified.
| ''' | ||
| Surrounds the provided node list with an OpenMP Parallel region. | ||
|
|
||
| :param nodes: list of Nodes to put within parallel region. |
There was a problem hiding this comment.
Document force_private.
There was a problem hiding this comment.
(But only on this transformation).
| if force_private: | ||
| new_region_directive = nodes[0].ancestor(RegionDirective) | ||
| if new_region_directive: | ||
| region_set = super()._check_symbol_table_vars( |
There was a problem hiding this comment.
use self._check_symbol_table_vars, no need to super class this.
| logger="psyclone.psyir.transformations"): | ||
| otrans._check_symbol_table_vars(parallel, ("j")) | ||
|
|
||
| long_string = \ |
There was a problem hiding this comment.
Instead of \ use (), i.e.
long_string = (
"Could not find 'j' in the ..."
"provided with the ..."
)
| "\"Could not find 'j' in the Symbol Table.\" has been "\ | ||
| "provided with the 'j' symbol name in the 'force_private' option, "\ | ||
| "but there is no such symbol in this scope." | ||
| assert (long_string in caplog.text) |
There was a problem hiding this comment.
Don't need parenthesis on the assert.
| assert len(psyir.walk(OMPParallelDirective)) == 1 | ||
| nodes = psyir.walk(Routine)[0].children[:] | ||
| assert isinstance(nodes[0], OMPParallelDirective) is True | ||
| pdir = psyir.walk(OMPParallelDirective) |
There was a problem hiding this comment.
Just use pdir = nodes[0] since you've asserted where it is already, will be more efficient than a big walk.
| psyir = fortran_reader.psyir_from_source(code) | ||
| # Apply loop_trans to all the loops possible. | ||
| ltrans = OMPLoopTrans(omp_schedule="static") | ||
| for loop in psyir.walk(Loop): |
There was a problem hiding this comment.
Any reason for this rather than just ltrans.apply(psyir.walk(Loop)[0]), ...)? This feels tidier to me if its the same functionality.
| # This is not an error, but we will log the missed string | ||
| logger = logging.getLogger(__name__) | ||
| logger.warning( | ||
| "%s has been provided with the '%s' symbol name in " |
There was a problem hiding this comment.
Can you use fstring here isntead of formatted strings please? I.e.
f"{err} has been provided with the '{symbol_name}' ..."
Out rules also that each line of the f string must begin f" even if no replacements are in that section of the string.
|
Many thanks Aidan for the CR, I'll get these updated! |
@MetBenjaminWent One other thing from discussion from Sergi, we prefer this format: for calling or defining functions - can you change that in your code as well? |




Allow users to force promote into parallel sections with
force_private, specifically fromMaximalOMPParallelRegionTrans, but implemented intoParallelRegionTrans, so all parallel regions can use the functionality, such as when called viaOMPParallelTransI've been having another play with master PSyclone, and the example recommended by Sergi, works nicely, but unfortunately doesn't quite cover everything currently.
It would seem we need a means to tell the
MaximalOMPParallelRegionTransthat we want things to be private.In
bdy_impl3.F90(I'm just prototyping with this one as its representative of much of the boundary layer, but also shorter), its a spanned parallel section with OMP do(s).When running it through a script based on this example, we need to ignore the dependencies on these variables for the new ii blocking loops.
(https://github.com/MetBenjaminWent/lfric_apps/blob/4834ffccfc8999b6482a6f2a94accc207a8b58fc/applications/lfric_atm/optimisation/meto-ex1a/transmute/boundary_layer/bdy_impl3.py)
However, some arrays like
temportemp_outneed to remain private, and the spanned parallel section defaults them to shared.