Reduce computation time for some tests

[tlunet]

Currently, there are a few tests that take quite a long time. For instance, pySDC/tests/test_projects/test_TOMS/test_AllenCahn_contracting_circle.py::test_AllenCahn_contracting_circle takes quite some time using all available computation processes. Usually this happen when testing projects. Overall, since those are done for every push on every branch / forks, this seems a bit like a lot of energy consumption.

Would it be conceivable not to run all those (large scale) projects each time for testing, but consider a separate file that test the project on a smaller scale ?

[pancetta]

Yes. This is the problem with the current test suite: it mixes integration and unit tests in an unhealthy way. Ideally, we have a set of unit tests which run for each commit, very quickly, and which guarantee correctness of the separate components of pySDC. Then the integration tests run to check that the components work together as intended. And finally, acceptance tests, which are basically the projects.

This will be one part of the upcoming DFG proposal. Reorganizing the tests will be quite a bit of work, especially since pySDC does not have many unit tests at the moment (and the way it is written does not make creating those an easy task).

[tlunet]

Ok, hope this DFG proposal get accepted then 😅.

Quick question tho : what about testing components that require other components to work, for instance the Collocation base class that need Nodes and Lagrange class to work. Are those supposed to be unit tests, or integration tests ?

[pancetta]

Unit tests are supposed to be tests that are isolated and rather low level. They may dependent on other units, but should not cover interplay of components. However, these are not strict "rules", so one challenge is to define what a unit test could look like for pySDC.

To me, this would cover tests like "does this sweeper do what it's supposed to do", without checking for convergence, iteration counts, errors, ... I.e. all core and implementation classes tested independently, using a given set of expected outcomes. So, testing the collocation class is a unit test to me.

[pancetta]

Integration tests then put together these components to do meaningful stuff, i.e. in the controller classes and maybe the tutorials. Acceptance tests are the projects, I'd say.