Currently each setting ("dataset") comes with its own Metric. This uses the PyTorch Lightning Metric API (https://torchmetrics.readthedocs.io/en/stable/). It might be, that this API is outdated, and it might also be that Metrics are consistent across datasets, hence leading to duplicity.
Goal:
- A better API for metrics, that is added to the Task part, like the losses.
- Independence from datasets
Culprits:
Currently each setting ("dataset") comes with its own Metric. This uses the PyTorch Lightning Metric API (https://torchmetrics.readthedocs.io/en/stable/). It might be, that this API is outdated, and it might also be that Metrics are consistent across datasets, hence leading to duplicity.
Goal:
Culprits: