Skip to content

feat: record training history#181

Closed
JemmaLDaniel wants to merge 1 commit into
mainfrom
feat-record-training-history
Closed

feat: record training history#181
JemmaLDaniel wants to merge 1 commit into
mainfrom
feat-record-training-history

Conversation

@JemmaLDaniel

Copy link
Copy Markdown
Collaborator

Record training history during calibrator fitting

Summary

Adds a TrainingHistory dataclass and integrates it into the calibrator's fit method so that training metrics (loss curve, validation scores, iteration count) are captured and persisted after training. This enables post-hoc analysis of training convergence and makes it easier to diagnose issues like underfitting or early stopping behaviour.

Changes

  • winnow/calibration/calibrator.py:
    • Adds TrainingHistory dataclass with fields: loss_curve, validation_scores, final_training_loss, final_validation_score, n_iter.
    • TrainingHistory.save(path) / TrainingHistory.load(path) — JSON serialisation for persisting histories.
    • TrainingHistory.plot(output_path, show) — matplotlib visualisation of loss and validation curves.
    • Calibrator.fit() now returns a TrainingHistory instance extracted from the fitted MLPClassifier.
  • winnow/calibration/__init__.py — re-exports TrainingHistory.
  • winnow/scripts/main.py — the train CLI command now saves the training history JSON alongside the model output.
  • winnow/configs/train.yaml — adds history_output config key for specifying the history file path.
  • docs/api/calibration.md — documents TrainingHistory and its methods.
  • tests/calibration/test_calibrator.py — tests that fit returns a TrainingHistory with expected fields, and tests for save/load/plot round-tripping.

@JemmaLDaniel JemmaLDaniel self-assigned this Apr 10, 2026
@JemmaLDaniel JemmaLDaniel added the enhancement New feature or request label Apr 10, 2026
@github-actions

Copy link
Copy Markdown

Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py00100% 
data_types.py40100% 
calibration
   __init__.py20100% 
   calibration_features.py316797%247–248, 443, 728, 916, 920, 1215
   calibrator.py1501689%158, 212–213, 215, 249–252, 277–278, 280, 305–306, 310, 337–338
compat
   __init__.py00100% 
   instanovo.py10640%12, 14–15, 17, 24–25
datasets
   __init__.py00100% 
   calibration_dataset.py1091784%155, 169, 171, 173, 183, 196, 249, 251–252, 258–261, 263–266
   data_loaders.py2701494%23, 189, 220–221, 414, 455, 847, 851, 900, 911, 1023–1024, 1052–1053
   interfaces.py30100% 
   psm_dataset.py250100% 
fdr
   __init__.py00100% 
   base.py581574%81, 85–86, 91, 98–99, 105, 126, 129–130, 135, 137–138, 144, 186
   database_grounded.py28196%52
   nonparametric.py25484%62, 68–69, 72
scripts
   __init__.py00100% 
   main.py1911910%8, 10–13, 16–20, 23–24, 26–28, 32, 39, 44, 47, 53, 55–56, 59, 68, 76, 79, 86, 88–90, 92, 94–99, 102, 104–105, 110, 125, 128, 135–141, 144–145, 148, 161–163, 166, 169, 174, 176–178, 180, 182–183, 186–187, 190, 192–193, 195, 197, 199–200, 202, 205–206, 209–210, 213–216, 221–222, 225–226, 229–231, 233, 236, 250–252, 254, 256, 261, 263–265, 267–268, 270–272, 277–278, 280–282, 284, 286, 288–289, 293–296, 298–299, 301–302, 304–305, 307, 310, 324–326, 329, 332, 337, 339–341, 343–345, 347–348, 351–352, 355, 357–358, 360, 362, 364–365, 367, 370–371, 377–378, 381–382, 385–386, 389–390, 398–400, 404, 407, 411, 414, 437, 450–451, 454, 476, 488–489, 492, 517, 530–531, 534, 549, 561–562, 565, 577, 589–590, 593, 608, 620–621
utils
   __init__.py40100% 
   config_formatter.py534024%29, 37–38, 40–42, 44, 55, 58–60, 62–63, 66–69, 72–74, 77–78, 80, 91, 102, 113, 127–128, 130–132, 145–147, 150, 153–154, 157–158, 160
   config_path.py76593%24–26, 117–118
   peptide.py160100% 
TOTAL134031676% 

Tests Skipped Failures Errors Time
299 0 💤 0 ❌ 0 🔥 36.751s ⏱️

@JemmaLDaniel

Copy link
Copy Markdown
Collaborator Author

This PR is encompassed in #190

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant