VERSIONS.md

Mass Versions

Mass Version 2 (2025-present)

2.0.4 April 14, 2026

• Add mass2.Channel.from_numpy() to read optical TES raw data. Tests to go with it.
• Some improvements to plotting:
  • Remove plt.tight_layout() in mass; prefer user set figure.constrained_layout.use : True in matplotlibrc.
  • Set most mass plots to call plot_zoomable, which turns on mouse zooming (same as typing 'O' in window).
  • Fix bugs: failing to plot channel histograms on existing axes.
• Make the parameters cut_pre and cut_post work with Fourier-domain filters.
• Make a recipe step to use the time-drift-correct algorithm (PR 127, issues 125+131).
• Make Channel.filter1lag() method to use single-lag optimal filters more easily (issue 124).
• Add options to the external-trigger reader, letting user pick what quantities to keep (issue 128).

2.0.3 March 13, 2026

• Add some convenient plotting features like Channel.plot_pulses().
• Add secondary x axis to that method and to Channels.plot_avg_pulse() to give time in ms.
• CI testing: use Python 3.10 and 3.14 (replacing 3.13).
• Add resample_pulses and resample_one_pulse to mass2.core.analysis_algorithms.
• Add an example notebook examples/f3db_sweep_example.py with multiple opt filters (PR 114).
• Interpret all times in the local timezone. Add ChangeTimeZoneStep to change that (PR 115).
• Let Channel.filter5lag() user can elect Fourier-based filters for very long records (PR 117).
• Check on Channel creation that raw pulses (if any) match the header's n_samples (issue 118).
• This is the last release fully reviewed by project Co-Director (now Emeritus) Galen O'Neil.

2.0.2 February 24, 2026

• Adapt to change in the numpy.linalg.svd API (issue 97).
• Add one-lag optimal filters for optical TESs (issue 99).
• Fix problem reading external trigger data with LJH files (issue 101).
• Seven security updates to uv.lock prompted by GitHub's "dependabot"

2.0.1 February 12, 2026

• Work around Polars bug: can't group_by a frame with an "index" column (issue 85).
• Make the OptimalFilterStep.dbg_plot() generate a new figure (issue 83).
• Fix combinatoric rough cal with one peak (PR 92).
• Add argument to Channel.plot_scatter() to plot some max number of points (PR 94).
• Fix combinatoric rough cal with two peaks and increasing gain (issue 95).

2.0.0 November 4, 2025

• Add a way to open LJH files for specific channels only, an include list (issue 81).

2.0.0 beta 3 September 19, 2025

• Added a way to generate Arrival-Time-Safe filters and make into a RecipeStep (issue 67).
• Made GitHub Actions CI system cache the big pulsedata requirement (issue 61).
• Require numpy version >= 2.0.
• Improve README installation instructions.
• Add a save/restore function for in-progress analysis (issue 70).
• Add mouseover and clickable features to scatter plots (issue 71).
• Restore analysis also finds LJH files and memmaps their raw data fields (issue 77).

2.0.0 beta 2 September 3, 2025

• Add convenience functions to return and plot key facts from a Channel's Recipe (issue 59).
• Add regression tests like Mass1 had (issue 62).
• Finish documentation strings; reorg doc pages made from them (issues 18, 19).

2.0.0 beta 1 August 29, 2025

• Add Recipe.trim_dead_ends() to generate a recipe that removes steps not needed to reach a certain end (issue 38).
• Optionally drop debugging information from steps in trim_dead_ends() to make MUCH smaller recipe pickles (issue 40).
• Add sub-keV fluorescence line shapes learned from HAUSKat 2024 measurements on C, N, O, Fe, Ni, Cu (issue 37).
• Figured out how to render matplotlib images in documentation with mkdocs-matplotlib plugin (issue 47).
• Reorganized documentation and make a Getting Started page (issues 45+46).
• Added complete modern type annotations, to run with zero mypy errors (issue 56).

2.0.0alpha August 20, 2025

• Complete rewrite of MASS for version 2. Base data wrangling on Pola.rs.
• Allow Channel objects to have callable to transform raw data (e.g., by inverting it) (issue 12).
• Raise minimum python version to 3.10 for type annotations to work.
• Make mypy type annotation tests run without errors.
• Support reading LJH file version 2.2, 2.1, and 2.0 (issue 26), and also OFF versions 0.1, 0.2, and 0.3.
• Use mass1-style noise analysis, where (if available), continuous noise is used to estimate high-lag autocorrelation better (issue 24).
• Add FilterMaker.compute_5lag_noexp() and .compute_5lag_constrained() for constrained optimal filters (issue 29).
• Read external trigger times into the Mass2 dataframe (issue 28).

Mass Version 1 (2010-2025)

0.8.9 June 12, 2025-

• Modernize README with Conda instructions and other setup help (issue 335).
• Disfavor Conda installation in the README.
• Remove misleading use of the word "cython" in code and docstrings.
• Don't return memmapped objects from LJHFile properties, but copies (issue 337).

0.8.8 May 27, 2025

• Make Fourier-domain optimal filters work (issue 320).
• Add doctests for the new filtering API (issue 323).
• Add Thomas Baker's improved method for correcting jumps by 1 (or more) flux quanta (PR 325).
• Remove 16 warnings that arise in testing (PR 326).
• Fix external trigger processing (issue 327).
• Better error message when opening MASS data that is longer than an existing HDF5 file (issue 329).
• Fix problem with new flux-jump algorithm (issue 331).
• Fix problems with OFF analysis: calibrateFollowingPlan didn't work with new calibration API (PR 334).

0.8.7 January 24, 2025

• Remove some dependencies in pyproject.toml that are no longer required.
• Use uv to install requirements within GitHub Actions. Disable flake8 linting test, which is broken somehow.
• Refactor energy-calibration code to use modern Python and MASS practices, and less spaghetti (issue 317).
• Replace both Cython and pure-Python summarize_data modes with a Numba version (stolen from MOSS).
• Remove all other Cython code, too. There is no more Cython in MASS.

0.8.6 December 19, 2024

• Completely refactor filter code to use modern Python and MASS practices, and less spaghetti (issue 312). Some user code that involves filters might be broken.
• Move the use of optimal filters out of the MicrocalDataSet object into the Filter (issue 314).

0.8.5 November 15, 2024

• Fix problem where numpy 2.0 was failing a regression test. Not a true regression, so we broaden the acceptance criteria (issue 295).
• Add installation instructions to README for https-based installation, in addition to ssh-based.
• Fix Ruff errors.
• Fix bug when no number of rows is stored (for µMUX data) (issue 298).
• Allow setting channels bad when in noise-only mode (issue 301).
• Make plot_noise() have the right x-axis (frequency) values (issue 303).
• Add EnergyCalibration.ismonotonic() and test it (issue 305).
• Fix errors when an experiment state appears 2+ times, as "PAUSE" often does (issue 309).
• Start testing on Python 3.13 (was 3.12) and oldest valid version: 3.9.
• Fix errors in the computation of V/dV and estimated variance for optimal filters (issue 307).

0.8.4 June 5, 2024

• Fix test failures on Py 3.12: store HDF5 cache files in temp directories, so tests don't share them. (issue 272). Correctly use the modern tempfile library's API.
• Use dynamic version numbering through setuptools-scm (issue 287).
• Go back to working correctly with inverted data, even as a memmap (issue 286).
• Remove all use of the legacy tempfile package in our tests (issue 291).

0.8.3 May 14, 2024

• Read LJH file row number correctly (issue 268).
• Improve behavior of powerspectrum and ExperimentalFilter objects.
• Make tests pass under version 8 of pytest (issue 269).
• Move primary repository to GitHub. Update README.
• Set up GitHub Actions for our new hosting service (issues 273, 274)
• Make ruff warnings go away (issue 277).
• Update/fix how we save (pickle) OFF recipes.

0.8.2 February 23, 2024

• Fix bug in reading certain external trigger files (issue 262).
• Improve installation instructions in README (issue 261).
• Parse *external_trigger.bin file for assumed row count (issue 263).
• Add K-alpha lines of Sm, Gd, Tb, Dy, and Ho (issue 264).
• Introduce the idea of subframe divisions to get timing info a rates faster than the frame rate for µMUX (issue 265).
• Add Pr, Nd, and Eu K line shape models (issue 266).
• Run ruff and pycodestyle until warnings are gone or disabled one-by-one (issue 267).

0.8.1 February 4, 2024

• Fix bug in fits where E-response has exponential tails (issue 250). Ensure tail_tau has consistent meaning indepdenent of energy scale. So keV-scale lines like MnKAlpha have same tail_tau meaning as the MeV lines as used in TrueBq.
• GenericLineModel.guess now requires dph_de as an argument. This allows the guessed fwhm value to be in eV as intended, rather than arbs. This may break some user code.
• Change high-E exponential tails models, so we vary not the low and high tail fractions, but the total tail fraction and the share due to the high tail. Thus both can be fixed to the [0,1] range safely (issue 252).
• Add an example MnKAlpha analysis with normal mass.
• Updates to use external trigger in processing OFF files, made Oct 2023 at the EBIT.
• Avoid scipy version 1.11.2, but not earlier or later, as only 1.11.2 makes our tests fail (issue 253).
• Sort filenames correctly in bin/ljh_merge (issue 254).
• Don't ignore the use_only and never_use lists in TESGroup constructor when making a pulse-only dataset (issue 255).
• Allow bin/ljh_merge to accept multiple patters, enabling shell brace-expansion to work (issue 258).
• Fix bug in the p_peak_time property so that peak times less than 0 are correct (issue 259).

0.8.0 August 10, 2023

• Use np.memmap on LJH files as well as OFF (issue 160).
• Remove iter_segments() and internal use of read_segment().
• Start to use setup.cfg instead of Python code to configure build/install (issue 238).
• Clean out code from the nonstandard directory (issue 241).
• Move tests out of the installed mass package (issue 242).
• Use ruff instead of very slow pylint (issue 243).
• Use numpy's new API for random numbers (issue 244).
• Remove the CythonMicrocalDataSet class; replace with pure Cython functions (issue 245).
• Remove the deprecated non-LMfit-based MaximumLikelihoodFitter (issue 246).
• Replace all unittest with pytest-based testing (issue 247).
• Remove the monkey-patch to np.histogram required by numpy 1.13 (issue 248)
• Fix error in summarize_data(..., use_cython=False) for multi-segment LJH files (issue 249)
• Improved/updated documentation file Cuts.md, which has always bugged me.

0.7.11 June 2023

• Add overwrite parameter to phase and time-drift corrections.
• Update line shapes: Fe L line, ClKAlpha.
• Realtime state slice fix

0.7.10 June 2, 2023

• Fix issue 229: bug in writing bin/ljh_merge; was not writing timestamp and row count.
• Fix issue 230: add Sc Kα and Kβ data from 2019-2020 publications by Dean et al.
• Fix issue 231: document LJH files; other requests need no changes.
• Fix issue 234: make doctest fails in our pipeline tests.
• Fix issue 228+235: skip test of pickling recipe books, which pins dill and Python versions.
• Fix issue 212: call garbage collector to close memmap files when deleting off.ChannelGroup.
• Fix issue 239: could not open TESGroup(..., noise_only=True).

0.7.9 January 11, 2023

• Run bitbucket pipeline with python 3.7 and 3.10 instead of 3.7 and 3.9.
• Add compute_noise() replacing compute_noise_spectra(); deprecate the latter.
• Fix issue 223: add assume_white_noise so data can be filtered w/o noise files.
• Fix issue 225: bug in using LJH files where no non-trivial experiment state was set.
• Fix issue 226: repair some warnings appearing in the test suite.
• Run autopep8 (and add autopep8 target to Makefile)
• Make a new script bin/ljh_merge to merge multiple LJH files into one.

0.7.8 May 17, 2022

• Fix issue 213: ds.plot_summaries() should accept valid=b, a boolean array.
• Fix issue 214: verify that cutAdd(...overwrite=True) works for OFF analysis.
• Fix issue 215: add Kα lines of magnesium, aluminum, silicon from Ménesguen 2022.
• Fix issue 216: calibration should fail more obviously when data aren't monotonic.
• Fix issue 217: problem with rounding error giving negative values in fit model.
• Fix issue 218: distutils.version is deprecated.
• Fix issue 219: should raise Exception when MLEModel.fit(...weights=w) for non-None weights.
• Fix issue 220: read any experiment_state.txt file and make into categorical cuts.
• Fix issue 221: bug in plotting noise autocorrelation.
• Fix issue 222: bug in re-making filters and overwriting in HDF5 backup file.
• Put mass.materials in default sub-packages for pip installation.

0.7.7 November 29, 2021

• Fix issue 205: add line shapes of Se, Y, Zr K lines from Ito et al 2020.
• Fix issue 206: update calibration curves: uncertainty and better smoothing spline theory.
• Fix issue 207: update usage of numpy/Cython; standardize imports of numpy/scipy/pylab as np/sp/plt.
• Fix issue 208: allow save/restore of filters to HDF5 even when too long for HDF5 attributes.
• Fix issue 209: replace dependency xraylib with xraydb.
• Fix issue 210: add line shapes of Ir, Pt, Au, Pb, Bi L lines.
• Fix issue 211: hide math warnings during pytest testing.

0.7.6 November 24, 2020

• Fix issue 189: clean up top-level directory and pytest configuration.
• Fix issue 191: typos in our CrKAlpha line and apparent typo in Hölzer on the FeKAlpha line.
• Fix issue 192: some problem with using MLEModel for fits that aren't spectral lines.
• Fix issue 193: problem in fitting to histograms with dtype=np.float32 for the bin edges.
• Fix issue 194: triggers the too-narrow-bins fit error when it should not, if dPH/dE >> 1.
• Fix issue 196: reorganize x-ray filter code; add Horton 2018 design.
• Fix issue 197: work around a problem opening certain noise HDF5 files.
• Fix issue 199: remove Qt4 (a Python 2-only package) and GUIs based on it.
• Fix issue 200: work with h5py version 3 (reads strings as bytes type).
• Fix issue 201: add examples in line_fitting.rst to show how to fit Gauss, Lorentz, or Voigt
• Fix issue 202: fluorescence models use parameter "integral" instead of ill-defined "amplitude".
• Fix issue 203: autocal uses new-style fits.
• Fix issue 204: add optional quantum efficiency model to line fits: now can fit line times QE.
• Replace line fitting documentation with a doctest document.

0.7.5 March 31, 2020

• This is the last version that supports Python 2.7.
• mass.spectra contains dictionary of class instances instead of class objects.
• Fix long lines and other pep8 warnings. Fix the Make targets pep8 and lint for Python3.

0.7.4 March 26, 2020

• Make mass installable by pip.
• Add entry points so some scripts are automatically installed.
• Fix minor annoyances and pep-8 violations.
• Fix issue 187: cannot plot_average_pulses or plot_filters.
• Fix issue 188: annoying log messages upon reading older LJH files.

0.7.3 January 2020-March 2020

• Reorganize code to live in mass/... instead of src/mass/...; always test installed Mass.
• Can now use "setup.py develop" to link site-packages back to local mass, so "installed Mass" can be your local.
• Fix issue 172: add an intrinsic Gaussian in SpectralLine: components can now be Voigts, not Lorentzians.
• Spectral lines can have a 2-sided exponential tail, not only a low-energy tail.
• Instantiating a GenericLineModel with the default has_tails=False forbids exponential tails, taking a shortcut around that (slow) code.
• Add doc/LineFitting.md to document use of the new fitting Models.
• Fix issue 165: use of LMFIT for fitting is now the default. Deprecate the old *Fitter objects.
• Fix issue 163: problems when the scaling of parameters (or their uncertainty) varies widely between parameters.
• Start using pytest for package testing instead of our own runtests.py script.
• Fix issue 181: deprecated import imp was removed; no longer needed with pytest.
• Remove global dictionaries fitter_classes and model_classes. Replace with functions make_line_fitter and make_line_model that generate instances from a SpectralLine object. Fixes issue 182.
• Fix issues 183-184: problems in mass.off.

0.7.2 December 2019-January 2020

• Allow ArrivalTimeSafeFilter to be full length, skipping the "shift-1" possibility that DASTARD doesn't need.
• Refactor calling MicrocalDataSet method compute_filter() into compute_5lag_filter() and compute_ats_filter().
• Add scripts/ljh2off.py and MicrocalDataSet.projectors_to_hdf5()
• Fix issue 177: read new and old stored filters.
• projectors include v_dv, noise_rms_median, noise_rms_sigma
• refactor projectors to pulse_model.PulseModel
• ExperimentStateFile stores unique states as slices
• faster getOffAttr (OFF file performance) for lists of 100s of slices
• data.refreshFromFiles added, this parses the experiment state file from the last point it was parsed to, then updates the off file mmaps, then recalculates all ds._statesDict based on the new states and info in the OFF file.
• Has tests on the internals of ExperimentStatefile and OffFile.
• Fix reading in Py3 of HDF5 calbrations written by Py2, and other Py3 problems.
• Silent testing.
• Support OFF files version 0.3.0.

0.7.1 November 2019-December 2019

• More neon HCI lines.
• compute_newfilter(...) takes option to turn off shift-by-1 (useful when working on Dastard data).
• Fix issue 175: use of basestring isn't Python 3 compatible.
• Fix issue 176: band limited ArrivalTimeSafeFilter were not DC-insensitive.
• In the above, we fundamentally changed the exact computation of a filter so filtered data will be changed by this version of MASS.

0.7.0 December 2018-October 2019

• Add partial use of LMFit for fitting.
• Add code for analysis of OFF files.
• Add K line shapes for elements K, S, Cl. Also Nb (K-beta only).
• Add fitters for certain highly charged ions from NIST Atomic Spectra Database.
• Many other small changes, particularly for use at the EBIT.

0.6.6 August-October 2019

• Fix issue 162: overcome biased fits when bins are too wide by allowing numerical integration in bin. How densely to sample can be chosen by user or a heuristic.
• Fix issue 164: silently "correct" the off-by-3 error on nPresamples in MATTER-produced LJH files.
• Fix issue 167: add molybdenum L line data from Mendenhall 2019.
• Fix issue 147: auto_cuts fails if other cuts already exists.
• Fix issue 140: pass tests on Python version 3.
• Fix issue 166: could not run phase_correct twice on same data (HDF5 error).

0.6.5 July 2018-June 2019

• Factor phase correction into its own class and source file.
• Add lineshape models for bromine K and tungsten L lines.
• Allow read-only opening of HDF5 files (don't try to update attributes in that case).
• Fix bugs in EnergyCalibration.plot()

0.6.4 May-July 2018

• Fix issue 156: phase correction with low statistics.
• Fix issue 157: noise fails if noise records aren't continuous.
• Fix issue 158: certain test failures.
• Make phase_correct method2017=True the default.
• Pep-8 fixes.

0.6.3 May 2018

• Refactor Fluorescence Line and Fitter Code.
• Fix incorrect fitters where Chantler reported Voigt peak height instead of area, such as VKalpha.
• Add fluorescence.md doc file.

0.6.2 November 2017 - April 2018

• Fixed setup.py to be compatible with Pip 10.0.0+.
• Fixed some minor bugs and redundancies.
• Added ToeplitzWhitener.solveW method and tests for ToeplitzWhitener.
• New filtering API: f_3db and fmax are set only at filter.compute() time.
• Added arguments to ArrivalTimeSafeFilter.compute() so you can emulate shorter records.
• Change plot_average_pulses and plot_noise to leave out bad channels by default.

0.6.1 September-November 2017

• Added some features to support analysis of microwave MUX data.
• Added some random-matrix techniques to mass.mathstat.utilities.
• Used a decorator to add methods to TESGroup that loop over channel objects.
• Galen added configuration of Sphinx to auto-generate HTML documentation from the docstrings.

0.6.0 September 2017

• Fixing a number of outstanding problems (issues 88-100 at least).
• Fixed problems in new-style filtering.
• Removed requirement that user specify a peak time.
• Lots of cleaning up: Remove code that is not still used or correct, or move it to nonstandard. Remove julia subdirectory and workarounds.py.

0.5.3 March-April 2017

• Joe added entropy, cross-entropy, and Kullback-Leibler divergence computation on distributions assuming a Laplace kernel-density estimator.
• Added a new, tentative "method2017" for phase correction.
• Also, bug fixes.

0.5.2 February 2017

• Young is working on some things.
• Joe added LJH modifier.
• Deprecated ROOT file format handler.

0.5.1 November 2016

• Fix a bug in how filters are restored from disk (issue 82).

0.5.0 October 2016

• Galen reorganized the source tree. That alone seems like worth a minor version number.
• Also created some 69 regression and other unit tests, which all pass now. Several bugs were fixed in the process.
• Added file pattern-matching to TESGroup constructor.
• New-style optimal filters became the default.
• Fixed issues 60, 62, and most of 70-81.

0.4.4 August 2016

• Young changed the version number, but I (JF) do not know why.

0.4.3 May 2016

• Reorganized code that fits spectral line models (Voigt, Gaussian, and specific Kα or Kβ lines).
• Added low-E tails to the Voigt and Gaussian fitters.
• Fixed issues #45-51, except for 48.

0.4.2 October 2015

• Return the main objects to pure Python, with Cython subclasses. Also not sure this is a good idea, but we'll experiment with it.

0.4.1 October 1, 2015

• Uses pure Cython for channel.pyx. We decided that this experiment was not a positive development, but it was worth trying.

0.4.0 September 2015

• New filtering methods available from Joe. Still experimental! Don't set MicrocalDataSet._use_new_filters = True unless you know what you're doing!