fix(moench): Mosaic3 all-on DMD (scalar SLMImage no-op) + premature stim-mask build + napari live-restart during MDA - #17
Open
hinderling wants to merge 4 commits into
Conversation
A scalar-bool SLMImage reaches the base engine as setSLMPixelsTo, which the Mosaic3 ignores without raising, so the previously latched pattern stays on the mirrors. Expand the scalar to a uint8 array in MoenchMDAEngine._set_event_slm_image so it routes through setSLMImage. This makes an intended all-off actually clear the DMD, and re-displays the pattern on every imaging capture instead of relying on the latched state surviving between frames.
The feed loop runs a few events ahead of the camera, so it reached a stim event and blocked on get_stim_mask for a frame that had not been shot yet. At minute-scale intervals the mask wait expired before the frame existed and the stim fell through to an all-off mask. Track which (t, p) imaging frames have reached the pipeline and wait for a stim's source frame ((t-1, p) in "previous" mode, (t, p) in "current") before asking for its mask. A frame that never arrives cannot have a mask, so a timed-out wait goes straight to the all-off fallback rather than spending the mask timeout again on a lookup that must fail.
… to live napari-micromanager can hold a live timer while the stream is already stopped, and that timer restarts live acquisition on the per-frame configSet/exposureChanged signals the engine emits, which then fights every snap. Stop continuous acquisition at MDA start without gating on isSequenceRunning(), since the emitted sequenceAcquisitionStopped is what clears the timer. Doing it in the engine covers calibration and bare mmc.mda.run() as well as experiments run through the Controller. Drive KeepDMDAlive from the live-acquisition signals rather than starting it at boot and bracketing every MDA: run() on live start (now idempotent), stop() on live stop (now a no-op when idle). The engine drives the DMD every event during a run, so no keep-alive is needed and there is no stop/restart pair left to race on a cancelled run.
hinderling
force-pushed
the
fix/mosaic3-slm-and-stim-gate
branch
from
August 5, 2026 22:43
2f1adb5 to
6e1c9e3
Compare
Projecting the calibration/test spots with one mmc.mda.run per event brackets each spot with setup/teardown_sequence, which stop and restart KeepDMDAlive; each restart re-displays the all-on live pattern and resets the SLM ExposureTime, so under OverlapMode a spot can end up on a short exposure that blanks before the camera opens. Run every event in one MDA (with hardware sequencing off -- otherwise pymmcore-plus tries to combine the consecutive SLM events and fails validation) so KeepDMDAlive pauses exactly once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three Moench (Andor Mosaic3 DMD) acquisition-reliability fixes, all confirmed on the scope. The first two also exist in
main, so feeding them back; the third (§3) is new.1.
fix(moench): expand scalarSLMImageto an array so the Mosaic3 actually updatesOn the Mosaic3,
core.setSLMPixelsTo(dev, N)is a silent no-op — no error, no effect; the previously-latched pattern stays displayed. The baseMDAEnginerenders a scalar-boolSLMImage(data=True/False) viasetSLMPixelsTo, so every scalar SLM command was doing nothing on this hardware. Two symptoms, one root cause:_build_stim_slmsendsSLMImage(data=False); the intended all-off never reached the DMD, so the latched all-on fired the UV full-field instead of the targeted pattern.MoenchMDAEngine._maybe_inject_dmd_wake_slminjects a scalarSLMImage(data=True)to hold the DMD all-on for each imaging capture — also a no-op, so imaging relied on the latched all-on plus the OverlapMode re-pulse and the 200 s hold. Without an active re-display each frame, the mirror occasionally parked mid-capture and the frame came back blank.Fix: in
MoenchMDAEngine._set_event_slm_image, expand a scalar-boolSLMImageto a fulluint8array before delegating, so it routes throughsetSLMImage(arrays are the only reliable Mosaic3 update path). This actively re-displays the intended pattern every frame, pinning the mirror against the blank-frame parking. Contained entirely in the microscope-specific engine as a hardware workaround.Not addressed by this fix: a separate every-other-frame imaging dimming — a ~0.74x signal-only even/odd (fluorescence scales, background stays flat, in focus) that recurs intermittently on some runs — is still open and under investigation. It is distinct from the all-on / blank-frame symptoms above; do not read §1 as closing it.
Diagnostic tell for old data: a stim readout channel that is full-field (≈100% of pixels lit) is a pre-fix run; targeted thin lines mean the fix was active.
2.
fix(controller): gate stim SLM build on predecessor frame acquisitionThe feed loop (
_run_mda_with_events) runs several events ahead of the camera (qsize<3 backpressure). Inpreviousstim mode it built a stim event's SLM — blockingget_stim_mask(t-1)— long before framet-1was acquired. At minute-scale intervals with slow segmentation, the mask wait (80 s) expired before the predecessor frame even existed, so_build_stim_slmreturnedSLMImage(data=False)(which, combined with bug 1, fired the DMD's stale pattern).Fix: track the
(t, p)of each acquired imaging frame and, before building a stim's SLM, wait for its predecessor to be acquired —(t-1, p)inpreviousmode,(t, p)incurrentmode.get_stim_maskthen only waits out segmentation of an already-acquired frame, and the 80 s timeout becomes a genuine pipeline-stuck detector again. No deadlock: the predecessor is always queued in an earlier feed-loop iteration; the wait is cancel/fatal-aware, mirroring the existingWaitEventacquisition gate. This is pure feed-loop scheduling — the MDA engine stays microscope-agnostic.3.
fix: stop live acquisition before every MDA; drive KeepDMDAlive from live eventsA third Moench acquisition-reliability fix in the same DMD / live-view area.
napari-micromanager restarts live mid-MDA → dropped/corrupted frames
With
napari-micromanagerattached to the same core (the usual Moench workflow), itsCoreViewerLinkconnectsconfigSet/exposureChanged→_restart_live, which restarts continuous acquisition whenever its live timer is set. The MDA engine sets the channel config and exposure on every frame, so a stale live timer (_live_timer_idset while the stream is already stopped — e.g. after live-picking FOVs) makes each frame restart a live stream that fights the engine'ssnapImage("... sequence acquisition is running"→ dropped frames). The controller already tried to stop live at MDA start but gated it onisSequenceRunning(), which misses the stale-timer case.Fix: stop continuous acquisition unconditionally at MDA start — in both
Controller._run_mda_with_eventsandMoenchMDAEngine.setup_sequence(covers experiments and baremmc.mda.run()/ calibration).stopSequenceAcquisition()emitssequenceAcquisitionStopped, which clears napari's stale timer, so it must run even whenisSequenceRunning()isFalse. It runs before acquisition starts, so it only ever stops a live preview, never a real hardware sequence.KeepDMDAlive tied to live acquisition (positive lifecycle)
KeepDMDAlivere-displays the DMD pattern so the Mosaic3 mirrors don't park after the 200 s hold. It previously ran from boot and was stopped/restarted around every MDA — a brittle "negative" lifecycle that could race on a cancelled run (a lateteardown_sequencerestarting the thread after the next run'ssetup_sequencestopped it). It's now positively tied to live acquisition:run()oncontinuousSequenceAcquisitionStarted,stop()onsequenceAcquisitionStopped, withrun()idempotent (repeated live-start events — e.g. napari re-arming live on config changes — can't spawn duplicate threads) andstop()a no-op when idle. During an MDA the engine drives the DMD every frame (the 200 s hold spans the inter-frame gaps), so no keep-alive is needed and there is no stop/restart bracketing to race.Testing
tests/test_stim_gate.py, a self-contained regression test that drives the real feed loop through the fake microscope with a deliberately slow camera and asserts every stim SLM is built only after its predecessor frame is acquired (nomotile/stardistdependency).