Skip to content

fix(moench): Mosaic3 all-on DMD (scalar SLMImage no-op) + premature stim-mask build + napari live-restart during MDA - #17

Open
hinderling wants to merge 4 commits into
pertzlab:mainfrom
hinderling:fix/mosaic3-slm-and-stim-gate
Open

fix(moench): Mosaic3 all-on DMD (scalar SLMImage no-op) + premature stim-mask build + napari live-restart during MDA#17
hinderling wants to merge 4 commits into
pertzlab:mainfrom
hinderling:fix/mosaic3-slm-and-stim-gate

Conversation

@hinderling

@hinderling hinderling commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Three Moench (Andor Mosaic3 DMD) acquisition-reliability fixes, all confirmed on the scope. The first two also exist in main, so feeding them back; the third (§3) is new.

1. fix(moench): expand scalar SLMImage to an array so the Mosaic3 actually updates

On the Mosaic3, core.setSLMPixelsTo(dev, N) is a silent no-op — no error, no effect; the previously-latched pattern stays displayed. The base MDAEngine renders a scalar-bool SLMImage (data=True/False) via setSLMPixelsTo, so every scalar SLM command was doing nothing on this hardware. Two symptoms, one root cause:

  • First stim frame fired full-field (all-on). When a stim mask isn't ready, _build_stim_slm sends SLMImage(data=False); the intended all-off never reached the DMD, so the latched all-on fired the UV full-field instead of the targeted pattern.
  • Occasional fully-blank imaging frame. MoenchMDAEngine._maybe_inject_dmd_wake_slm injects a scalar SLMImage(data=True) to hold the DMD all-on for each imaging capture — also a no-op, so imaging relied on the latched all-on plus the OverlapMode re-pulse and the 200 s hold. Without an active re-display each frame, the mirror occasionally parked mid-capture and the frame came back blank.

Fix: in MoenchMDAEngine._set_event_slm_image, expand a scalar-bool SLMImage to a full uint8 array before delegating, so it routes through setSLMImage (arrays are the only reliable Mosaic3 update path). This actively re-displays the intended pattern every frame, pinning the mirror against the blank-frame parking. Contained entirely in the microscope-specific engine as a hardware workaround.

Not addressed by this fix: a separate every-other-frame imaging dimming — a ~0.74x signal-only even/odd (fluorescence scales, background stays flat, in focus) that recurs intermittently on some runs — is still open and under investigation. It is distinct from the all-on / blank-frame symptoms above; do not read §1 as closing it.

Diagnostic tell for old data: a stim readout channel that is full-field (≈100% of pixels lit) is a pre-fix run; targeted thin lines mean the fix was active.

2. fix(controller): gate stim SLM build on predecessor frame acquisition

The feed loop (_run_mda_with_events) runs several events ahead of the camera (qsize<3 backpressure). In previous stim mode it built a stim event's SLM — blocking get_stim_mask(t-1) — long before frame t-1 was acquired. At minute-scale intervals with slow segmentation, the mask wait (80 s) expired before the predecessor frame even existed, so _build_stim_slm returned SLMImage(data=False) (which, combined with bug 1, fired the DMD's stale pattern).

Fix: track the (t, p) of each acquired imaging frame and, before building a stim's SLM, wait for its predecessor to be acquired — (t-1, p) in previous mode, (t, p) in current mode. get_stim_mask then only waits out segmentation of an already-acquired frame, and the 80 s timeout becomes a genuine pipeline-stuck detector again. No deadlock: the predecessor is always queued in an earlier feed-loop iteration; the wait is cancel/fatal-aware, mirroring the existing WaitEvent acquisition gate. This is pure feed-loop scheduling — the MDA engine stays microscope-agnostic.

3. fix: stop live acquisition before every MDA; drive KeepDMDAlive from live events

A third Moench acquisition-reliability fix in the same DMD / live-view area.

napari-micromanager restarts live mid-MDA → dropped/corrupted frames

With napari-micromanager attached to the same core (the usual Moench workflow), its CoreViewerLink connects configSet/exposureChanged_restart_live, which restarts continuous acquisition whenever its live timer is set. The MDA engine sets the channel config and exposure on every frame, so a stale live timer (_live_timer_id set while the stream is already stopped — e.g. after live-picking FOVs) makes each frame restart a live stream that fights the engine's snapImage ("... sequence acquisition is running" → dropped frames). The controller already tried to stop live at MDA start but gated it on isSequenceRunning(), which misses the stale-timer case.

Fix: stop continuous acquisition unconditionally at MDA start — in both Controller._run_mda_with_events and MoenchMDAEngine.setup_sequence (covers experiments and bare mmc.mda.run() / calibration). stopSequenceAcquisition() emits sequenceAcquisitionStopped, which clears napari's stale timer, so it must run even when isSequenceRunning() is False. It runs before acquisition starts, so it only ever stops a live preview, never a real hardware sequence.

The root cause is arguably a napari-micromanager bug: _restart_live lacks the self._mmc.mda.is_running() guard that its sibling _image_snapped already has. This is the microscope-side defense; the one-line upstream fix is being reported separately.

KeepDMDAlive tied to live acquisition (positive lifecycle)

KeepDMDAlive re-displays the DMD pattern so the Mosaic3 mirrors don't park after the 200 s hold. It previously ran from boot and was stopped/restarted around every MDA — a brittle "negative" lifecycle that could race on a cancelled run (a late teardown_sequence restarting the thread after the next run's setup_sequence stopped it). It's now positively tied to live acquisition: run() on continuousSequenceAcquisitionStarted, stop() on sequenceAcquisitionStopped, with run() idempotent (repeated live-start events — e.g. napari re-arming live on config changes — can't spawn duplicate threads) and stop() a no-op when idle. During an MDA the engine drives the DMD every frame (the 200 s hold spans the inter-frame gaps), so no keep-alive is needed and there is no stop/restart bracketing to race.

Testing

  • Hardware (§1/§2): confirmed on the Moench — targeted stim on the first stim frame, and stable imaging with no all-on/blank frames.
  • Hardware (§3): confirmed that a stale napari live timer no longer breaks snaps (unconditional stop clears it and frames come back clean), and the KeepDMDAlive lifecycle — off at init, on during live, idempotent, off during an MDA, resumes when live restarts.
  • Unit: adds tests/test_stim_gate.py, a self-contained regression test that drives the real feed loop through the fake microscope with a deliberately slow camera and asserts every stim SLM is built only after its predecessor frame is acquired (no motile/stardist dependency).

@hinderling hinderling changed the title fix: Mosaic3 all-on DMD (scalar SLMImage no-op) + premature stim-mask build fix(moench): Mosaic3 all-on DMD (scalar SLMImage no-op) + premature stim-mask build + napari live-restart during MDA Aug 5, 2026
A scalar-bool SLMImage reaches the base engine as setSLMPixelsTo, which
the Mosaic3 ignores without raising, so the previously latched pattern
stays on the mirrors. Expand the scalar to a uint8 array in
MoenchMDAEngine._set_event_slm_image so it routes through setSLMImage.

This makes an intended all-off actually clear the DMD, and re-displays
the pattern on every imaging capture instead of relying on the latched
state surviving between frames.
The feed loop runs a few events ahead of the camera, so it reached a
stim event and blocked on get_stim_mask for a frame that had not been
shot yet. At minute-scale intervals the mask wait expired before the
frame existed and the stim fell through to an all-off mask.

Track which (t, p) imaging frames have reached the pipeline and wait for
a stim's source frame ((t-1, p) in "previous" mode, (t, p) in "current")
before asking for its mask. A frame that never arrives cannot have a
mask, so a timed-out wait goes straight to the all-off fallback rather
than spending the mask timeout again on a lookup that must fail.
… to live

napari-micromanager can hold a live timer while the stream is already
stopped, and that timer restarts live acquisition on the per-frame
configSet/exposureChanged signals the engine emits, which then fights
every snap. Stop continuous acquisition at MDA start without gating on
isSequenceRunning(), since the emitted sequenceAcquisitionStopped is
what clears the timer. Doing it in the engine covers calibration and
bare mmc.mda.run() as well as experiments run through the Controller.

Drive KeepDMDAlive from the live-acquisition signals rather than
starting it at boot and bracketing every MDA: run() on live start (now
idempotent), stop() on live stop (now a no-op when idle). The engine
drives the DMD every event during a run, so no keep-alive is needed and
there is no stop/restart pair left to race on a cancelled run.
@hinderling
hinderling force-pushed the fix/mosaic3-slm-and-stim-gate branch from 2f1adb5 to 6e1c9e3 Compare August 5, 2026 22:43
Projecting the calibration/test spots with one mmc.mda.run per event
brackets each spot with setup/teardown_sequence, which stop and restart
KeepDMDAlive; each restart re-displays the all-on live pattern and resets
the SLM ExposureTime, so under OverlapMode a spot can end up on a short
exposure that blanks before the camera opens. Run every event in one MDA
(with hardware sequencing off -- otherwise pymmcore-plus tries to combine
the consecutive SLM events and fails validation) so KeepDMDAlive pauses
exactly once.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant