Build and run openpilot natively on Windows for development (MSYS2 CLANG64) - #1

- url_file: os.register_at_fork does not exist on Windows; every route
  download died on import
- prefix: cleanup is best effort, Windows refuses to delete queue files
  that sockets in this process still map
- ui: skip the libc vasprintf log callback (no vasprintf in the Windows
  CRT), and treat the missing AF_UNIX socket family like a missing
  system D-Bus so WifiManager degrades instead of crashing the UI

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

With the PDCurses console backend used on Windows the UI alternated
between the curses screen and the original console and the timeline bar
showed no engaged segments and black boxes under it:

- is_termresized() stays true until resize_term() acknowledges the
  resize, and the UI polls it almost every frame, so every frame tore the
  screen down with endwin() and rebuilt it. Acknowledge the resize and
  only rebuild the windows when the size actually changed.
- chgat() takes the colour as a pair number; passing A_COLOR as the
  attribute selects pair 255 on PDCurses (ncursesw stores the pair
  separately, which is why it worked there). Use A_NORMAL.
- chgat() never changes the character, so the ACS_S3 markers were
  attribute-only blanks that PDCurses draws as boxes. Draw the glyph with
  hline() instead.
- The 256-colour indices are rejected by 16-colour terminals; fall back to
  the basic colours there.

PDCurses' wbkgd() only applies an attribute-only background (A_REVERSE
without a colour pair) to cells it clears itself, so the title bar was
reversed under the text only; erase the window after setting it.

The alert and bookmark markers under the timeline draw '_' like the
legend does instead of the ACS_S3 scan line.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

/tmp/comma_download_cache resolved to <drive>:\tmp on Windows. Use the
user's temp directory there (Path::tmp_dir() and hw.TMP_DIR, which is
also what shm_path() already used); Linux and macOS keep /tmp.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

64 of the 67 test modules failed to import because common/test.py pulls
in the manager, whose helpers import fcntl at module level for the
device-only unblock_stdout(). Import it there instead, and guard the
fcntl import in the gpio and i2c modules the same way (their ioctls only
work on comma hardware anyway).

The webrtc test raises unittest.SkipTest when teleoprtc is missing (no
Windows wheels), and the test runner now reports a module that raises
SkipTest at import as skipped, like unittest discovery does, instead of
a collection error. The locationd scenario lock uses msvcrt.locking on
Windows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

time.monotonic() on Windows Python 3.12 is GetTickCount64 and only ticks
every 15.6 ms, so consecutive SubMaster updates measured dt=0 and every
frequency check divided by zero. Python 3.13 switched monotonic to
QueryPerformanceCounter; do the same in openpilot/__init__.py until the
Python pin moves past 3.12 (a TODO marks it). The shim assigns through
setattr(): ty on Linux treats the Windows-only block as unreachable, so
a plain assignment's suppression comment counts as unused there, while
on Windows the assignment itself is a type error; ruff's B010 objection
is the same on both platforms, hence the noqa.

common/timeout.py used SIGALRM, which Windows lacks. There a timer thread
raises a real SIGINT: it lands as a KeyboardInterrupt in the main thread,
which __exit__ turns into the TimeoutException, and it also wakes
time.sleep(), which waits on the interpreter's SIGINT event that only the
C signal handler sets (_thread.interrupt_main() only marks the signal
pending: a 1 s Timeout around sleep(5) fired after 5 s). A blocking wait
on a child process or pipe is not interruptible either way; the docstring
says so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

NativeProcess launched daemons through multiprocessing + os.execvp. On
Windows execvp spawns a new process and exits the wrapper, so the manager
tracked a pid that was already dead, never stopped the real daemon and a
test run left dozens of loggerd/encoderd processes behind (which also
kept the test workers from exiting).

On Windows the daemon is now a subprocess.Popen in its own process group
(PopenProcess, with the pid/exitcode/join surface ManagerProcess uses).
Stopping sends CTRL_BREAK_EVENT, the one console signal that can be
delivered to a single process group; the C++ ExitHandler catches it as
SIGBREAK so loggerd & co. close their segments. Relative binaries are
resolved against cwd because CreateProcess resolves them against the
parent's directory instead. signal.SIGKILL does not exist on Windows;
SIGTERM terminates just the same through os.kill().

Python daemons are multiprocessing children in the manager's own console
group, so no console event can reach one of them alone, and os.kill()
with any other signal is TerminateProcess: their KeyboardInterrupt
handlers never run. On a PC that skips a log line and a window close;
params writes are atomic. A fix was built and measured (PopenProcess +
"-m ...manager.process" launcher in its own group, a SetConsoleCtrlHandler
mapping CTRL_BREAK to a real SIGINT): daemons in msgq polls or time.sleep
then exit cleanly in ~0.1 s, but one parked in threading.Event.wait
(deleter, 30 s) still hits the 5 s terminate because Windows lock waits
ignore signals. About 30 Windows-only lines for partial parity; deferred
as not worth it for development use.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

SwaglogState is a function static, so on Windows its destructor runs from
DllMain of every extension module during ExitProcess, after the zmq I/O
thread has already been terminated. zmq_ctx_destroy() then waits forever
for that thread and every test worker hung at exit (lldb backtrace:
zmq_ctx_term <- ~SwaglogState <- _execute_onexit_table <- DllMainCRTStartup
<- ExitProcess <- Py_Exit). The process is going away, so leak the context.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

- xattr_cache and loggerd's preserve marker store the attribute in an
  NTFS alternate data stream (path:name) on Windows, so the uploader,
  deleter and athenad markers work there; a missing stream maps to ENODATA
  like a missing xattr.
- The bootlog launch log lives in Path::tmp_dir(), which is /tmp
  everywhere but Windows.
- Upload keys and listDataDirectory results are server paths: build them
  with '/' instead of os.path.join, and match the immediate folders on the
  key rather than the local path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

…dows

- Child processes get module level targets: Windows spawns instead of
  forking and cannot pickle a socket or a closure. The spawned publisher
  signals when it is up, since spawning takes a while.
- NamedTemporaryFile handles are closed before another process opens the
  file (delete_on_close=False keeps the cleanup).
- Binaries are launched by absolute path; CreateProcess resolves ./name
  against the parent's cwd, not the cwd argument.
- strftime("%s") is a glibc extension, use datetime.timestamp().
- A refused local connection takes seconds on Windows: the upload retry
  test waits for the requeue instead of sleeping 0.1 s, the logmessaged
  test waits for the daemon to answer instead of sleeping 0.5 s.
- The mock websocket holds a connected loopback TCP pair with a byte in
  flight, so the proxy's select() passes on every platform (Linux reports
  an unconnected socket readable, Windows does not) and the socket stays
  an IP socket, which startLocalProxy() needs for its IP_TOS setsockopt;
  an AF_UNIX pair made that call fail on Linux and left the echo thread
  in accept(), hanging the runner's worker. Shutting down the listening
  socket is tolerated to fail.
- The launch log is written without newline translation, and the tests
  use xattr_cache.getxattr rather than os.getxattr.
- test_native appends sysconfig's EXE suffix to the binary path (.exe on
  Windows), so the native tests run instead of being skipped as not
  built.

The hardcoded /tmp resolves to the current drive's root on Windows,
which only exists by accident. TMP_DIR stays /tmp elsewhere.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

process_replay.stop() used signal.SIGKILL, which Windows lacks, so the
stop raised, the replayed daemon lived on and every test worker hung at
exit joining it. Use the manager's SIGKILL alias. The second stop() then
signalled a process that was still terminating, which os.kill() reports
as a PermissionError on Windows; treat that as already gone.

The runner flushes each verbose line so a redirected log shows progress.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

Writing 30 MB of log takes more than 0.5 s when eight workers share the
machine; poll the log size instead of sleeping. The ready checks the
Windows port's setup writes share the log files, and a slow daemon start
(ten of them on a loaded runner) can eat a 10 KB slack, so compare
against the size after setup and wait up to 30 s for the 30 MB to land.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

Windows draws the title bar itself; DwmSetWindowAttribute with
DWMWA_USE_IMMERSIVE_DARK_MODE switches it, applied whenever the theme is.
The function is declared by hand because dwmapi.h needs the GDI types
our lean windows.h wrapper leaves out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

tinygrad's CPU device already handles Windows (VirtualAlloc, no clock_gettime),
so use DEV=CPU with clang64's clang like Darwin instead of skipping the
SConscript. The dmonitoring compile3 step gets a "./"-relative ONNX path
because tinygrad's fetch() only treats paths starting with "/" or "." as local
files and would try to open C:\... as a URL. PYTHONPATH in that command now
uses os.pathsep.

model_replay.py keeps its frame cache under TMP_DIR and without the "|" from
the route name, which is not a valid file name character on Windows.

Verified: scons builds all three pickles (driving 80 MB, dmonitoring, dm
warps) and model_replay.py on Windows matches the master reference logs
within the PC tolerance (modelV2 ~97 ms, driverStateV2 ~69 ms per frame on
the CPU; PYTHONUTF8=1 is needed for its table when stdout is redirected).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

proclogd and journald read /proc and the systemd journal, so enable them on
Linux only instead of everywhere but macOS. The webcam camerad opens cameras
by index outside Linux; only Linux has /dev/video nodes.

webrtcd is disabled on Windows instead of failing on its first stream
request: teleoprtc does not install there (libdatachannel-py has no
Windows wheels). A TODO marks the gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

Like VideoToolbox on macOS, D3D11VA is the platform's own hardware decode
interface and works on any GPU vendor. The decode path is unchanged: the
frame is downloaded with av_hwframe_transfer_data as NV12 and copied like
the CUDA and VideoToolbox frames. The Windows ffmpeg wheel from the
dependencies fork enables the d3d11va2 hwaccels.

Verified with replay --demo: no "fallback to CPU decoding" in the log pane,
frame chroma statistics identical to --no-hw-decoder, and replay's CPU time
drops from ~4.0 s to ~1.3 s per 30 s on an RTX 5090.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

Found by running USE_WEBCAM=1 manager.py on Windows, which had never been
tried; each daemon that macOS runs now runs there too.

- unblock_stdout() returns early on Windows: no pty or fork, and the console
  does not block the manager.
- bootlog is launched by absolute path; CreateProcess resolves "./bootlog"
  against the parent's cwd, not the cwd argument.
- PopenProcess (the Windows NativeProcess) gains is_alive(), which the
  manager's status line calls; starting stream_encoderd killed the manager.
- loggerd config falls back to shutil.disk_usage where os.statvfs does not
  exist, keeping the statvfs path (and the deleter tests' fake) elsewhere.
- hardwared imports fcntl inside the device-only touch thread.
- webrtcd falls back to signal.signal where the event loop has no signal
  handlers (Windows raises NotImplementedError).

hardwared still exits on Windows because LinuxSystemStats reads /proc/stat;
that is the same on macOS and left alone.

Verified: manager runs logmessaged, ui, deleter, pandad, and with driver view
and IsLiveStreaming on, webcamerad, dmonitoringmodeld, dmonitoringd, soundd,
stream_encoderd and webrtcd; a teleoprtc client receives the x264-encoded
driver camera from webrtcd at ~1.5 MB/s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

MoveFileEx with MOVEFILE_REPLACE_EXISTING fails with ERROR_SHARING_VIOLATION
while another process has the destination open, and readers open params
without FILE_SHARE_DELETE, so a put racing a get was silently dropped.
Reads take microseconds; wait them out for up to 200 ms instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

op.sh and setup_dependencies.sh learn the MSYS2 CLANG64 shell: pacman
installs the toolchain, the venv lives in Scripts/, the git-lfs filters
and hook use that layout, and PYTHONUTF8 keeps redirected output UTF-8.
Git for Windows has to be the git in that shell: the venv's native
git-lfs corrupts paths when MSYS2's Cygwin-style git drives it. The
README gets the Windows setup next to the macOS one.

A fresh MSYS2 has no unzip and the uv shell installer needs it; the
runner's native unzip on the inherited PATH cannot open MSYS paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

A build_windows job in tests.yaml next to build_mac, with the pipeline's
env, triggers and concurrency rules: op.sh setup and build in an MSYS2
CLANG64 shell that inherits the runner's PATH for Git for Windows, then
the unit tests, the part that validates the port. op.sh build activates
the venv before scons, like a developer's shell does: the msys2 shell
puts the CLANG64 bin directory, and with it the mingw python3 the
toolchain group installs, ahead of the venv on PATH, so the codegen steps
scons runs would find a python without numpy. The runner's display
driver has no OpenGL, so the job installs Mesa and puts its llvmpipe
opengl32.dll next to the interpreter, the Windows counterpart of the
headless raylib backend the Linux job uses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

…ies#107 publishes

--- TODO REMOVE AFTER DEPENDENCY PR MERGES (commaai/dependencies#107) ---

This commit is dropped from the PR once the win_amd64 wheels are on PyPI.
Until then it points uv at a GitHub release of the same wheels, built from
that PR's branch on my fork, so the Windows CI job runs here meanwhile:
the fifteen comma-deps packages for sys_platform == 'win32', with
comma-deps-eigen listed directly because uv sources only apply to direct
dependencies (rednose needs it). Linux and macOS resolve from PyPI as
before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

--- TODO REMOVE AFTER THE SUBMODULE PRS MERGE (commaai/msgq#709, commaai/panda#2427, commaai/rednose#61, commaai/opendbc#3724) ---

This commit is dropped from the PR once the four submodule PRs have
merged and the pointers move to comma's commits. Until then the pointers
are commits on those PR branches on my fork (the series itself, below
the drafts' own TEMP index commits, whose uv sources would otherwise
leak into this lockfile; msgq: the same series on openpilot's current
msgq pin, since comma's msgq master has moved past it), so .gitmodules
fetches them from there and the CI here can check them out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

Fork only, dropped when upstreaming: the dependencies fork builds the
libdatachannel-py wheel that PyPI lacks for Windows and serves it from the
same comma-deps-windows flat index as the other wheels. uv sources only
apply to direct dependencies, so libdatachannel-py is listed for win32 next
to teleoprtc, like comma-deps-eigen for rednose. teleoprtc's 14 tests pass
on Windows and webrtcd imports. Upstream keeps teleoprtc off Windows:
libdatachannel-py is not comma's project.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt

Fork only, dropped when upstreaming: workflow_dispatch only works from
the default branch, so the push trigger lists the branch the fork
develops on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt