Build and run openpilot natively on Windows for development (MSYS2 CLANG64) - #1
Closed
AmyJeanes wants to merge 25 commits into
Closed
Build and run openpilot natively on Windows for development (MSYS2 CLANG64)#1AmyJeanes wants to merge 25 commits into
AmyJeanes wants to merge 25 commits into
Conversation
AmyJeanes
force-pushed
the
windows-dev
branch
2 times, most recently
from
September 7, 2026 11:23
7085989 to
3cbf0f0
Compare
Process replay diff reportReplays driving segments through this PR and compares the behavior to master. ✅ 0 changed, 66 passed, 0 errors |
AmyJeanes
force-pushed
the
windows-dev
branch
16 times, most recently
from
September 7, 2026 20:12
07d48a7 to
c42c049
Compare
This was referenced Sep 7, 2026
AmyJeanes
force-pushed
the
windows-dev
branch
4 times, most recently
from
September 7, 2026 23:58
d03a242 to
47f4cea
Compare
Toolchain: SCons uses the mingw tools with clang/clang++, spawns every build command through MSYS2 bash so the POSIX shell syntax in the SConscripts keeps working, converts path separators for bash and links the C++ runtime statically (Python does not search PATH for extension DLLs); msgq's cython tool names the extension modules .pyd. The vendored native dependencies come from the comma-deps win_amd64 wheels. Code: _WIN32 shims in common (mkdir/fsync/setenv/localtime_r/strptime, LockFileEx and MoveFileEx behind util::lock_file_exclusive and util::replace_file, plain directory instead of a params symlink, %TEMP% as the shared-memory directory on both the C++ and Python side, tcp loopback for swaglog since libzmq has no ipc:// on MinGW), a lean common/win32.h wrapper so windows.h never meets the capnp/params enumerators, prefix.h split into a .cc (its rm -rf shell-outs become std::filesystem::remove_all), and Win32 variants of the POSIX-only bits in cabana, replay and jotpluggler (signal handling, bridge/downloader subprocesses, clipboard, executable path, PDCurses spelling). The two file helpers and the Windows fsync shim live in common/file.h rather than util.h: cabana's settings.cc also includes CoreFoundation on macOS, whose MacTypes Rect collides with util.h's. msgq, rednose, panda and opendbc point at their Windows ports: msgq (queues, events and vision buffers as shared-memory sections, the visionipc socket off /tmp, the poll deadline on the steady clock), rednose (the generated filters build with the mingw toolchain), panda and opendbc (libpanda and libsafety as DLLs, the CANPacket_t layout for cffi). Gated on Windows for now: modeld model compilation, pandad, the v4l decoder, socketcan, teleoprtc (no libdatachannel-py wheels). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
- url_file: os.register_at_fork does not exist on Windows; every route download died on import - prefix: cleanup is best effort, Windows refuses to delete queue files that sockets in this process still map - ui: skip the libc vasprintf log callback (no vasprintf in the Windows CRT), and treat the missing AF_UNIX socket family like a missing system D-Bus so WifiManager degrades instead of crashing the UI Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Two Windows-only failures in the replay library, which cabana shares: - replay.exe segfaulted right after loading a route. currentSeconds() subtracts two uint64 timestamps, and at stream start cur_mono_time_ is deliberately 1 ns behind route_start_ts_, so the difference wrapped to ~1.8e10 s. Casting that to int gives INT_MIN, the displayed route time went negative and the UCRT ctime() returned NULL where glibc prints a 1901 date; ConsoleUI then built a std::string from NULL. Make the difference signed (correct on every platform) and guard the ctime result. - Playback ran unthrottled (cabana showed 1x but played ~20x). winpthreads' clock_nanosleep rejects CLOCK_MONOTONIC with EINVAL, which the loop treated as "done sleeping". Windows now waits on a high-resolution waitable timer in 5 ms slices, polling interrupt_requested between slices because there is no SIGUSR1 to cut the wait short. Measured: 100 ms sleeps land within 0.5 ms, 2 ms sleeps within 0.3 ms, and a seek interrupts a 1 s sleep within ~7 ms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
With the PDCurses console backend used on Windows the UI alternated between the curses screen and the original console and the timeline bar showed no engaged segments and black boxes under it: - is_termresized() stays true until resize_term() acknowledges the resize, and the UI polls it almost every frame, so every frame tore the screen down with endwin() and rebuilt it. Acknowledge the resize and only rebuild the windows when the size actually changed. - chgat() takes the colour as a pair number; passing A_COLOR as the attribute selects pair 255 on PDCurses (ncursesw stores the pair separately, which is why it worked there). Use A_NORMAL. - chgat() never changes the character, so the ACS_S3 markers were attribute-only blanks that PDCurses draws as boxes. Draw the glyph with hline() instead. - The 256-colour indices are rejected by 16-colour terminals; fall back to the basic colours there. PDCurses' wbkgd() only applies an attribute-only background (A_REVERSE without a colour pair) to cells it clears itself, so the title bar was reversed under the text only; erase the window after setting it. The alert and bookmark markers under the timeline draw '_' like the legend does instead of the ACS_S3 scan line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
/tmp/comma_download_cache resolved to <drive>:\tmp on Windows. Use the user's temp directory there (Path::tmp_dir() and hw.TMP_DIR, which is also what shm_path() already used); Linux and macOS keep /tmp. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
64 of the 67 test modules failed to import because common/test.py pulls in the manager, whose helpers import fcntl at module level for the device-only unblock_stdout(). Import it there instead, and guard the fcntl import in the gpio and i2c modules the same way (their ioctls only work on comma hardware anyway). The webrtc test raises unittest.SkipTest when teleoprtc is missing (no Windows wheels), and the test runner now reports a module that raises SkipTest at import as skipped, like unittest discovery does, instead of a collection error. The locationd scenario lock uses msvcrt.locking on Windows. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
time.monotonic() on Windows Python 3.12 is GetTickCount64 and only ticks every 15.6 ms, so consecutive SubMaster updates measured dt=0 and every frequency check divided by zero. Python 3.13 switched monotonic to QueryPerformanceCounter; do the same in openpilot/__init__.py until the Python pin moves past 3.12 (a TODO marks it). The shim assigns through setattr(): ty on Linux treats the Windows-only block as unreachable, so a plain assignment's suppression comment counts as unused there, while on Windows the assignment itself is a type error; ruff's B010 objection is the same on both platforms, hence the noqa. common/timeout.py used SIGALRM, which Windows lacks. There a timer thread raises a real SIGINT: it lands as a KeyboardInterrupt in the main thread, which __exit__ turns into the TimeoutException, and it also wakes time.sleep(), which waits on the interpreter's SIGINT event that only the C signal handler sets (_thread.interrupt_main() only marks the signal pending: a 1 s Timeout around sleep(5) fired after 5 s). A blocking wait on a child process or pipe is not interruptible either way; the docstring says so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
NativeProcess launched daemons through multiprocessing + os.execvp. On Windows execvp spawns a new process and exits the wrapper, so the manager tracked a pid that was already dead, never stopped the real daemon and a test run left dozens of loggerd/encoderd processes behind (which also kept the test workers from exiting). On Windows the daemon is now a subprocess.Popen in its own process group (PopenProcess, with the pid/exitcode/join surface ManagerProcess uses). Stopping sends CTRL_BREAK_EVENT, the one console signal that can be delivered to a single process group; the C++ ExitHandler catches it as SIGBREAK so loggerd & co. close their segments. Relative binaries are resolved against cwd because CreateProcess resolves them against the parent's directory instead. signal.SIGKILL does not exist on Windows; SIGTERM terminates just the same through os.kill(). Python daemons are multiprocessing children in the manager's own console group, so no console event can reach one of them alone, and os.kill() with any other signal is TerminateProcess: their KeyboardInterrupt handlers never run. On a PC that skips a log line and a window close; params writes are atomic. A fix was built and measured (PopenProcess + "-m ...manager.process" launcher in its own group, a SetConsoleCtrlHandler mapping CTRL_BREAK to a real SIGINT): daemons in msgq polls or time.sleep then exit cleanly in ~0.1 s, but one parked in threading.Event.wait (deleter, 30 s) still hits the 5 s terminate because Windows lock waits ignore signals. About 30 Windows-only lines for partial parity; deferred as not worth it for development use. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
SwaglogState is a function static, so on Windows its destructor runs from DllMain of every extension module during ExitProcess, after the zmq I/O thread has already been terminated. zmq_ctx_destroy() then waits forever for that thread and every test worker hung at exit (lldb backtrace: zmq_ctx_term <- ~SwaglogState <- _execute_onexit_table <- DllMainCRTStartup <- ExitProcess <- Py_Exit). The process is going away, so leak the context. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
- xattr_cache and loggerd's preserve marker store the attribute in an NTFS alternate data stream (path:name) on Windows, so the uploader, deleter and athenad markers work there; a missing stream maps to ENODATA like a missing xattr. - The bootlog launch log lives in Path::tmp_dir(), which is /tmp everywhere but Windows. - Upload keys and listDataDirectory results are server paths: build them with '/' instead of os.path.join, and match the immediate folders on the key rather than the local path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
…dows
- Child processes get module level targets: Windows spawns instead of
forking and cannot pickle a socket or a closure. The spawned publisher
signals when it is up, since spawning takes a while.
- NamedTemporaryFile handles are closed before another process opens the
file (delete_on_close=False keeps the cleanup).
- Binaries are launched by absolute path; CreateProcess resolves ./name
against the parent's cwd, not the cwd argument.
- strftime("%s") is a glibc extension, use datetime.timestamp().
- A refused local connection takes seconds on Windows: the upload retry
test waits for the requeue instead of sleeping 0.1 s, the logmessaged
test waits for the daemon to answer instead of sleeping 0.5 s.
- The mock websocket holds a connected loopback TCP pair with a byte in
flight, so the proxy's select() passes on every platform (Linux reports
an unconnected socket readable, Windows does not) and the socket stays
an IP socket, which startLocalProxy() needs for its IP_TOS setsockopt;
an AF_UNIX pair made that call fail on Linux and left the echo thread
in accept(), hanging the runner's worker. Shutting down the listening
socket is tolerated to fail.
- The launch log is written without newline translation, and the tests
use xattr_cache.getxattr rather than os.getxattr.
- test_native appends sysconfig's EXE suffix to the binary path (.exe on
Windows), so the native tests run instead of being skipped as not
built.
The hardcoded /tmp resolves to the current drive's root on Windows,
which only exists by accident. TMP_DIR stays /tmp elsewhere.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
process_replay.stop() used signal.SIGKILL, which Windows lacks, so the stop raised, the replayed daemon lived on and every test worker hung at exit joining it. Use the manager's SIGKILL alias. The second stop() then signalled a process that was still terminating, which os.kill() reports as a PermissionError on Windows; treat that as already gone. The runner flushes each verbose line so a redirected log shows progress. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Writing 30 MB of log takes more than 0.5 s when eight workers share the machine; poll the log size instead of sleeping. The ready checks the Windows port's setup writes share the log files, and a slow daemon start (ten of them on a loaded runner) can eat a 10 KB slack, so compare against the size after setup and wait up to 30 s for the 30 MB to land. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Windows draws the title bar itself; DwmSetWindowAttribute with DWMWA_USE_IMMERSIVE_DARK_MODE switches it, applied whenever the theme is. The function is declared by hand because dwmapi.h needs the GDI types our lean windows.h wrapper leaves out. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
tinygrad's CPU device already handles Windows (VirtualAlloc, no clock_gettime), so use DEV=CPU with clang64's clang like Darwin instead of skipping the SConscript. The dmonitoring compile3 step gets a "./"-relative ONNX path because tinygrad's fetch() only treats paths starting with "/" or "." as local files and would try to open C:\... as a URL. PYTHONPATH in that command now uses os.pathsep. model_replay.py keeps its frame cache under TMP_DIR and without the "|" from the route name, which is not a valid file name character on Windows. Verified: scons builds all three pickles (driving 80 MB, dmonitoring, dm warps) and model_replay.py on Windows matches the master reference logs within the PC tolerance (modelV2 ~97 ms, driverStateV2 ~69 ms per frame on the CPU; PYTHONUTF8=1 is needed for its table when stdout is redirected). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
proclogd and journald read /proc and the systemd journal, so enable them on Linux only instead of everywhere but macOS. The webcam camerad opens cameras by index outside Linux; only Linux has /dev/video nodes. webrtcd is disabled on Windows instead of failing on its first stream request: teleoprtc does not install there (libdatachannel-py has no Windows wheels). A TODO marks the gate. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Like VideoToolbox on macOS, D3D11VA is the platform's own hardware decode interface and works on any GPU vendor. The decode path is unchanged: the frame is downloaded with av_hwframe_transfer_data as NV12 and copied like the CUDA and VideoToolbox frames. The Windows ffmpeg wheel from the dependencies fork enables the d3d11va2 hwaccels. Verified with replay --demo: no "fallback to CPU decoding" in the log pane, frame chroma statistics identical to --no-hw-decoder, and replay's CPU time drops from ~4.0 s to ~1.3 s per 30 s on an RTX 5090. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Found by running USE_WEBCAM=1 manager.py on Windows, which had never been tried; each daemon that macOS runs now runs there too. - unblock_stdout() returns early on Windows: no pty or fork, and the console does not block the manager. - bootlog is launched by absolute path; CreateProcess resolves "./bootlog" against the parent's cwd, not the cwd argument. - PopenProcess (the Windows NativeProcess) gains is_alive(), which the manager's status line calls; starting stream_encoderd killed the manager. - loggerd config falls back to shutil.disk_usage where os.statvfs does not exist, keeping the statvfs path (and the deleter tests' fake) elsewhere. - hardwared imports fcntl inside the device-only touch thread. - webrtcd falls back to signal.signal where the event loop has no signal handlers (Windows raises NotImplementedError). hardwared still exits on Windows because LinuxSystemStats reads /proc/stat; that is the same on macOS and left alone. Verified: manager runs logmessaged, ui, deleter, pandad, and with driver view and IsLiveStreaming on, webcamerad, dmonitoringmodeld, dmonitoringd, soundd, stream_encoderd and webrtcd; a teleoprtc client receives the x264-encoded driver camera from webrtcd at ~1.5 MB/s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
MoveFileEx with MOVEFILE_REPLACE_EXISTING fails with ERROR_SHARING_VIOLATION while another process has the destination open, and readers open params without FILE_SHARE_DELETE, so a put racing a get was silently dropped. Reads take microseconds; wait them out for up to 200 ms instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
op.sh and setup_dependencies.sh learn the MSYS2 CLANG64 shell: pacman installs the toolchain, the venv lives in Scripts/, the git-lfs filters and hook use that layout, and PYTHONUTF8 keeps redirected output UTF-8. Git for Windows has to be the git in that shell: the venv's native git-lfs corrupts paths when MSYS2's Cygwin-style git drives it. The README gets the Windows setup next to the macOS one. A fresh MSYS2 has no unzip and the uv shell installer needs it; the runner's native unzip on the inherited PATH cannot open MSYS paths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
A build_windows job in tests.yaml next to build_mac, with the pipeline's env, triggers and concurrency rules: op.sh setup and build in an MSYS2 CLANG64 shell that inherits the runner's PATH for Git for Windows, then the unit tests, the part that validates the port. op.sh build activates the venv before scons, like a developer's shell does: the msys2 shell puts the CLANG64 bin directory, and with it the mingw python3 the toolchain group installs, ahead of the venv on PATH, so the codegen steps scons runs would find a python without numpy. The runner's display driver has no OpenGL, so the job installs Mesa and puts its llvmpipe opengl32.dll next to the interpreter, the Windows counterpart of the headless raylib backend the Linux job uses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
…ies#107 publishes --- TODO REMOVE AFTER DEPENDENCY PR MERGES (commaai/dependencies#107) --- This commit is dropped from the PR once the win_amd64 wheels are on PyPI. Until then it points uv at a GitHub release of the same wheels, built from that PR's branch on my fork, so the Windows CI job runs here meanwhile: the fifteen comma-deps packages for sys_platform == 'win32', with comma-deps-eigen listed directly because uv sources only apply to direct dependencies (rednose needs it). Linux and macOS resolve from PyPI as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
--- TODO REMOVE AFTER THE SUBMODULE PRS MERGE (commaai/msgq#709, commaai/panda#2427, commaai/rednose#61, commaai/opendbc#3724) --- This commit is dropped from the PR once the four submodule PRs have merged and the pointers move to comma's commits. Until then the pointers are commits on those PR branches on my fork (the series itself, below the drafts' own TEMP index commits, whose uv sources would otherwise leak into this lockfile; msgq: the same series on openpilot's current msgq pin, since comma's msgq master has moved past it), so .gitmodules fetches them from there and the CI here can check them out. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Fork only, dropped when upstreaming: the dependencies fork builds the libdatachannel-py wheel that PyPI lacks for Windows and serves it from the same comma-deps-windows flat index as the other wheels. uv sources only apply to direct dependencies, so libdatachannel-py is listed for win32 next to teleoprtc, like comma-deps-eigen for rednose. teleoprtc's 14 tests pass on Windows and webrtcd imports. Upstream keeps teleoprtc off Windows: libdatachannel-py is not comma's project. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Fork only, dropped when upstreaming: workflow_dispatch only works from the default branch, so the push trigger lists the branch the fork develops on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
AmyJeanes
force-pushed
the
windows-dev
branch
from
September 8, 2026 00:25
47f4cea to
3da7a5c
Compare
Owner
Author
|
Superseded by the upstream draft commaai#38810; the fork's windows-dev branch keeps running CI on pushes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-internal draft: runs this repository's own CI on the
windowsseries before it is proposed upstream. The base branchwindows-baseis the upstream commit the series builds on. Not for merging.🤖 Generated with Claude Code
https://claude.ai/code/session_016AgqYZYWLpwE2T5vS3nVEt
Opened from
windows-devrather thanwindows: the upstreamable prefix cannot install the comma-deps wheels on Windows until they are on PyPI, so its Windows job would fail for that reason alone. The Linux and macOS jobs test the same code either way; the tail only adds the fork's wheel index, teleoprtc on Windows and the CI trigger.