Skip to content

Cross-platform compat: Windows 11 + non-UTF-8 locales (incl. Korean cp949) - #25

Merged
sp3arm4n merged 3 commits into
mainfrom
codex
May 10, 2026
Merged

Cross-platform compat: Windows 11 + non-UTF-8 locales (incl. Korean cp949)#25
sp3arm4n merged 3 commits into
mainfrom
codex

Conversation

@sp3arm4n

Copy link
Copy Markdown
Owner

Summary

  • Pin encoding=\"utf-8\" on every external-tool subprocess (apktool / jadx / Ghidra / Frida) so stdout/stderr decoding doesn't crash under Windows cp949/cp1252 or non-UTF-8 POSIX locales. Frida log is also written as utf-8.
  • Cross-platform test stub helper at sample/tests/_subproc_stub.py. Replaces the #!/bin/sh stub scripts that previously crashed Windows tests with WinError 193. On POSIX it returns a shebang Python script with the exec bit; on Windows it returns a .cmd shim that delegates to the same Python via sys.executable. All inline shell-stub fixtures across test_apk_decoder, test_jadx_runner, test_android_pipeline, and test_cli_android_audit migrated to the helper.
  • test_pipeline.py: python3sys.executable (no python3 alias on Windows).
  • test_cli_android_audit.py: pin utf-8 when reading bundled poc .sh files (was crashing on cp949 default).
  • .gitignore: broaden venv/ to also cover sibling per-platform virtualenvs (venv-*/).
  • Bundled in: pre-existing fix multi so analysis fallback commit on the branch.

Test plan

  • Windows 11 / Python 3.11.8: python -m unittest discover -s sample/tests805/805 pass (16 env-dependent skipped). Pre-fix run was 27 errors.
  • End-to-end CLI under a Korean output path: venomhook offset-e2e --static-json sample/examples/static_meta.sample.json --target 샘플.exe --out-dir out_win/한글_경로_테스트 produces the expected venomhook.{json,db,md,js} artifacts.
  • All changes guarded by os.name == \"nt\" branches or are strictly utf-8-additive — POSIX behavior unchanged.
  • macOS M2 / Ubuntu 24.04 / Kali: not run on this machine. Pattern is safe (POSIX path uses shebang #!{sys.executable} + chmod +x, identical to existing test stubs), but a fresh python -m unittest discover -s sample/tests on each target OS before merge would close the loop.
  • Live external tools: only stub-level coverage in CI. Real apktool/jadx/Ghidra/Frida i18n stdout was not exercised; the encoding fix is the right direction but worth a real-tool sanity run on Windows.

Out of scope

  • Windows long-path (>260 chars) handling — not changed; relies on Python 3.11's default behavior + Windows long-path opt-in.
  • .claude/skills/... and other unrelated working-tree changes — left untouched.

🤖 Generated with Claude Code

sp3arm4n and others added 3 commits May 10, 2026 22:33
- pin encoding="utf-8" on all subprocess.run text=True calls
  (apktool / jadx / Ghidra / Frida) so external-tool output doesn't
  crash decoding under Windows cp949/cp1252 or non-UTF-8 POSIX locales
- write frida log with explicit utf-8 encoding
- add sample/tests/_subproc_stub.py: a cross-platform "stub binary"
  helper so tests no longer hand POSIX-only #!/bin/sh scripts to
  subprocess.run on Windows (WinError 193). On POSIX it returns a
  shebang Python script with the exec bit; on Windows it returns a
  .cmd shim that delegates to the same Python via sys.executable
- migrate all inline shell-stub fixtures in test_apk_decoder,
  test_jadx_runner, test_android_pipeline, and test_cli_android_audit
  to the helper
- test_pipeline: use sys.executable instead of "python3" (no python3
  alias on Windows)
- test_cli_android_audit: pin utf-8 when reading bundled poc .sh files

Verified: 805/805 unit tests pass on Windows 11 / Python 3.11 and the
offset-e2e CLI smoke test produces the expected artifacts under a
Korean-character output path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sibling of venv/ for the Windows-side editable install. Should not be
tracked, same as venv/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Covers venv-mac, venv-linux, venv-kali, etc. — matches the project's
target of running on macOS / Ubuntu / Kali / Windows from sibling
per-platform virtualenvs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sp3arm4n
sp3arm4n merged commit 14baf06 into main May 10, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant