Skip to content

fix(prs): decode gh/git/claude output as UTF-8, not the Windows cp1252 codec#1980

Closed
LukeTheoJohnson wants to merge 1 commit into
Graphify-Labs:v8from
LukeTheoJohnson:fix-prs-cp1252-subprocess-decode
Closed

fix(prs): decode gh/git/claude output as UTF-8, not the Windows cp1252 codec#1980
LukeTheoJohnson wants to merge 1 commit into
Graphify-Labs:v8from
LukeTheoJohnson:fix-prs-cp1252-subprocess-decode

Conversation

@LukeTheoJohnson

Copy link
Copy Markdown
Contributor

Problem

On Windows, graphify prs crashes against any repo that has non-ASCII PR metadata. prs.py reads gh, git, and claude output through subprocess.run(..., text=True) with no explicit encoding=, so stdout is decoded with the locale codec, which is cp1252 on a default Windows install. gh emits UTF-8 JSON whose PR titles and author logins routinely contain non-Latin1 bytes (emoji, or the Persian title in this repo's own PR #1281), and cp1252 has no mapping for bytes such as 0x81.

The failure is indirect. The capture reader thread raises UnicodeDecodeError, subprocess.run swallows that and returns stdout=None, and the caller then dies one line later:

  • _gh runs json.loads(None) and raises TypeError
  • fetch_pr_files / fetch_worktrees run None.splitlines() and raise AttributeError

Neither is in _gh's except (TimeoutExpired, JSONDecodeError, FileNotFoundError), so the command aborts, and a spurious reader-thread traceback is printed to stderr as well.

Root cause

text=True without encoding= uses locale.getpreferredencoding() for decoding (cp1252 on Windows). CI runs on ubuntu-latest only, where the locale is UTF-8, so this path is never exercised upstream.

Fix

Pass encoding="utf-8", errors="replace" to all five subprocess reads in prs.py. This is the decode-side counterpart of #1505, which applied the same change to the llm.py claude-cli subprocess; prs.py was not touched then.

Reproduction

Windows 11, CPython 3.13, cp1252 locale:

>>> from graphify import prs
>>> prs._gh("pr", "view", "1281", "--repo", "safishamsi/graphify", "--json", "title")
# before: TypeError: the JSON object must be str, bytes or bytearray, not NoneType
# after:  {'title': 'docs: add Persian (فارسی) translation and language selector link'}

Tests

tests/test_prs.py::TestSubprocessOutputEncoding asserts encoding="utf-8" on each of the five calls, mirroring the existing tests/test_charmap_encoding.py. They are red without the fix (the four behavioural asserts fail with assert None == 'utf-8'). A guard test confirms the fixture bytes are genuinely cp1252-undecodable, so the tests exercise the real failure surface. A behavioural decode test can't be cross-platform here, since Linux CI decodes UTF-8 regardless, so the tests assert the contract instead.

Scope

Five subprocess calls in prs.py; no behaviour change on Linux or macOS, where UTF-8 is already what they decode with. Other modules have their own text=True calls but decode ASCII-only output (git hashes, refs), so I left them out to keep this focused. Happy to harden those in a follow-up if you'd like.

…2 codec

`graphify prs` reads gh, git, and claude output via subprocess.run(text=True)
with no explicit encoding=, so on Windows stdout is decoded with the cp1252
locale codec. gh emits UTF-8 JSON whose PR titles and logins routinely carry
non-Latin1 bytes (emoji, or the Persian title in PR Graphify-Labs#1281), and cp1252 has no
mapping for bytes such as 0x81. The capture reader thread raises
UnicodeDecodeError, subprocess.run then returns stdout=None, and the caller dies
one line later on json.loads(None) (TypeError) or None.splitlines()
(AttributeError). Neither is caught by _gh's except clause, so the whole `prs`
command aborts, with a spurious reader-thread traceback on stderr, against any
repo that has non-ASCII PR metadata.

Pass encoding="utf-8", errors="replace" to all five subprocess reads in prs.py
so decoding matches Linux and macOS. This is the decode-side sibling of Graphify-Labs#1505,
which applied the same change to the llm.py claude-cli subprocess. CI runs Linux
only (a UTF-8 locale), so this path is never exercised there.

Reproduced on Windows 11 / CPython 3.13 with a cp1252 locale:
prs._gh("pr", "view", "1281", ...) crashed with TypeError before and returns the
decoded title after. The added regression tests assert encoding="utf-8" on each
call, mirroring tests/test_charmap_encoding.py, and are red without the fix.
@LukeTheoJohnson
LukeTheoJohnson marked this pull request as ready for review July 17, 2026 23:05
safishamsi added a commit that referenced this pull request Jul 18, 2026
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@safishamsi

Copy link
Copy Markdown
Collaborator

Landed on v8 (ships in 0.9.19) in a86666a, cherry-picked to preserve your authorship. Thanks @luke J — all gh/git/claude subprocess reads in prs.py now decode as UTF-8 with errors=replace (matching #1505), fixing the Windows cp1252 mojibake/crash on emoji and non-ASCII PR titles/diffs.

@safishamsi safishamsi closed this Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants