fix(java): compile and document as UTF-8, not the platform default - #18
Merged
Conversation
The generated sources are UTF-8 and carry the contract's own punctuation — em dashes, ellipses, curly quotes — lifted straight from the OpenAPI descriptions. javac and javadoc default to the JVM's PLATFORM encoding, which on a CI runner is US-ASCII. Every such character becomes `?`, javac emits an `unmappable character (0xE2)` line per occurrence, and because those are warnings rather than failures the build stays GREEN while the sources and javadoc jars published to Maven Central ship the corruption. Observed on opendpp-node's SDK regen verification: hundreds of error-looking lines across src/main and src/test, with the echoed source showing the damage directly — "Developers ??? API keys", "Tenant identity is token-bound ??? it is derived from your API key". Bytecode is unaffected (the damage is in comments), so this is cosmetic — but cosmetic on a public artifact carrying the company name, and it worsens with every typographic character added to the contract. `options.encoding` on JavaCompile covers main and test compilation. Javadoc needs all three: `encoding` is how it READS the sources, `charSet` and `docEncoding` are how it WRITES the HTML — set one and the jar is still corrupted at the other end. Fixed here rather than by setting LANG in the consuming workflow: this repo is where the Maven artifact is actually published, and the same corruption hits anyone building locally on an ASCII-defaulted machine. Fixes OpenDPP/opendpp-node#1156
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Pins the Java build to UTF-8 for compilation and javadoc, instead of inheriting the JVM's platform encoding.
plus
encoding/charSet/docEncodingonJavadoc.Why
The generated sources are UTF-8 and carry the contract's own punctuation — em dashes, ellipses, curly quotes — lifted straight from the OpenAPI descriptions.
javacandjavadocdefault to the platform encoding, which on a CI runner isUS-ASCII. Every such character becomes?, javac emits anunmappable character (0xE2)line per occurrence, and — because those are warnings rather than failures — the build stays green while the artifacts ship the corruption.Observed on opendpp-node's SDK regen verification: hundreds of error-looking lines across
src/mainandsrc/test, with the echoed source showing the damage directly:0xE2is the UTF-8 lead byte for—/…/’.Impact. Bytecode is unaffected — the damage is in comments — so this is cosmetic. But it is cosmetic on a public artifact carrying the company name: the sources and javadoc jars on Maven Central contain
???where punctuation belongs, and anyone reading the SDK in an IDE sees it. It also worsens with every typographic character added to the contract; an upstream PR is about to add a Versioning & compatibility section full of them.Why here rather than in the consumer
opendpp-node's verification job could have set
LANG=C.UTF-8and gone quiet — but this repo is where the Maven artifact is actually published (publish-sdk-java.yml), so that would have hidden the symptom while still shipping corrupted jars. The same corruption also hits anyone building locally on an ASCII-defaulted machine. Fixing the build is the fix; an env var in one consumer is not.Verification
I could not build locally (no JDK on this machine), so this leans on CI: the
Java SDK — generate-drift, version-lock, build, testjob compilessrc/mainandsrc/testand builds the javadoc jar. The expected signal is the disappearance of everyunmappable characterline from that job's log — previously hundreds, now none.Worth a reviewer's eye on one thing I could not check: whether any generated string literal (as opposed to a doc comment) carries non-ASCII. If so this was never merely cosmetic, and the fix matters more than stated above.
Fixes OpenDPP/opendpp-node#1156