Feature/update vocab, updating vocab to version 0.3.0 - #96
Merged
egekorkan merged 22 commits intoAug 26, 2026
Conversation
…is now expressed as a form-level big/little string instead of a boolean, updated the vocabulary and related documentation with the investigation how often are the various terms used in the device decoding
…actor Records the payload schemas emitted for the curated examples, the decoded values for every test vector, and the reference-device catalog coverage plus a per-device round-trip digest. Extracts convert_catalog() from generate_device_tds so the snapshot builder shares the catalog walk instead of duplicating it.
Member
|
Let's use this repository going forward as the reference. I am putting my feedback based on the first comment of this PR and not the git diff.
|
egekorkan
reviewed
Aug 24, 2026
| For a `ports` layout, don't forget to pass the frame port: | ||
|
|
||
| `--fport` is required for a `ports` layout, because the frame port selects which | ||
| layout to apply. Other layouts ignore it. |
Member
There was a problem hiding this comment.
Suggested change
| layout to apply. Other layouts ignore it. | |
| layout to apply; without it the interpreter cannot select the right field set Other layouts ignore it. |
This was referenced Aug 24, 2026
Reviewer feedback on eclipse-thingweb#96: this repository is the reference for the LoRaWAN binding going forward, so the six curated `*.td.json` + `*.vectors.json` pairs belong here, reviewed like any other source file. They were previously ignored and mirrored in from eclipse-thingweb/examples on demand, which meant the files the test suite ran against were whatever happened to be on disk. `sync-examples` is removed rather than kept: it copies upstream over the local files, so leaving it would let a stale mirror silently overwrite the canonical copies. The mirror direction is now outbound. The examples-missing skip guard in test_golden goes with it - the examples can no longer be absent, and a guard that never fires only hides the day it should.
Reviewer feedback on eclipse-thingweb#96, in three parts. schema:brand / schema:model become schema:manufacturer / schema:mpn. schema:brand ranges over Brand and schema:model over ProductModel, so both invite a nested object where a device repository carries a plain identifier. schema:manufacturer and schema:mpn (manufacturer part number) are what a catalogue entry actually holds. The hardware and firmware revisions move off TD core's version object onto schema:version and schema:softwareVersion. TD core's version/model and version/instance version the *Thing Description*: they say which revision of the document you are holding, not which firmware the device is running. A TD can be revised without the device changing at all, which is exactly the moment you would want to read the firmware version and could not. No lorav: term changes; all four names were already withdrawn in 0.3.0, so this only retargets what REMOVED_TERMS points at and what the migration script emits. Also from review: * README: say why --fport is required for a ports layout (the frame port selects which layout applies, so without it the interpreter cannot tell which field set the bytes belong to) rather than restating that it is. * README: document lorav:derived with two worked examples from the Decentlab catalogue -- a ref + polynomial fit and a guarded compute -- and a table of what each of the five keys means, since "this value is computed rather than read from the wire" was not readable from the one-line mention it had.
The AM102 example was written from the vendor codec rather than from a device, and its payloads were plausible-looking rather than observed. The EM300-TH shares that codec byte-for-byte, so swapping the example costs nothing in coverage and buys uplinks that were actually received on the air, on fPort 85: 0367F800046862 -> temperature 24.8, humidity 49.0 0175640367F900046862 -> battery 100, temperature 24.9, humidity 49.0 The synthetic negative-temperature vector is kept because no captured frame covers the sign-extension path, and that path is the one most likely to regress. Renames em300-zld to milesight-em300-zld so both Milesight examples are vendor-qualified and sort together, and fixes two long-standing defects in its vectors file: the description was copy-pasted from the AM102, and both cases were named "nominal_reading", which meant the golden snapshot silently recorded only the second one. Naming them apart restores the first case to the snapshot. Also finishes 17fe113 in the examples: four of them still carried device hardware and firmware revisions in TD core's "version" object, which that commit's README now says versions the *document*. Moved to schema:version and schema:softwareVersion so the examples agree with the prose.
Categorical readings were emitted as a data schema of "oneOf" entries
pairing "const" with "title", chosen so the binding would not have to
mint a term for something TD core looked able to express. Reviewing it
against the WoT BACnet and Modbus bindings showed the reasoning was
right but the placement was wrong, and measuring it showed the result
was broken: all 21 such schemas across the examples are unsatisfiable.
{"type": "string", "oneOf": [{"const": 0, "title": "normal"}]}
rejects the decoded "normal" and the raw 0 alike. "const" held the wire
integer inside the schema of the decoded value, so the schema described
neither end, and "title" -- a display label with a multi-language
"titles" sibling -- was carrying machine-readable data, so rewording or
translating a Thing Description silently changed what its payloads
decoded to. Nothing caught this because the decode tests compared
decoded values only against the vectors, never against the TD.
Split it the way BACnet does. The values a reading may take stay in the
data schema as a plain TD core "enum"; which wire encoding yields which
of them moves to the form as lorav:valueMap, a list of {wireValue,
value} pairs. That correspondence is a fact about the transfer, which
TD core has no vocabulary for and a protocol binding exists to state --
the categorical counterpart of lorav:wireType. BACnet reaches the same
split with bacv:hasValueMap / bacv:hasProtocolVal / bacv:hasLogicalVal,
Modbus with modv:type. lorav:enum therefore becomes a rename rather
than a withdrawal, and the migration script now splits an 0.2.x table
in two, retyping the event to match the labels it now carries.
tests/test_decode.py validates every vector's decoded value against its
own event's data schema, closing the gap that let this through.
No behaviour changes: every payload schema, lookup table and decoded
value across the 157 catalog devices and 7 curated examples is
byte-identical. The only golden additions belong to the new example.
Also adds the MClimate Multipurpose Button example, which exercises
affine raw-byte scaling (lorav:multiplier with lorav:addend) and a
lorav:bitmask status bit against the official TTN codec.
The thermistor status bit is set when the thermistor is *absent*, which the example expressed as "lorav:multiplier": -1 with "lorav:addend": 1 -- arithmetic that turns 1 into 0 without ever saying that 1 meant "disconnected". The polarity survived only in the prose description, and the decoded value was a bare 0 or 1 that a consumer still had to interpret. A lorav:valueMap says it directly: the data schema offers "connected" / "disconnected" and the form maps the bit onto them. pressEvent is deliberately left as a raw integer, which surfaced a limitation worth recording. The MultiTech interpreter applies a lookup positionally -- 'if 0 <= value < len(lookup)' -- so it reads the table as a list indexed by the wire value rather than as a mapping. The button numbers its press codes 1..3, so that table has length 3 and wire value 3 falls outside the guard, decoding to the bare integer 3 rather than "triple". Three RadioBridge rbs30x events in the generated catalog have the same shape. This is pinned by a test rather than asserted away, so a future submodule bump that fixes the interpreter will fail it and tell us the gap has closed. The same test records that the failure is at least detectable: the leaked wire value does not validate against its own data schema, which is precisely the check the withdrawn oneOf/const spelling could never have provided. Also replaces a tautological test added with lorav:valueMap, which validated a literal against a literal and would have passed no matter what the converter did, with one that asserts the converter leaves the data schema alone.
…ed tier The bit says whether the thermistor is connected, so the honest decoded value is true/false rather than the string pair it was given. TD core's "boolean" already enumerates its own value space, so the data schema needs no 'enum' beside it -- which shows that a value map does not depend on one, and the 'value' of an entry may be any JSON type. Generalised the wording in vocab.py, the form schema and the ontology accordingly; each had said the mapped value must appear in the data schema's 'enum'. Booleans had not appeared in a test vector before, and Python's True == 1 meant test_decode_matches_expected would have accepted a decoder that returned the raw bit instead of mapping it. Compares booleans by identity, before the numeric branches. Corrects the Tier column for lorav:valueMap from "common" to "rare". The column is defined as frequency across the bundled device catalog, and a census puts the term in 4 of 157 devices (2.5%) -- the same share as lorav:derived, which is listed as rare. The figure was carried over rather than measured when the term was added. Every other row in the table was checked against the same census and is accurate. Also trims the two README passages added with the term, which restated the argument the surrounding section had already made.
The two Milesight examples describe the same wire format with the same tag fields, yet one said "ctv" and the other "tlv". Nothing in the converter reads the difference -- both fall into the same branch -- so the two names produced byte-identical schemas and the disagreement was noise a reader had to resolve. 'tlv' wins on weight of usage: all 85 tagged devices in the generated catalog carry it, and 'ctv' now appears in no document at all. The golden snapshot is unchanged by this commit, which is the evidence that the rename decides only what the file is called. The same example was also the only one in the repository with a form href other than "uplink" -- it used "ctv", repeating the layout name in a slot that holds a resource locator relative to the Thing's base. The other 28 curated forms and all 1623 generated ones use "uplink". Adds a test pinning ctv and tlv to the same output. The alias is invisible in the corpus now, so a reader can reasonably assume it does something; the test says it does not, and would fail if a later change gave one name behaviour the other lacks. vocab.py and the README record the same, including the reason the distinction does not arise: the names differ over whether a length travels with each value, and the width comes from the wire type either way.
Contributor
Author
|
Ready to be merged. Breaking revision of the vocab. (There will still be small iterations on the vocab in the future) addressing the comments @egekorkan from above
One term added:
"leakage_status": {
"data": { "type": "string", "enum": ["normal", "leak"] },
"forms": [{
"lorav:wireType": "u8",
"lorav:valueMap": [
{ "wireValue": 0, "value": "normal" },
{ "wireValue": 1, "value": "leak" }
]
}]
}Add the example TD back to the examples folder A new example TD is added: More information is in the README under /python/lorawan |
The sentence named the layout that needs the flag but not what goes wrong without it, which is the part a reader hits at the command line.
Hao-Siemens
marked this pull request as ready for review
August 26, 2026 12:09
egekorkan
reviewed
Aug 26, 2026
Member
|
This is looking good now. There are some small things to update in the examples but I will do a bigger pass through everything later on. We need this PR in so that the rest of the PRs can be incorporated |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Version 0.3.0 moved decoded values from
propertiestoeventsand renamed orwithdrew the terms below. Also the corresponding toolchain, documentation, and examples are updated. And the vocab is simplied to reuse existing terms from TD and schema.
propertieseventslorav:typelorav:wireTypetypelorav:offsetlorav:addendlorav:byteOffsetlorav:lengthlorav:byteLengthlorav:presenceField,lorav:presenceBitlorav:presentWhen: { field, bit }lorav:switchField,lorav:switchValuelorav:presentWhen: { field, value }lorav:varlorav:aliaslorav:ref,lorav:polynomial,lorav:compute,lorav:guard,lorav:transformlorav:derived: { … }lorav:validRangedata.minimum/data.maximumlorav:enumdata.oneOfwithconst+titlelorav:unecedata.unitunitalready carries UN/CEFACT codeslorav:brand,lorav:modelschema:brand,schema:modellorav:hardwareVersion,lorav:softwareVersionversion.model,version.instancelorav:endDeviceIdidortitle