Multiple hardening and expansion improvements to the package - #46
Open
gurkohli wants to merge 2 commits into
Open
Multiple hardening and expansion improvements to the package#46gurkohli wants to merge 2 commits into
gurkohli wants to merge 2 commits into
Conversation
Make the BLE HID client descriptor-driven, lossless, and robust enough to support multiple HID peripherals and composite report layouts. The previous implementation made several assumptions about GATT discovery, Report IDs, field widths, and parser state. These assumptions could cause valid characteristics or events to be ignored, misdecode packed and signed fields, suppress repeated relative inputs, or read beyond malformed and truncated packets. GATT and connection handling ---------------------------- - Discover and subscribe to every HID Report characteristic rather than assuming a single report handle. - Read each Report Reference descriptor and associate its Report ID and Report Type with the originating characteristic. - Discover Boot Keyboard and Boot Mouse input characteristics and preserve their traffic as raw events until fixed-format decoders are implemented. - Track notification registration independently for every characteristic. - Handle devices with no subscribable Report characteristics without leaving the component stuck in the registration state. - Do not block component readiness on the optional connection-parameter update callback. - Filter GATT events by client interface, connection ID, peer address, and characteristic handle so multiple BLE clients on one ESP do not consume each other's events. - Clear discovered handles, pending reads, report metadata, and parser state on disconnect so reconnection performs clean discovery. - Recover from characteristic-read and notification-registration failures instead of leaving pending operations indefinitely. - Validate characteristic data before accessing battery, PnP ID, Report Reference, and string values. - Correct PnP ID decoding and safely construct non-null-terminated GATT strings. Report provenance and lossless events ------------------------------------- Attach the source of every notification to its decoded Home Assistant event: - BLE peer address - GATT handle - characteristic UUID - Report ID - HID Report Type - notification or indication transport - packet length - raw packet bytes - timestamp Decoded events retain the existing `esphome.hid_events` event name and the existing `usage` and `value` fields. Additional numeric fields include `usage_page`, `usage_id`, `raw_value`, and `relative`. When a valid notification cannot be decoded, emit its complete raw envelope instead of silently discarding it. This includes reports with unknown Report IDs, unsupported usages, non-Input Report Types, Boot Protocol reports, and traffic received before a usable Report Map is available. Including the BLE peer address also makes events from multiple HID devices on the same ESPHome node distinguishable. HID parser correctness ---------------------- - Pass packet lengths through the parsing API and bounds-check every field. - Reject truncated short items and safely skip valid HID long items. - Avoid undefined shifts and integer overflows while decoding bit fields. - Decode signed and unsigned fields up to 32 bits, including packed, non-byte-aligned values and unsigned values above INT32_MAX. - Widen report sizes, counts, offsets, logical ranges, and decoded values so valid descriptors are not truncated to eight or sixteen bits. - Validate Report IDs and route notifications using Report Reference metadata. - Decode only Input layouts as input events; Output and Feature report characteristics remain visible through their raw envelopes. - Preserve every nonzero relative delta, including consecutive identical Wheel, Pan, and pointer movements. - Suppress unchanged absolute values while continuing to emit releases. - Track array controls by active usage rather than array slot, preventing keyboard reordering from producing false release/press pairs. - Support repeated final usages for variable fields whose Report Count exceeds the explicit usage list. - Correct local usage range handling, logical-minimum interpretation, global PUSH/POP state, delimiter handling, and local-state cleanup after Main items. - Add ownership and overflow checks when constructing parsed report layouts. Testing and maintenance ----------------------- - Add native C++ tests for signed HID item values and descriptor parsing. - Add a device-neutral composite HID fixture covering multiple Report IDs, packed signed 12-bit axes, buttons, Wheel, AC Pan, arrays, relative values, Output/Feature routing, and release behavior. - Add truncated-descriptor cases and deterministic malformed-input smoke fuzzing. - Build native tests with `-Wall -Wextra -Werror`. - Add an ESPHome firmware fixture containing two independent BLE HID clients to exercise multi-device code generation. - Add strict ESP32 and ESP32-S3 CI builds that reject component compiler warnings. - Add reproducible setup, test, and build targets through the Makefile and pinned build requirements. - Rewrite the README around installation, multi-device configuration, event fields, development, diagnostics, current limitations, and roadmap. Compatibility ------------- No new YAML options are required. Existing single-device configurations, optional sensors, the `esphome.hid_events` event name, and the `usage`/`value` event fields remain supported. Consumers should tolerate the additional event metadata. Output and Feature report writes, Protocol Mode switching, and semantic Boot Keyboard/Mouse decoding remain out of scope. Their traffic is now observable and attributable so those features can be added without device-specific heuristics. Verification ------------ - Native parser and value tests pass with warnings treated as errors. - Parser tests pass under AddressSanitizer and UndefinedBehaviorSanitizer. - ESPHome configuration validation passes with two BLE HID clients. - Strict ESP-IDF builds pass for ESP32 and ESP32-S3.
Turn the component into a device-neutral HID-over-GATT inspector and decoder that can handle unfamiliar and more complex peripherals without relying on device-specific assumptions. Discovery and connection lifecycle ---------------------------------- - Inventory complete GATT structures, including multiple HID services, included services, descriptors, and 16-, 32-, and 128-bit UUIDs. - Add `discovery_mode`; the default `standard` policy reads relevant standard attributes, while opt-in `forensic` mode also reads other readable values and subscribes to advertised notification sources for troubleshooting. - Add `protocol_mode` so users can leave the peripheral unchanged or request Report or Boot Protocol using the standard characteristic. - Serialize and bound GATT work, report degraded readiness when optional operations fail, and prevent setup from waiting indefinitely. - Track notification registration and CCCD enablement separately, including a component-owned CCCD fallback for ESPHome versions that do not perform the write and one retry after a failed framework-owned write. - Subscribe to Battery Level and Service Changed where available. Service Changed now invalidates the cached profile and triggers clean rediscovery. - Capture target advertisements and scan responses and add stable diagnostic records for GATT topology, values, HID schema, subscriptions, and readiness. - Generate handle-independent GATT and Report Map fingerprints to make profile comparisons across reconnects and firmware revisions practical. HID schema and report handling ------------------------------ - Retain collection hierarchy and complete Input, Output, and Feature field metadata, including physical ranges, units, strings, designators, usage aliases, and Main-item flags. - Assign stable service, collection, and field identifiers so events can be traced back to the schema that produced them, including duplicate usages at different bit offsets. - Validate full report lengths before changing parser state. Short reports are rejected without partial transitions; long reports decode their defined portion and are marked accordingly. - Add fixed-format Boot Keyboard and Boot Mouse decoding with independent transition state per characteristic. - Preserve unsupported, schema-less, mismatched, and unknown traffic as raw envelopes instead of guessing at its meaning or silently dropping it. - Give every incoming notification one monotonic sequence ID and log its raw bytes before routing or decoding. Emitted events carry the source sequence span, decode status, profile identity, service and field provenance, and application collection while keeping the existing event name and core `usage`/`value` fields. Ordered event coalescing ------------------------ Add the optional `event_sampling_interval` setting for noisy or high-frequency inputs. Relative scalar values are summed, suitable absolute values retain the latest sample, and buttons, keys, arrays, raw reports, and other discrete events act as ordering barriers. The bounded buffer keeps first-observed order and flushes accumulated values before a later barrier. This means a sequence such as: press, +1, +2, release is emitted as: press, +3, release rather than allowing the release to overtake the movement. Sequence ranges show which notifications contributed to a coalesced event, and saturating addition prevents relative-value overflow. A zero interval remains the default and preserves immediate event delivery. Usage data and documentation ---------------------------- - Update the source data to USB-IF HID Usage Tables 1.7. - Replace the large collection of generated headers and runtime maps with one sorted, flash-resident table and binary-search lookup functions. This also supports usage page IDs wider than eight bits and exposes published usage kinds for aggregation decisions and diagnostics. - Rewrite the README around configuration, lifecycle and readiness, forensic logging, raw-event provenance, privacy considerations, coalescing behavior, compatibility, development, and deliberate limitations. Testing ------- - Cover complete descriptor metadata, duplicate usages, malformed collection state, strict packet lengths, Boot keyboard/mouse transitions, ordered and saturating coalescing, subscription fallback and timer wraparound, and HID Usage Table lookup. - Add a sanitizer target for the native suite and compile firmware fixtures with sampling, forensic discovery, and explicit Report Protocol settings. Compatibility ------------- Existing configurations continue to use standard discovery, leave Protocol Mode unchanged, and emit events immediately. The `esphome.hid_events` name, the existing compatibility fields, and the optional battery and last-event entities remain available. Forensic discovery and event coalescing are opt-in. Arbitrary Output and Feature writes remain out of scope. Their layouts and traffic are retained for diagnostics, but unknown vendor data is never interpreted or written except for standard CCCDs. Verification: `make test`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.