Accessibility: Lazarus tracker targets + audio/speech toggle hotkeys#8474
Open
mojsior wants to merge 64 commits intodiasurgical:accessfrom
Open
Accessibility: Lazarus tracker targets + audio/speech toggle hotkeys#8474mojsior wants to merge 64 commits intodiasurgical:accessfrom
mojsior wants to merge 64 commits intodiasurgical:accessfrom
Conversation
Removed translations for 'Amulet', 'Chest', and 'Inventory'.
If these needs to appear with different translations then the other instances they will need to use the context keyword P_("bodypart", "Chest") instead of _("Chest").
These files had mixed CRLF/LF line endings. Normalize to CRLF to match the .editorconfig convention for C++ source files. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Allow the player to automatically walk toward the currently selected tracker target by pressing M. Supports all tracker categories: items, chests, doors, shrines, objects, breakables, and monsters. The walk stops when the target is within interaction range or becomes invalid. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix OBJ_SIGNCHEST validation mismatch (Chests case now uses ValidateAutoWalkObjectTarget matching IsTrackedChestObject) - Add spoken feedback for all silent early returns (bounds checks, MyPlayer nullptr) - M key now toggles: press again to cancel in-progress auto-walk - A/Shift+A cancels auto-walk silently before attacking - Auto-walk cancels when in-game menu opens - Extract IsTrackedMonster() predicate (replaces 3 inline checks) - Remove default: from Monsters switch cases to enable -Wswitch exhaustiveness checking - Add defensive re-validation in ResolveObjectTrackerTarget - Add documentation comments Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CancelAutoWalk() now sends a walk-to-current-position command to immediately clear the walk path buffer, instead of only preventing new segments. Move the M key cancellation check above the CanPlayerTakeAction guard so it works mid-walk. Add CancelAutoWalk() calls to secondary action (D), spell action (W), and arrow-key walk so all player inputs properly cancel an in-progress auto-walk. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use object->position instead of the archway tile position when reporting door locations in FindFirstClosedDoorOnWalkPath and FindFirstTrackerPathBlock. Archway tiles (negative dObject refs) resolve to the parent door object, but the returned position was the archway tile rather than the door itself, causing the player to stop one tile short. Also: move DOOR_CLOSED/DOOR_OPEN/DOOR_BLOCKED enum from objects.cpp to objects.h so diablo.cpp can use the canonical constants directly instead of maintaining duplicate constexpr values. Add static_assert for MAXOBJECTS fitting in int8_t, defensive assertions in AddDoor(), and clarify comments around door state checks. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…peech Fix inconsistent keyboard navigation when moving through multi-tile items in the inventory. Previously, navigating into a large item (e.g., a 3x2 axe) and then back out would exit from a different row than you entered from. Changes: - Add entry point tracking to remember which row/column you entered an item from - Use tracked entry points when exiting items to maintain navigation consistency - Add row and column position to accessibility speech (e.g., "Row 1, Column 3: Short Sword") Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Always enter belt at slot 1 when navigating down from inventory - Return to exact inventory position when navigating up from belt - Keep left/right navigation within belt area only - Track last inventory slot before entering belt with SlotBeforeBelt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove slot tracking when entering/exiting the belt. Now consistently returns to row 4 column 1 when navigating up from the belt. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move ~4,100 lines of accessibility code from diablo.cpp into 6 new modules, reducing diablo.cpp from ~7,700 to ~3,660 lines. New modules: - controls/accessibility_keys: UI key handlers and guard functions (IsPlayerDead, IsGameRunning, CanPlayerTakeAction, etc.) - utils/walk_path_speech: BFS pathfinding, PosOk variants, walk direction helpers, and path-to-speech formatting - utils/accessibility_announcements: periodic announcements for low HP warning sound, durability, boss health, monsters, doors - controls/town_npc_nav: town NPC selection cycling and auto-walk - utils/navigation_speech: exit/stairs/portal/unexplored speech and keyboard walk key handlers - controls/tracker: target category cycling, candidate finding, pathfinding navigation, and auto-walk tracker Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove ~4,042 lines of extracted function bodies and forward declarations from diablo.cpp. Add includes for the 6 new module headers. Update CancelAutoWalkInternal() to delegate to ResetAutoWalkTracker() and ResetAutoWalkTownNpc(). Remove unused includes from diablo.cpp that were only needed by the extracted code (<queue>, format_int.hpp, tile_properties.hpp). Add the 6 new .cpp files to Source/CMakeLists.txt so they are compiled into libdevilutionx. Also includes post-review cleanups: - Remove unused automap.h include from accessibility_keys.cpp - Remove unused walk_path_speech.hpp include from diablo.cpp - Replace .size() == 0 with .empty() in accessibility_keys.cpp - Simplify CanAutomapBeToggledOff to direct return expression - Remove unreachable dead code and dead FindPreferredExitTriggerIndex function from navigation_speech.cpp Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix copy-paste comment errors in SpellBookKeyPressed() that referred to "inventory" instead of "spellbook", and QuestLogKeyPressed() that said "character quest log" instead of "quest log". Correct a misleading FindPath comment in town NPC auto-walk, and update a stale SourceX path reference in diablo.cpp. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The variable is only used within tracker.cpp, so remove the unnecessary extern declaration from the header and place it in the anonymous namespace alongside the other file-local tracker state. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Guard against MyPlayer being nullptr in IsPlayerDead(), CycleSpellHotkeys(), RefreshTownNpcOrder(), SpeakSelectedTownNpc(), and ListTownNpcsKeyPressed() to prevent crashes during early init or after disconnect. Also reset AutoWalkTrackerTargetId when MyPlayer is null in the auto-walk tracker to prevent the walk loop from retrying endlessly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a LogWarn call when none of the expected audio file paths for the low HP warning sound can be loaded, making it easier to diagnose missing or misplaced sound assets. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Guard against MyPlayer being nullptr in UpdateAutoWalkTownNpc() to match the pattern used in UpdateAutoWalkTracker(). Without this, MyPlayer could be dereferenced if CanPlayerTakeAction() changes its null-handling behavior in the future. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…NPC nav When MyPlayer is null, the town NPC navigation functions now provide spoken feedback instead of returning silently, matching the pattern used in the tracker auto-walk code. RefreshTownNpcOrder also resets SelectedTownNpc to avoid leaving stale state. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Hold Shift when pressing the health percentage key to hear mana instead. Extract ComputePercentage helper to deduplicate the HP/mana calculation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace 7 individual navigation/tracker key registrations with 3 unified tracker keys: PageUp (previous), PageDown (next), Home (navigate/walk). Unbind legacy town NPC nav keys (now superseded by tracker Npcs category). Update health key description to mention Shift+mana. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…d modules Upstream added new tracker categories and speech fixes inline in diablo.cpp. Our branch has already extracted that code into separate accessibility modules, so all 6 conflict regions are resolved by keeping our slimmed-down version. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove the legacy town_npc_nav tracker system which was superseded by the unified tracker. The old tracker was still firing on PgUp/PgDown due to hardcoded fallback logic in PressKey(). Changes: - Remove PgUp/PgDown fallbacks in PressKey() (keep store/chat handling) - Remove 5 keymapper registrations (ListTownNpcs, PreviousTownNpc, NextTownNpc, SpeakSelectedTownNpc, GoToSelectedTownNpc) - Remove UpdateAutoWalkTownNpc() and ResetAutoWalkTownNpc() calls - Remove migration code references in options.cpp - Delete town_npc_nav.cpp and town_npc_nav.hpp - Update CMakeLists.txt Town NPC tracking is now handled by the unified tracker's NPCs category. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add WM_DIABTWARPUP to trigger message checks so the tracker's entrances/exits and stairs categories detect town warps from Catacombs L1, Caves L1, Hell L1, Hive, and Crypt back to town. Previously only regular stairs up (WM_DIABPREVLVL) were detected. TriggerLabelForSpeech already handles WM_DIABTWARPUP and returns "Warp up", so labeling works automatically. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ntegration Add comprehensive Android accessibility support following RetroArch's proven architecture to enable screen reader functionality for visually impaired players. The implementation uses Android's native accessibility framework (announceForAccessibility) to integrate with TalkBack and other screen readers, providing the same accessibility features available on Windows (Tolk) and Linux (speech-dispatcher). ## Functionality Status ✅ **Working:** - Compiles successfully for all Android architectures - TTS starts correctly after game data loading - TTS interrupts speech when needed (using appropriate `force=true`) - TTS uses Android's default text-to-speech engine⚠️ **Known Issues (mod-related, not TTS):** - Stats screen navigation: pressing C or equivalent button focuses on "Strength" and prevents navigation through other attributes - Tracking keys provide incorrect/imprecise directions, preventing proper navigation to desired destinations - Mod sounds (doors, chests) not playing - likely due to incorrect file paths or Android-specific issues See #12 for details on known issues. ## Usage Instructions⚠️ **IMPORTANT:** To play, you must suspend the screen reader (TalkBack) after loading the game data (.mpq). The accessibility mod takes control of TTS through Android's native API. ## Key Technical Changes • Added JNI bridge between C++ game code and Java accessibility APIs • Implemented thread-safe JNIEnv caching with pthread thread-local storage • Created native methods for screen reader detection and text-to-speech • Fixed initialization order: nativeInitAccessibility() must be called after super.onCreate() to ensure SDL loads the native library first • Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration ## Technical Details - Java Layer (DevilutionXSDLActivity.java): • isScreenReaderEnabled(): Checks TalkBack and touch exploration state • accessibilitySpeak(): Uses announceForAccessibility() API - JNI Bridge (Source/platform/android/android.cpp): • GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment • SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++ • IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code - Platform Integration (Source/utils/screen_reader.cpp): • Routes accessibility calls to platform-specific implementations • Maintains consistent API across Windows, Linux, and Android ## Architecture Notes - Follows RetroArch's accessibility implementation pattern - Uses global JNI references for thread-safe Activity access - Caches method IDs to avoid repeated JNI lookups - Handles thread attachment/detachment automatically - No external dependencies required (Android SDK only) ## Documentation Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md Fixes startup crash caused by calling native methods before library loading. Verified on physical device: app launches successfully, accessibility features functional. Related files: • CMake/Dependencies.cmake: Skip external dependencies for Android • Source/CMakeLists.txt: Add Android-specific screen reader configuration • android-project/app/build.gradle: Enable SCREEN_READER_INTEGRATION flag
…ntegration Add comprehensive Android accessibility support following RetroArch's proven architecture to enable screen reader functionality for visually impaired players. The implementation uses Android's native accessibility framework (announceForAccessibility) to integrate with TalkBack and other screen readers, providing the same accessibility features available on Windows (Tolk) and Linux (speech-dispatcher). ## Functionality Status ✅ **Working:** - Compiles successfully for all Android architectures - TTS starts correctly after game data loading - TTS interrupts speech when needed (using appropriate `force=true`) - TTS uses Android's default text-to-speech engine⚠️ **Known Issues (mod-related, not TTS):** - Stats screen navigation: pressing C or equivalent button focuses on "Strength" and prevents navigation through other attributes - Tracking keys provide incorrect/imprecise directions, preventing proper navigation to desired destinations - Mod sounds (doors, chests) not playing - likely due to incorrect file paths or Android-specific issues See #12 for details on known issues. ## Usage Instructions⚠️ **IMPORTANT:** To play, you must suspend the screen reader (TalkBack) after loading the game data (.mpq). The accessibility mod takes control of TTS through Android's native API. ## Key Technical Changes • Added JNI bridge between C++ game code and Java accessibility APIs • Implemented thread-safe JNIEnv caching with pthread thread-local storage • Created native methods for screen reader detection and text-to-speech • Fixed initialization order: nativeInitAccessibility() must be called after super.onCreate() to ensure SDL loads the native library first • Enabled SCREEN_READER_INTEGRATION flag in Android CMake configuration ## Technical Details - Java Layer (DevilutionXSDLActivity.java): • isScreenReaderEnabled(): Checks TalkBack and touch exploration state • accessibilitySpeak(): Uses announceForAccessibility() API - JNI Bridge (Source/platform/android/android.cpp): • GetJNI(): Thread-safe JNIEnv retrieval with automatic thread attachment • SpeakTextAndroid(): Calls Java accessibilitySpeak() from C++ • IsScreenReaderEnabledAndroid(): Queries TalkBack status from native code - Platform Integration (Source/utils/screen_reader.cpp): • Routes accessibility calls to platform-specific implementations • Maintains consistent API across Windows, Linux, and Android ## Architecture Notes - Follows RetroArch's accessibility implementation pattern - Uses global JNI references for thread-safe Activity access - Caches method IDs to avoid repeated JNI lookups - Handles thread attachment/detachment automatically - No external dependencies required (Android SDK only) ## Documentation Added comprehensive technical documentation in docs/ANDROID_ACCESSIBILITY.md Fixes startup crash caused by calling native methods before library loading. Verified on physical device: app launches successfully, accessibility features functional. Closes #12 ```
…ablo-access into diablo-access-testing
Member
|
Thanks, I'll look in to addressing the conflicts and replace the old PR with one directly from your repo so things are tracked automatically. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation