Add MyAccessibilityService to fill in some notification parsing gaps during startup/launch

.gitignore

@@ -7,3 +7,5 @@ local.properties
 /captures
 .externalNativeBuild
 .cxx
+
+scratch/

docs/Handling Obscured Notifications.md

@@ -0,0 +1,124 @@
+# Handling Obscured Notifications
+
+## What is an "obscured" notification?
+
+An **obscured notification** is one whose content is absent from the
+[`NotificationListenerService`](https://developer.android.com/reference/android/service/notification/NotificationListenerService)
+(NLS) payload — no title, text, or extras — but whose content IS visible in the notification
+shade and IS readable via the
+[`AccessibilityService`](https://developer.android.com/reference/android/accessibilityservice/AccessibilityService)
+accessibility tree.
+
+This happens when an app posts a `GROUP_SUMMARY` notification that carries only metadata
+(flags, groupKey, etc.) and relies on the rendered shade row — not the NLS extras — to display
+content to the user.
+
+---
+
+## Two sub-classes of obscured notification
+
+### 1. Stale-obscured (confirmed, common)
+
+**Behavior during live delivery (app running):**
+The app posts a silent `GROUP_SUMMARY` *immediately followed* by one or more content-bearing
+child notifications (e.g. `CHAT_CHIME`, `MessagingStyle`) within milliseconds. NLS sees both.
+The 300 ms `PENDING_LOOKUP_DELAY_MS` cancellation window in `schedulePendingLookup` gives the
+content-bearing child time to arrive and cancel the accessibility lookup before it fires.
+
+Result: **the normal NLS path speaks the content; the accessibility tree is not consulted for the
+content itself.** Note: `MyAccessibilityService` must still be enabled — it is a hard requirement
+that gates all NLS processing, including live notifications (see
+[Requirement.ACCESSIBILITY_SERVICE] in `StartupCoordinator.kt`).
+
+**Behavior during stale catch-up (app just started):**
+When the app restarts and `initializeActiveNotifications` calls `getActiveNotifications()`, the
+content-bearing child notifications may already be gone — consumed, dismissed, or never
+re-posted. Only the empty `GROUP_SUMMARY` remains. No cancel arrives within 300 ms, so the
+accessibility lookup fires: the shade is opened, collapsed rows are expanded, and the row
+content is read from the accessibility tree.
+
+**Confirmed stale-obscured packages:**
+- `com.google.android.apps.dynamite` — Google Chat. Live delivery: GROUP_SUMMARY + CHAT_CHIME
+  sibling. Stale catch-up: GROUP_SUMMARY only. Accessibility is the sole source of content at
+  catch-up time.
+
+### 2. Always-obscured (theoretical, unconfirmed)
+
+**Hypothesis:** some apps never post a content-bearing sibling notification — not even during
+live delivery. The GROUP_SUMMARY is the only notification they ever post for a given message.
+Content is only available in the shade. These packages would require the accessibility path for
+*both* live and stale delivery, and the `PENDING_LOOKUP_DELAY_MS` cancellation window would
+never fire regardless of whether the app was just started or has been running for hours.
+
+**Theory on which apps this applies to:** if this category exists, it is almost certainly
+limited to **system-signed / AOSP / Google apps**. Third-party apps generally provide readable
+content in the NLS payload or in a sibling notification. System apps often have parallel
+delivery channels (e.g. FCM data messages delivered directly to the app process) and may use
+the `GROUP_SUMMARY` notification solely as a visual shade badge.
+
+**Currently uncharacterized packages** (appeared in logs, live vs. stale behavior not yet
+confirmed):
+- `com.google.android.googlequicksearchbox` — Google search / assistant
+
+---
+
+## Detection and corpus-building
+
+`ObscuredNotificationLogger` records every accessibility lookup outcome
+(`FOUND` / `NOT_FOUND` / `ACCESSIBILITY_UNAVAILABLE`) keyed by `packageName`.
+
+To identify candidates for the **always-obscured** category, look for packages that
+consistently reach the accessibility path even during live notification delivery — not only at
+app startup. A `packageName` that triggers `findRowForAppLabel` on freshly arriving
+notifications (and consistently returns `FOUND`) is a strong candidate.
+
+---
+
+## Architecture summary
+
+```
+onNotificationPosted(sbn)
+  └─ defaultOnNotificationPosted → ParsedIgnored
+        └─ schedulePendingLookup(300 ms)
+              │
+              ├─ [live, stale-obscured] sibling child arrives within 300 ms
+              │     └─ cancelPendingLookup → spoken via NLS path
+              │
+              └─ [stale catch-up, or always-obscured live] 300 ms elapses with no cancel
+                    └─ MyAccessibilityService.findRowForAppLabel(appLabel)
+                          └─ ShadeRowSearchQueue: open shade → expand rows → match appName
+                                └─ callback → speakShadeRows → TTS
+```
+
+Multiple concurrent `findRowForAppLabel` calls (e.g. Chat and Google both firing within 30 ms
+at startup) are serialized by `ShadeRowSearchQueue` — each search waits for the previous to
+finish before the shade is handed to it.
+
+---
+
+## Key constants
+
+| Constant | Location | Value | Purpose |
+|---|---|---|---|
+| `PENDING_LOOKUP_DELAY_MS` | `MyNotificationListenerService` | 300 ms | Window for content-bearing child to cancel the accessibility lookup |
+| `ShadeDelays.FAST.shadeSettle` | `ShadeDelays` | 600 ms | Shade animation settle time (production) |
+| `ShadeDelays.SLOW.shadeSettle` | `ShadeDelays` | 3000 ms | Shade animation settle time (debug/observation) |
+| `ShadeDelays.FAST.preExpand` | `ShadeDelays` | 0 ms | Pause before each row expansion (production) |
+| `ShadeDelays.SLOW.preExpand` | `ShadeDelays` | 2000 ms | Pause before each row expansion (debug/observation) |
+| `MAX_SCROLL_ATTEMPTS` | `ShadeRowSearchQueue` | 10 | Max scroll attempts before giving up on off-screen rows |
+
+Toggle `MyAccessibilityService.DEBUG_SLOW_MODE = true` to use the SLOW delays for visual
+step-by-step observation of the shade automation. Set false and confirm FAST values once
+minimum working delays are established.
+
+---
+
+## Known limitations
+
+- **Multi-account apps:** if two accounts of the same app post GROUP_SUMMARY simultaneously,
+  the first matching shade row wins; the second is not spoken.
+- **Screen locked:** when the screen is locked, expand/collapse behavior via the accessibility
+  tree is unknown and the shade row content may be unavailable.
+- **`com.google.android.gm` (Gmail):** sends `tickerText` with partial content via NLS; the
+  accessibility path is not currently required but a bespoke `NotificationParser` may improve
+  the spoken output.

mobile/README.md

@@ -1,9 +1,22 @@
 
-## Architecture:
+## Architecture
+
+### Startup / permissions
 - Single Activity + Compose UI gate (no navigation trampoline)
 - System SplashScreen API (androidx.core.splashscreen)
-- StartupCoordinator (ViewModel) as single source of truth
-- Requirement enum  → hard gates  (blocks isReady)
-- Advisory enum     → soft hints  (never blocks isReady)
-- ListenerEnabledMonitor → callbackFlow/ContentObserver for instant mid-session revocation detection (no ON_RESUME polling needed for NOTIFICATION_LISTENER specifically)
+- `StartupCoordinator` (ViewModel) as single source of truth
+- `Requirement` enum → hard gates (blocks `isReady`)
+- `Advisory` enum → soft hints (never blocks `isReady`)
+- `ListenerEnabledMonitor` → callbackFlow/ContentObserver for instant mid-session revocation detection (no ON_RESUME polling needed for NOTIFICATION_LISTENER specifically)
 - DisposableEffect + ON_RESUME still re-checks POST_NOTIFICATIONS and BATTERY_OPTIMIZATION which have no observable Settings key
+
+### Notification delivery
+Two paths — see [`notification/parsers/README.md`](src/main/java/llc/lookatwhataicando/notifai/notification/parsers/README.md) for full ground truth.
+
+- **`MyNotificationListenerService`** — primary path; handles all live notifications via `onNotificationPosted` + per-package `NotificationParser` subclasses
+- **`MyAccessibilityService`** — fallback for "obscured" notifications (empty `GROUP_SUMMARY` payload, content only in the shade); **primarily needed at launch** to catch up on notifications that arrived before the app started; largely not needed for live delivery
+
+### Accessibility permission — true scope
+The `ACCESSIBILITY_SERVICE` requirement is a hard gate in `StartupCoordinator`, but its real
+impact is narrow: **without it, only the launch catch-up path is degraded**. Live notifications
+are unaffected. See `StartupCoordinator.Requirement.ACCESSIBILITY_SERVICE` KDoc for details.

mobile/src/androidTest/java/llc/lookatwhataicando/notifai/ExampleInstrumentedTest.kt

@@ -1,13 +1,11 @@
 package llc.lookatwhataicando.notifai
 
-import androidx.test.platform.app.InstrumentationRegistry
 import androidx.test.ext.junit.runners.AndroidJUnit4
-
+import androidx.test.platform.app.InstrumentationRegistry
+import org.junit.Assert.assertEquals
 import org.junit.Test
 import org.junit.runner.RunWith
 
-import org.junit.Assert.*
-
 /**
  * Instrumented test, which will execute on an Android device.
  *

mobile/src/main/AndroidManifest.xml

@@ -79,6 +79,19 @@
                 android:resource="@xml/notification_listener_service_meta" />
         </service>
 
+        <service
+            android:name=".MyAccessibilityService"
+            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
+            android:exported="true"
+            tools:ignore="AccessibilityPolicy">
+            <intent-filter>
+                <action android:name="android.accessibilityservice.AccessibilityService"/>
+            </intent-filter>
+            <meta-data
+                android:name="android.accessibilityservice"
+                android:resource="@xml/my_accessibility_service"/>
+        </service>
+
         <receiver
             android:name=".BootReceiver"
             android:enabled="true"

mobile/src/main/java/llc/lookatwhataicando/notifai/ActiveNotificationsSnapshot.kt

@@ -5,7 +5,6 @@ import android.app.NotificationManager
 import android.content.Context
 import android.service.notification.NotificationListenerService
 import android.service.notification.StatusBarNotification
-import com.smartfoo.android.core.FooString
 import com.smartfoo.android.core.logging.FooLog
 import com.smartfoo.android.core.notification.FooNotification
 

mobile/src/main/java/llc/lookatwhataicando/notifai/AppShutdownManager.kt

@@ -2,7 +2,6 @@ package llc.lookatwhataicando.notifai
 
 import android.app.ActivityManager
 import android.content.Context
-import android.util.Log
 import com.smartfoo.android.core.logging.FooLog
 import java.util.concurrent.atomic.AtomicBoolean
 import kotlin.system.exitProcess

mobile/src/main/java/llc/lookatwhataicando/notifai/DebugShadeScan.kt

@@ -0,0 +1,152 @@
+package llc.lookatwhataicando.notifai
+
+import android.os.Handler
+import android.os.Looper
+import android.view.accessibility.AccessibilityNodeInfo
+import android.view.accessibility.AccessibilityWindowInfo
+import com.smartfoo.android.core.logging.FooLog
+import llc.lookatwhataicando.notifai.DebugShadeScan.Companion.EXPAND_SETTLE_MS
+
+/**
+ * Full top-to-bottom shade scan used when [MyAccessibilityService.DEBUG_FULL_SCAN_MODE] is true.
+ *
+ * Opens the shade, expands every collapsed row one at a time (with a settle delay between each),
+ * scrolls to reveal off-screen rows, and logs every [NotificationShadeSnapshot.ShadeRow] found. No app-label matching;
+ * no TTS. Guarded by [active] so concurrent triggers are ignored.
+ *
+ * Disable [MyAccessibilityService.DEBUG_FULL_SCAN_MODE] when switching back to the normal
+ * app-label search path.
+ */
+internal class DebugShadeScan(
+    private val delays: ShadeDelays,
+    private val getWindows: () -> List<AccessibilityWindowInfo>?,
+    private val getLastSnapshot: () -> List<NotificationShadeSnapshot.ShadeRow>,
+    private val openShade: () -> Unit,
+    private val closeShade: () -> Unit,
+) {
+    companion object {
+        private val TAG = FooLog.TAG(DebugShadeScan::class)
+        private const val EXPAND_SETTLE_MS = 1000L
+        private const val SCROLL_SETTLE_MS = 500L
+        private const val MAX_SCROLL_ATTEMPTS = 10
+        private const val MAX_EXPAND_PASSES_PER_SCROLL = 20
+    }
+
+    private val mainHandler = Handler(Looper.getMainLooper())
+    private var active = false
+
+    fun debugFullShadeScan() {
+        if (active) {
+            FooLog.d(TAG, "debugFullShadeScan: scan already in progress, ignoring")
+            return
+        }
+        active = true
+        FooLog.i(TAG, "debugFullShadeScan: starting — opening shade")
+        openShade()
+        mainHandler.postDelayed(
+            { debugScanPass(linkedSetOf(), MAX_SCROLL_ATTEMPTS, MAX_EXPAND_PASSES_PER_SCROLL) },
+            delays.shadeSettle,
+        )
+    }
+
+    /**
+     * Expands collapsed rows one at a time, each separated by [EXPAND_SETTLE_MS].
+     *
+     * Each call scans all visible rows via actionList:
+     *  - Already expanded (ACTION_COLLAPSE present): skip.
+     *  - Collapsed (ACTION_EXPAND present): expand row, schedule next pass after [EXPAND_SETTLE_MS],
+     *    then RETURN — only one expansion per delay interval.
+     *  - Neither action: skip.
+     *
+     * Once no collapsed row is found, proceeds to [debugScanCollect].
+     * Guarded by [expandPassesLeft] to prevent an unbounded loop.
+     */
+    private fun debugScanPass(
+        accumulated: LinkedHashSet<NotificationShadeSnapshot.ShadeRow>,
+        scrollAttemptsLeft: Int,
+        expandPassesLeft: Int,
+    ) {
+        val rows = NotificationShadeSnapshot.getLiveRowNodes(getWindows())
+        FooLog.v(TAG, "debugScanPass: ${rows.size} rows visible (expandPassesLeft=$expandPassesLeft)")
+        rows.forEachIndexed { _, rowNode ->
+            if (rowNode.actionList.any { it.id == AccessibilityNodeInfo.ACTION_COLLAPSE }) return@forEachIndexed
+            if (rowNode.actionList.any { it.id == AccessibilityNodeInfo.ACTION_EXPAND }) {
+                rowNode.performAction(AccessibilityNodeInfo.ACTION_EXPAND)
+                if (expandPassesLeft > 0) {
+                    mainHandler.postDelayed(
+                        { debugScanPass(accumulated, scrollAttemptsLeft, expandPassesLeft - 1) },
+                        EXPAND_SETTLE_MS,
+                    )
+                } else {
+                    FooLog.w(TAG, "debugScanPass: expand pass limit reached")
+                    debugScanCollect(accumulated, scrollAttemptsLeft)
+                }
+                return  // only one expansion per delay interval
+            }
+        }
+        debugScanCollect(accumulated, scrollAttemptsLeft)
+    }
+
+    /** Snapshot current rows, merge into [accumulated], then attempt a scroll. */
+    private fun debugScanCollect(accumulated: LinkedHashSet<NotificationShadeSnapshot.ShadeRow>, scrollAttemptsLeft: Int) {
+        val snapshot = getLastSnapshot()
+        val sb = StringBuilder("debugScanCollect: ${snapshot.size} rows in snapshot:")
+        snapshot.forEachIndexed { i, row -> sb.append("\n  [$i] $row") }
+        FooLog.i(TAG, sb.toString())
+        accumulated.addAll(snapshot.filter { !it.isEmpty })
+        val rowCountBeforeScroll = NotificationShadeSnapshot.getLiveRowNodes(getWindows()).size
+        debugScanScroll(accumulated, scrollAttemptsLeft, rowCountBeforeScroll)
+    }
+
+    /**
+     * Attempt [AccessibilityNodeInfo.ACTION_SCROLL_FORWARD] on the container. After a short
+     * settle, compare row count: if new rows appeared, continue scanning; otherwise finish.
+     */
+    private fun debugScanScroll(
+        accumulated: LinkedHashSet<NotificationShadeSnapshot.ShadeRow>,
+        scrollAttemptsLeft: Int,
+        rowCountBefore: Int,
+    ) {
+        if (scrollAttemptsLeft <= 0) {
+            FooLog.i(TAG, "debugScanScroll: scroll attempts exhausted")
+            debugScanFinish(accumulated)
+            return
+        }
+        val container = NotificationShadeSnapshot.getLiveContainerNode(getWindows())
+        if (container == null) {
+            FooLog.w(TAG, "debugScanScroll: container not found")
+            debugScanFinish(accumulated)
+            return
+        }
+        val scrolled = container.performAction(AccessibilityNodeInfo.ACTION_SCROLL_FORWARD)
+        FooLog.v(TAG, "debugScanScroll: ACTION_SCROLL_FORWARD=$scrolled ($scrollAttemptsLeft attempts left)")
+        if (!scrolled) {
+            FooLog.i(TAG, "debugScanScroll: scroll returned false — at bottom or scroll unsupported")
+            debugScanFinish(accumulated)
+            return
+        }
+        mainHandler.postDelayed({
+            val rowCountAfter = NotificationShadeSnapshot.getLiveRowNodes(getWindows()).size
+            if (rowCountAfter <= rowCountBefore) {
+                FooLog.i(TAG, "debugScanScroll: no new rows after scroll ($rowCountBefore → $rowCountAfter) — at bottom")
+                debugScanFinish(accumulated)
+            } else {
+                FooLog.v(TAG, "debugScanScroll: new rows after scroll ($rowCountBefore → $rowCountAfter) — continuing")
+                debugScanPass(accumulated, scrollAttemptsLeft - 1, MAX_EXPAND_PASSES_PER_SCROLL)
+            }
+        }, SCROLL_SETTLE_MS)
+    }
+
+    /** Log the deduplicated corpus then close the shade after the observation delay. */
+    private fun debugScanFinish(accumulated: LinkedHashSet<NotificationShadeSnapshot.ShadeRow>) {
+        val rows = accumulated.toList()
+        val sb = StringBuilder("debugScanFinish: complete corpus (${rows.size} rows):")
+        rows.forEachIndexed { i, row -> sb.append("\n  [$i] $row") }
+        FooLog.i(TAG, sb.toString())
+        mainHandler.postDelayed({
+            FooLog.i(TAG, "debugScanFinish: closing shade")
+            closeShade()
+            active = false
+        }, delays.preClose)
+    }
+}

mobile/src/main/java/llc/lookatwhataicando/notifai/MainActivity.kt

@@ -299,6 +299,23 @@ fun PermissionsGateScreen(
                 onClick = { FooNotification.startActivityNotificationListenerSettings(context) }
             ) else null
         )
+        Spacer(Modifier.height(12.dp))
+        // ── Required ACCESSIBILITY_SERVICE card ──────────────────────
+        val accessibilityMissing = Requirement.ACCESSIBILITY_SERVICE in snapshot.missing
+        RequirementCard(
+            icon        = Icons.Outlined.Lock,
+            title       = "Accessibility Service",
+            description = "Required for all notification processing. Without this, no " +
+                    "notifications are spoken — not live arrivals, nor launch catch-up. " +
+                    "Also enables reading content from apps like Google Chat that don't " +
+                    "expose it via standard APIs. " +
+                    "Enable `NotifAI` in the Accessibility settings that open.",
+            isMissing   = accessibilityMissing,
+            primaryAction = if (accessibilityMissing) PermissionAction(
+                label   = "Enable NotifAI Accessibility",
+                onClick = { FooPlatformUtils.showAccessibilitySettings(context) }
+            ) else null
+        )
         // ── Recommended BATTERY_OPTIMIZATION card ────────────────────
         if (forceShowAdvisories || Advisory.BATTERY_OPTIMIZATION in snapshot.advisories) {
             Spacer(Modifier.height(12.dp))