Skip to content

fix: re-sweep and durably invalidate DashPay backfill coverage when contact accounts register after the scan #1546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HashEngineering merged 2 commits into from
Aug 27, 2026
Merged

fix: re-sweep and durably invalidate DashPay backfill coverage when contact accounts register after the scan #1546

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7d52352
fix: re-sweep and invalidate backfill coverage when contact accounts …
bfoss765 Aug 24, 2026
ca2d347
fix: serialize the DashPay registration signal against every coverage…
HashEngineering Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
296 changes: 228 additions & 68 deletions wallet/src/de/schildbach/wallet/service/platform/sdk/DashPayBackfillGate.kt
View file
Open in desktop

Large diffs are not rendered by default.

46 changes: 46 additions & 0 deletions wallet/src/de/schildbach/wallet/service/platform/sdk/DashSdkService.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,35 @@ data class DashPayContactDrainReport(
val built: Int get() = (queuedBefore - queuedAfter).coerceAtLeast(0)
}

/**
* How completely the bound SDK wallet's DIP-15 receival accounts cover its
* established (RECEIVED-request) DashPay contacts — the drain-time
* diagnostic that names the DARK contacts: an established contact with NO
* `dashpayReceivingFunds` account has its receiving addresses in no watched
* script set, so its payments can never match a filter. Under the SDK's
* known re-enqueue asymmetry such a contact can stay dark PERMANENTLY (the
* sweep re-enqueues builds per contact, but a build that keeps failing is
* indistinguishable from one that merely has not run yet), and this report
* is what lets a field log say exactly which contact is stuck.
*
* Pure Room reads, diagnostics only — never load-bearing for the gate.
*/
data class DashPayReceivalCoverage(
/** Distinct contact identities with a RECEIVED contact request (the receival-account universe). */
val establishedContacts: Int,
/** Registered `dashpayReceivingFunds` accounts on the bound wallet. */
val receivalAccounts: Int,
/** Established contacts with NO receival account — the permanently-dark candidates. */
val darkContacts: Int,
/** Up to [DARK_CONTACT_SAMPLE_LIMIT] dark contact identity ids (base58, untruncated). */
val darkContactIdSample: List<String>
) {
companion object {
/** Sample size for [darkContactIdSample] — enough to name the stuck contacts, bounded for one log line. */
const val DARK_CONTACT_SAMPLE_LIMIT = 5
}
}

/**
* The two read-only signals the app-side DIP-15 backfill gate
* ([DashPayBackfillGate]) needs to tell "the coreHeight backfill is still
Expand Down Expand Up @@ -633,6 +662,23 @@ interface DashSdkService {
*/
suspend fun dashPayPendingAccountBuilds(walletIdHex: String): Int?

/**
* Read-only receival-account coverage of the wallet's established
* DashPay contacts — see [DashPayReceivalCoverage] for what it names and
* why. Pure Room reads through [databaseOrNull] (no [ensureStarted], no
* native call, no sweep — the same posture as
* [readDashPayBackfillSignals]); null when the SDK is down, the wallet id
* is malformed, or the read failed. Never throws. Default null so
* read-only fakes stay source-compatible.
*
* @param walletIdHex the bound wallet id ([bindAppWallet]'s return).
* @param ownerIdentityId our 32-byte platform identity id.
*/
suspend fun readDashPayReceivalCoverage(
walletIdHex: String,
ownerIdentityId: ByteArray
): DashPayReceivalCoverage? = null

/**
* Read-only snapshot of the two signals the app-side DIP-15 backfill
* gate reasons over — see [DashPayBackfillSignals] for what each means
Expand Down
70 changes: 70 additions & 0 deletions wallet/src/de/schildbach/wallet/service/platform/sdk/DashSdkServiceImpl.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,40 @@ internal suspend fun healIdentityKeys(
)
}

/**
* Fold the SDK's persisted DashPay rows into a [DashPayReceivalCoverage] —
* pure (no Room, no native, no I/O) so the dark-contact set logic is
* host-testable on the plain JVM.
*
* [receivedContactIds] are the contact identity ids of the owner's RECEIVED
* contact requests (the receival-account universe; duplicates collapse), and
* [receivalAccountFriendIds] the `friendIdentityId`s of the wallet's
* registered `dashpayReceivingFunds` accounts. A contact with no matching
* account is DARK: its receiving addresses are in no watched script set, so
* its payments can never match a filter. Ids compare byte-wise; the sample
* is base58 (the same encoding the app's userId and the SDK's identity row
* keys use), in the order the contact rows were supplied — deterministic, so
* consecutive drains name the same stuck contacts.
*/
internal fun computeDashPayReceivalCoverage(
receivedContactIds: List<ByteArray>,
receivalAccountFriendIds: List<ByteArray>,
sampleLimit: Int = DashPayReceivalCoverage.DARK_CONTACT_SAMPLE_LIMIT
): DashPayReceivalCoverage {
val covered = receivalAccountFriendIds.mapTo(HashSet()) { it.toList() }
val contacts = LinkedHashSet<List<Byte>>()
receivedContactIds.forEach { contacts.add(it.toList()) }
val dark = contacts.filter { it !in covered }
return DashPayReceivalCoverage(
establishedContacts = contacts.size,
receivalAccounts = receivalAccountFriendIds.size,
darkContacts = dark.size,
darkContactIdSample = dark.take(sampleLimit).map {
org.bitcoinj.core.Base58.encode(it.toByteArray())
}
)
}

/**
* Default [DashSdkService] implementation — the Phase 3 bootstrap scaffold
* (`docs/kotlin-sdk-migration-plan.md`).
Expand Down Expand Up @@ -997,6 +1031,42 @@ class DashSdkServiceImpl @Inject constructor(
}
}

/**
* See [DashSdkService.readDashPayReceivalCoverage]. Pure Room reads via
* [databaseOrNull] — the same posture as [readDashPayBackfillSignals]:
* no [ensureStarted], no native call, no sweep. Established contacts =
* distinct RECEIVED (`isOutgoing == false`) contact-request senders;
* receival accounts = the wallet's `dashpayReceivingFunds` rows
* ([ACCOUNT_TYPE_TAG_DASHPAY_RECEIVING_FUNDS]), matched by their
* `friendIdentityId`. Never throws — null means "unavailable".
*/
override suspend fun readDashPayReceivalCoverage(
walletIdHex: String,
ownerIdentityId: ByteArray
): DashPayReceivalCoverage? {
return try {
val database = databaseOrNull() ?: return null
val walletId = walletIdFromHex(walletIdHex) ?: return null
val receivedContactIds = database.dashpayDao()
.getContactRequestsByOwner(ownerIdentityId)
.filterNot { it.isOutgoing }
.map { it.contactIdentityId }
val receivalFriendIds = database.accountDao()
.observeByWallet(walletId).first()
.filter { it.accountType == ACCOUNT_TYPE_TAG_DASHPAY_RECEIVING_FUNDS }
.mapNotNull { it.friendIdentityId }
computeDashPayReceivalCoverage(receivedContactIds, receivalFriendIds)
} catch (e: kotlinx.coroutines.CancellationException) {
throw e
} catch (e: Exception) {
log.debug(
"failed to read DashPay receival coverage for {}…: {}",
walletIdHex.take(8), e.message
)
null
}
}

/**
* One-shot bring-up; caller holds [lock]. On any failure every
* partially-created resource is torn down and the exception rethrown,
Expand Down
188 changes: 156 additions & 32 deletions wallet/src/de/schildbach/wallet/service/platform/sdk/SdkWalletBinder.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -594,14 +594,69 @@ class SdkWalletBinder internal constructor(
// short — and before this, the queue was only ever
// drained on a pass the gate allowed through, i.e. on
// almost no launch after the first.
drainDeferredAccountBuilds(walletId)
val registeredNew = drainDeferredAccountBuilds(walletId, identityId)
// Post-drain follow-up sweep, at most ONE per cycle:
// accounts this drain just REGISTERED were not there
// when any sweep's rescan reconcile ran, so the SDK's
// per-contact rescan guard has never fired for them.
// The gate's registration signal is set (and its
// coverage durably invalidated) by the note inside
// the drain — consult it again and give the reconcile
// its second look NOW, in this same cycle, instead of
// leaving the money invisible until a relaunch.
if (registeredNew) {
val followUp = backfillGate.evaluate(walletId, identityId, userId)
if (followUp.shouldRun) {
if (followUp.armedToWrite != null) armedRewind = true
log.info(
"DashPay follow-up sweep on {}…: the drain registered " +
"receival account(s) no sweep's rescan reconcile has " +
"seen — sweeping again in this cycle",
walletId.take(8)
)
runProvisioningSweep(walletId, identityId)
} else {
// e.g. a backfill replay in flight, which a
// registration must never interrupt. The debt
// is recorded (durably invalidated coverage +
// the in-memory signal) for a later pass.
log.info(
"DashPay follow-up sweep withheld on {}…: {} — the " +
"registration debt stays recorded for a later pass",
walletId.take(8), followUp.reason
)
}
}
if (armedRewind) {
watchArmedBackfillRewind(walletId, identityId, userId)
}
return
}
armedRewind = decision.armedToWrite != null
}

val report = sdkService.provisionDashPayContactAccounts(walletId)
identityId?.let { backfillGate.recordPassOutcome(walletId, it, report) }
val registrationOutstanding = runProvisioningSweep(walletId, identityId)
// Post-drain follow-up sweep (run path), at most ONE per
// cycle: this pass's own step-2 drain registered receival
// accounts AFTER its sweep reconciled the rewind — the exact
// ordering defect behind the restored-wallet dark contacts
// (sweep at 17:20:08 against zero accounts, 29 registered
// 17:20:11–31, no rewind until a manual restart). The gate's
// registration branches turn the signal recordPassOutcome
// re-raised into a permitted, re-armed pass.
if (registrationOutstanding && identityId != null) {
val followUp = backfillGate.evaluate(walletId, identityId, userId)
if (followUp.shouldRun) {
if (followUp.armedToWrite != null) armedRewind = true
log.info(
"DashPay follow-up sweep on {}…: this pass's drain registered " +
"receival account(s) after its sweep reconciled — sweeping " +
"again in this cycle",
walletId.take(8)
)
runProvisioningSweep(walletId, identityId)
}
}
// The pass we just armed rewinds the SPV synced height, but the
// drop only becomes DURABLE ~9-60 s later, and it stays visible
// only until the scan climbs back out of it. recordPassOutcome
Expand All @@ -613,30 +668,6 @@ class SdkWalletBinder internal constructor(
if (armedRewind && identityId != null) {
watchArmedBackfillRewind(walletId, identityId, userId)
}
// The sweep is a long native op: its SDK lines sat in the
// logcat buffer until the bridge's next 30 s / 5 min poll and
// were routinely rolled over before then. Pull them into
// wallet.log NOW, while they are still there — cheap,
// bounded, never throws, and we are on a background
// coroutine, not the main thread.
NativeLogBridge.drainNow()
when {
!report.bound -> log.debug(
"DashPay friend-chain provisioning: SDK wallet {}… not loaded yet",
walletId.take(8)
)
report.pendingBefore > 0 || report.drainScheduled -> log.info(
"DashPay friend-chain provisioning on {}…: sweep ok={}/err={}, " +
"{} account build(s) queued, drainScheduled={}",
walletId.take(8), report.syncSuccess, report.syncErrors,
report.pendingBefore, report.drainScheduled
)
else -> log.debug(
"DashPay friend-chain provisioning on {}…: steady " +
"(sweep ok={}/err={}, nothing queued)",
walletId.take(8), report.syncSuccess, report.syncErrors
)
}
} finally {
provisioning.set(false)
}
Expand All @@ -651,6 +682,87 @@ class SdkWalletBinder internal constructor(
}
}

/**
* One provisioning pass — the SDK sweep + drain
* ([DashSdkService.provisionDashPayContactAccounts]) with its gate
* accounting, wallet.log lines and post-drain receival-coverage
* diagnostics. Throws freely; the caller
* ([provisionContactAccountsIfEnabled]) contains the fallout.
*
* @return whether the pass's own drain left a registration OUTSTANDING —
* the gate's signal, consumed and then re-raised from the pass's built
* count by [DashPayBackfillGate.recordPassOutcome] — i.e. whether a
* follow-up sweep is owed. Always false without an identity, and with
* a gate that records nothing ([DashPayBackfillGate.ALWAYS_RUN]).
*/
private suspend fun runProvisioningSweep(walletId: String, identityId: ByteArray?): Boolean {
val report = sdkService.provisionDashPayContactAccounts(walletId)
identityId?.let { backfillGate.recordPassOutcome(walletId, it, report) }
// The sweep is a long native op: its SDK lines sat in the
// logcat buffer until the bridge's next 30 s / 5 min poll and
// were routinely rolled over before then. Pull them into
// wallet.log NOW, while they are still there — cheap,
// bounded, never throws, and we are on a background
// coroutine, not the main thread.
NativeLogBridge.drainNow()
when {
!report.bound -> log.debug(
"DashPay friend-chain provisioning: SDK wallet {}… not loaded yet",
walletId.take(8)
)
report.pendingBefore > 0 || report.drainScheduled -> log.info(
"DashPay friend-chain provisioning on {}…: sweep ok={}/err={}, " +
"{} account build(s) queued, drainScheduled={}",
walletId.take(8), report.syncSuccess, report.syncErrors,
report.pendingBefore, report.drainScheduled
)
else -> log.debug(
"DashPay friend-chain provisioning on {}…: steady " +
"(sweep ok={}/err={}, nothing queued)",
walletId.take(8), report.syncSuccess, report.syncErrors
)
}
if (identityId != null && report.bound) {
logReceivalCoverageDiagnostics(walletId, identityId)
}
return identityId != null && backfillGate.readBackfillStatus().registrationOutstanding
}

/**
* The dark-contact diagnostic, one line per drain: how many established
* contacts have NO receival account. Such a contact's receiving addresses
* are in no watched script set, and under the SDK's re-enqueue asymmetry
* a build that keeps failing is re-enqueued forever without ever
* registering — so a delta that persists across drains is the fingerprint
* of a PERMANENTLY dark contact, and the sampled ids let a field log name
* exactly which one is stuck. Never throws; an unavailable read logs
* nothing (the drain line above it already proves the pass ran).
*/
private suspend fun logReceivalCoverageDiagnostics(walletId: String, identityId: ByteArray) {
try {
val coverage = sdkService.readDashPayReceivalCoverage(walletId, identityId) ?: return
val sample = if (coverage.darkContactIdSample.isEmpty()) {
""
} else {
coverage.darkContactIdSample.joinToString(
prefix = " [", separator = ", ",
postfix = if (coverage.darkContacts > coverage.darkContactIdSample.size) ", …]" else "]"
) { "${it.take(8)}…" }
}
log.info(
"DashPay receival-account coverage on {}…: establishedContacts={}, " +
"receivalAccounts={}, dark={}{} — a dark contact's receiving addresses " +
"are in no watched script set (permanently-dark candidate under the SDK's " +
"re-enqueue asymmetry)",
walletId.take(8), coverage.establishedContacts, coverage.receivalAccounts,
coverage.darkContacts, sample
)
} catch (t: Throwable) {
if (t is CancellationException) throw t
log.debug("receival-coverage diagnostics unavailable: {}", t.message)
}
}

/**
* Drain the SDK's deferred DashPay account-build queue and say what
* happened — queued / built / still queued. The counts are the only view
Expand All @@ -667,26 +779,38 @@ class SdkWalletBinder internal constructor(
*
* Never throws: an unavailable drain (locked device, seed verify) is a
* normal state and the queue survives for the next pass.
*
* @return whether the drain's registrations were accepted as NEW by the
* gate ([DashPayBackfillGate.noteAccountBuildsRegistered] — which has
* also durably invalidated any recorded coverage by the time it
* answers), i.e. whether a follow-up sweep is owed in THIS cycle.
* False for an empty/muted/failed drain.
*/
private suspend fun drainDeferredAccountBuilds(walletId: String) {
try {
private suspend fun drainDeferredAccountBuilds(walletId: String, identityId: ByteArray): Boolean {
return try {
val report = sdkService.drainDashPayContactAccountBuilds(walletId)
log.info(
"DashPay account-build drain on {}…: {}",
walletId.take(8), describeContactDrain(report)
)
// Accounts that only exist NOW were not there when the last sweep
// reconciled the DIP-15 rewind, so a sweep is owed for them; the
// gate turns this into a re-provision on its next consultation
// instead of leaving the money invisible until a relaunch.
backfillGate.noteAccountBuildsRegistered(report.built)
// gate records that debt durably (coverage invalidation) and its
// verdict lets THIS cycle pay it with a follow-up sweep instead
// of leaving the money invisible until a relaunch.
val registeredNew = backfillGate.noteAccountBuildsRegistered(report.built)
if (report.bound) {
logReceivalCoverageDiagnostics(walletId, identityId)
}
registeredNew
} catch (t: Throwable) {
if (t is CancellationException) throw t
log.warn(
"DashPay account-build drain failed; the contacts' receiving addresses stay " +
"unwatched until the next pass",
t
)
false
}
}

Expand Down
Loading
Loading