fix(payments): cancel abandoned 3DS challenges - #6402
brunomiguelpinto wants to merge 1 commit into
Conversation
|
Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green HOLD — the selector treats every Stripe
|
|
Automated pre-review (advisory, not a required check) — verdict: PASS · CI green PASS — The scan is narrowly limited to stale invoice payments awaiting an interactive Stripe action, and the service rechecks lifecycle and gated-activation state before cancelling. The specs cover selection exclusions, job delegation, provider no-op behavior, and unlocking only after cancellation moves the payment out of processing. Non-blocking: the ItemMetadata and Organization annotation-only changes are unrelated to this behavior. |
brunomiguelpinto
changed the title
|
|
||
| scope :abandoned_at_authentication, lambda { | ||
| where(payable_type: "Invoice", status: "requires_action", payable_payment_status: :processing) | ||
| .where(created_at: ..AUTHENTICATION_ABANDON_PERIOD.ago) |
There was a problem hiding this comment.
This uses the row's created_at as the start of the 3DS abandonment window, but payment rows can be reused across attempts while they are still in payable_payment_status = pending (Invoices::Payments::CreateService does a find_or_create_by! on that status before Stripe updates the row). If an old pending row is retried today and only then moves to requires_action, the hourly scan will treat it as already older than 24h and cancel a fresh authentication challenge almost immediately. The cutoff should be based on when the payment entered requires_action (for example the updated_at from that transition or a dedicated timestamp), or the retry path needs to create a fresh row for the challenge.
## Context When an off-session charge fails with `authentication_required`, the retry drops `off_session` so the provider can offer a 3D Secure challenge. The resulting PaymentIntent sits in `requires_action` until the end customer completes it. When they never do, nothing in Lago ever moves it: providers do not expire a PaymentIntent created through the API, and the only reset is a `payment_intent.canceled` webhook, which requires the merchant to cancel an intent that Lago created and owns from the provider's own dashboard. While that payment is stuck the invoice is unreachable through every path at once. `ready_for_payment_processing` stays false, so retries, dunning and payment requests all refuse it and the Pay invoice button is disabled. The unique partial index on pending and processing provider payments also reserves the invoice's only payment slot, so no replacement payment can be created even if the flag were reset by hand. Cancelling the stuck payment is the only way back, and the volume of invoices locked this way is significant. ## Description An hourly clock job treats an authentication challenge older than 24 hours as abandoned, cancels it at the provider and makes the invoice payable again. The window matches the expiry Lago already applies to its own payment intents, and the challenge expires at the issuer long before that, so nothing completable is cut short. It runs ahead of dunning so a freed invoice is collectable within the same hour. `requires_action` alone is not enough to identify an abandoned challenge. The same status covers payments that are waiting on money rather than on the customer: an incoming wire, ACH microdeposit verification, and offline vouchers all sit there legitimately for days. Nor is the stored next action reliable, since it is a snapshot written when the intent was created and never refreshed. So before cancelling anything the provider is asked what the intent is doing now, and only a card still awaiting an authentication next action is treated as abandoned. Everything else is left to arrive. The abandon window is measured from the payment's last update rather than its creation, because a payment row is reused across attempts while it is still pending. Its creation time can be months older than the challenge it currently carries, which would have a fresh challenge cancelled on the following tick. Cancelling reuses the existing provider-agnostic cancellation service, which already tolerates an intent that moved on without us. The invoice payment status is left alone: the provider's own cancellation webhook is what lands the definitive status, and writing it here as well would duplicate that and invent a status we have not confirmed. Payments that gate a subscription activation are skipped, as those already have their own cancellation on the subscription clock. A partial index keeps the hourly scan off a full table scan of payments.
e94324f to
d73d19a
Compare
brunomiguelpinto
changed the title
2 participants
Context
Reported by a customer whose end customer has invoices that cannot be paid through any Lago path. Details, affected accounts and measured volumes are in the Linear ticket.
When an off-session charge fails with
authentication_required, the retry dropsoff_sessionanderror_on_requires_actionso Stripe can offer a 3DS challenge (stripe/payments/create_service.rb:184). The PaymentIntent then sits inrequires_action, which is aPROCESSING_STATUS, so the invoice is saved withready_for_payment_processing = false.If the customer never completes the challenge, nothing in Lago ever moves it again:
PaymentIntents::ExpireServiceexpires the Lago record and the Checkout Session, never the PaymentIntent, andExpirePaymentIntentsJobonly runs on a provider settings change.payment_intent.canceled/payment_intent.payment_failedwebhook, which requires the merchant to go into the Stripe Dashboard and cancel an intent that Lago created and owns.The invoice is unreachable through every path at once
Invoices::Payments::RetryService:22payment_processor_is_currently_handling_paymentDunningCampaigns::ProcessCustomerService:74PaymentRequests::CreateService:87invoices_not_ready_for_payment_processingAnd it is worse than the flag alone suggests:
index_payments_on_payable_id_and_payable_typeis a unique partial index over(payable_id, payable_type)wherepayable_payment_status IN ('pending','processing'). The stuck payment holds the invoice's only payment slot, so no replacement payment can be created even if the flag were reset by hand. Cancelling the stuck payment is the only way back.Description
An hourly clock job treats an authentication challenge older than 24 hours as abandoned, cancels it at the provider, and makes the invoice payable again. The first pass also recovers the invoices that are already stuck.
It reuses the existing provider-agnostic
PaymentProviders::CancelPaymentService, whose only caller until now was the gated-subscription flow. After the cancellation, Stripe's ownpayment_intent.canceledwebhook lands the definitive status: the invoice becomesfailedwithready_for_payment_processing = true, which is what puts it back into dunning and retry. This PR only lifts the lock locally, so that a webhook which never arrives cannot leave the invoice stuck; it deliberately does not writefaileditself, which would duplicate the webhook and invent a status we have not confirmed.requires_actionis not only 3DS, and the stored next action is staleTwo traps, and the measured data says both are real.
The same status covers payments that are waiting on money rather than on the customer: a
customer_balancewire sits ondisplay_bank_transfer_instructionsfor days, ACH onverify_with_microdeposits, and Boleto/OXXO/Konbini on their own display details. The repo's own integration spec proves they are indistinguishable by status alone —simple_payment_integration_spec.rb:240shows acustomer_balancepayment atstatus = requires_action,payable_payment_status = processing,ready_for_payment_processing = false.And
provider_payment_datacannot settle it either: it is a snapshot written when the intent was created and never refreshed, so it can be months out of date. Measured against production,provider_payment_method_datais additionally empty on every stuck row, so there is no local way at all to tell a card challenge from a wire.So the provider is asked directly.
PaymentProviders::Stripe::Payments::CheckAbandonedAuthenticationServiceretrieves the intent with the payment method expanded, and a payment is only abandoned when the live intent is stillrequires_action, on acard, with an authentication next action. Cards are the only method that does 3DS, which also closes the redirect-based APMs (iDEAL, Bancontact, Link, crypto) that shareredirect_to_url. The stored next action stays only as a cheap candidate filter, so the hourly scan does not call the provider for every wire in flight.The abandon window is measured from the last update, not creation
A payment row is reused across attempts while it is still
pending—Invoices::Payments::CreateServicedoes afind_or_create_by!on that status. Itscreated_atcan therefore be months older than the challenge it currently carries, and selecting on it would cancel a challenge raised minutes ago on the very next tick. Enteringrequires_actionis the last write the row takes, soupdated_atis the time of that transition; any later write only pushes the deadline out, which is the safe direction. Covered by a regression spec. Thanks @TiagoLago for catching this.Why 24 hours
There is no provider-side timeout to copy — that absence is the bug. The two anchors are that the 3DS challenge itself expires at the issuer within minutes, and that hosted payment sessions converge on 24h (Stripe Checkout Sessions default and cap there). Lago already picked the same window for its own intents:
PaymentIntent#expires_atdefaults to24.hours.from_now. It is a single constant,Payment::AUTHENTICATION_ABANDON_PERIOD.The job runs at
*:00, ahead of dunning at*:45, so a freed invoice is collectable within the same hour.Payment-gated subscriptions are skipped
Those already cancel their own abandoned payment through
Subscriptions::ActivationRules::CancelService, on the subscription clock. Skipping them also keeps this job from triggering a gated activation resolution as a side effect.Changes
app/models/payment.rbAUTHENTICATION_ABANDON_PERIOD,AUTHENTICATION_NEXT_ACTIONS,abandoned_at_authenticationscopeapp/services/payment_providers/stripe/payments/check_abandoned_authentication_service.rbapp/services/invoices/payments/cancel_abandoned_service.rbapp/jobs/invoices/payments/cancel_abandoned_job.rbpayments/providersqueueapp/jobs/clock/cancel_abandoned_payments_job.rbunique :until_executedclock.rb*:00updated_at, addedCONCURRENTLY, so the hourly scan is not a seq scan overpaymentsRollout note
The first tick cancels every invoice that is already stuck. The measured backlog is large enough that this means a burst of provider calls, the same number of webhooks coming back, and then that many invoices entering dunning and retry at once across several organizations. It stays within Stripe's rate limits, but it means real charges and real dunning emails going out at once to end customers who have heard nothing for months. This needs a deliberate rollout — a first-run limit, or organization by organization — rather than a quiet deploy. Figures are in the Linear ticket.
Out of scope
CreatePaymentFactory, andPaymentRequests::Payments::CreateService#update_invoices_payment_statuspropagatesready_for_payment_processing = falseto every invoice in the request, so one abandoned challenge locks the request and all of its invoices. Measured volume is non-trivial; kept out to keep this reviewable and tracked separately.RetryServicecancelling and reopening online, which is what the customer explicitly asked for. It changes visible UI/API behaviour and deserves its own PR.Testing
98 examples, 0 failures. RuboCop clean.
spec/models/payment_spec.rb— scope boundaries: age, payable type, payment status,next_actiontype, and a reused row carrying a fresh challengespec/services/payment_providers/stripe/payments/check_abandoned_authentication_service_spec.rb— card challenge, SDK challenge, wire, non-card redirect, completed since, no payment method, unreadable intentspec/services/invoices/payments/cancel_abandoned_service_spec.rb— cancel and unlock, payment status left untouched, bank transfer and ACH microdeposits left alone, provider refusing the cancellation, already paid, voided, gated activation, non-invoice payablespec/jobs/clock/cancel_abandoned_payments_job_spec.rb,spec/jobs/invoices/payments/cancel_abandoned_job_spec.rbFixes BIL-670.