feat(payments): add list filters to payments (REST + GraphQL) - #6325
Draft
sarkissianraffi wants to merge 9 commits into
Draft
sarkissianraffi wants to merge 9 commits into
sarkissianraffi wants to merge 9 commits into
Conversation
added 3 commits
September 7, 2026 14:57
## Context Payments list filters need a shared development dataset for REST and browser QA. ## Description Seed an isolated organization with payment variants, boundary amounts, dates, and an expected-record manifest. Store disposable credentials only in ignored local files.
## Context Finance needs the same payment filters through both list APIs, including exact invoice and receipt matches and amounts beyond JavaScript integer precision. ## Description Add composable validated payment filters and expose currency and search on REST. Preserve payment responses, visibility, ordering, and pagination. Generate the GraphQL schemas and add replayable HTTP QA and a 100,000-payment benchmark.
## Context Payment filters need reproducible evidence across the API, UI and clients. ## Description Record HTTP assertions, cross-client comparisons and query plans on synthetic development data. Document replay commands and existing compatibility constraints.
This was referenced Sep 7, 2026
added 6 commits
September 8, 2026 20:55
- receipt_number and invoice_number resolve the receipt / invoice ids first through organization-scoped lookups, then reach payments by primary key or (payable_type, payable_id); no function on the joined column, no DISTINCT - payment_provider_type resolves provider ids in Ruby, scoped to the organization - payment_method_type evaluates the jsonb type and the saved-method fallback as two plain predicates instead of a COALESCE across a LEFT JOIN, and no longer matches soft-deleted payment methods - external_customer_id resolves the customer id first instead of joining customers - spec fixtures use customers and providers of the payment's organization - SQL tripwire spec: no DISTINCT, no function on indexed payments columns, no joins on the lookup tables
Removed for performance reasons: the count of a rare method type walks the whole organization and cannot be served by an index while the saved-method fallback branch exists. The way back is a denormalised payments column with its own partial index. Removed from the query, contract, REST whitelist, GraphQL resolver and enum, regenerated schema, specs and the QA script. The superseded 100k-row benchmark script and its EXPLAIN dump are replaced by script/perf/payments_filters/.
script/perf/payments_filters/: synthetic dataset generator (one 5M-payment organization plus 50 smaller ones, illustrative skews), case matrix, plan capture through PaymentsQuery (list and COUNT(*), EXPLAIN ANALYZE BUFFERS, median of 3), load test (HTTP and DB-only), before/after scoreboard, and the read-only replica queries for the production-scale inputs. Plans captured on the synthetic dataset before and after the rewrites and indexes are committed with identifiers redacted; the internal performance document quotes them.
Plans, DB-only and HTTP load-test results captured on the synthetic dataset after the rewrites and indexes, plus the before/after comparison.
Counting every matching payment of a large organization is the slow half of the payments query. Reuse BaseQuery::CappedTotalCount and expose PaymentCollectionMetadata with totalCountCapped and hasNextPage, exactly as the invoices collection does, so the client shows "10,000+" and paginates on hasNextPage.
lpad truncates longer strings, so customers 1000+ shared the slugs (and receipt numbers) of customers 1-999. Pad like the receipt trigger does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Payments can now be filtered consistently through REST, customer-scoped REST and GraphQL. Exact invoice filters cover both payable paths without duplicate rows or incorrect pagination counts; REST also forwards currency and search.
Roadmap Task
User-provided
/paymentsfiltering specification; no tracking ticket supplied.Context
The payments page needs the same filter capabilities as the public API while retaining existing payment visibility, ordering and response objects.
Description
payment_status/payment_statusespaymentStatusamount_from,amount_toamountFrom,amountToreceipt_numberreceiptNumbercreated_at_from,created_at_tocreatedAtFrom,createdAtTopayment_provider_typepaymentProviderTypecurrencycurrencyinvoice_numberinvoiceNumberexternal_customer_idexternalCustomerIdpayment_typepaymentTypepayable_typepayableTypesearch_termsearchTermAll filters AND together; array entries OR together. Existing page, per_page, invoice_id and GraphQL pagination/search arguments remain supported.
Adds chainable query predicates, contract validation, shared REST concern wiring, GraphQL arguments and regenerated schema.graphql/schema.json. Provider method vocabulary lives on PaymentMethod and the Stripe customer constant remains an alias. No distinct or serializer changes.
Verification
API and client QA, 75 HTTP assertions, cross-client IDs, UI QA and recording
Reproduce on a running development API (synthetic fixtures, no gateway calls):
Seed script: 30 payments / 29 visible, three customers, stub Stripe and GoCardless, all statuses/types/methods, receipts, two-invoice PaymentRequest, two currencies, dates at organization-timezone boundaries, zero and amounts through 9223372036854775807. Credentials stay in ignored mode-0600 files.
Performance
All shipped filters meet the internal latency targets on a synthetic dataset of ~5M payments for one organization (plus 50 smaller organizations), PostgreSQL 15, 20 concurrent clients. Plans, dataset generator and load test are reproducible from
script/perf/payments_filters/(see its README). Performance analysis: internal document "Payments list filters: performance analysis" (Raffi; not public).Query rewrites in this PR (from the plans):
receipt_numberandinvoice_numberresolve the receipt / invoice ids first through organization-scoped sub-selects, then reach payments by primary key /(payable_type, payable_id).payment_provider_typeresolves provider ids in Ruby, scoped to the organization.external_customer_idresolves the customer id first.Rollout order: #6341 (indexes,
CREATE INDEX CONCURRENTLY, migration-only) first, then this PR. Confirmpg_stat_user_indexes.idx_scanis non-zero on the new indexes once the filters ship.Not shipped for performance reasons
payment_method_type(RESTpayment_method_type[], GraphQLpaymentMethodType): counting a rare method type has to walk the whole organization while the saved-method fallback exists, and no index changes that. Removed in one commit (feat(payments): drop the payment_method_type list filter), so it can come back once the method type is a real, indexed column. Same removal on lago-front and lago-openapi; the SDK clients and CLI must be regenerated from the updated spec.Known, pre-existing
REST
meta.total_countruns an unboundedCOUNT(*)with the per-row invoice visibility check on every page; on very large organizations that count, not the filters, dominates response time. GraphQL now uses the capped count invoices already use (PaymentCollectionMetadatawithtotalCountCapped/hasNextPage, the UI shows "10,000+"). The REST count strategy is a follow-up.Compatibility notes
Invoice::VISIBLE_STATUSincludes draft. Preserve the existing base scope and visibility condition; the draft fixture remains visible and an open-invoice fixture stays hidden.ISO8601Daterejects malformed dates before resolution, preserving scalar behavior.PaymentMethodTypeEnummeans manual/provider. The new provider-method enum isPaymentProviderMethodTypeEnum; payment type keeps its existing enum.validation_errors(plural).meta.next_pageis a page number, not a URL; clients repeat filters without changing response metadata.Not in scope: serializer/payment response changes or CSV/data export.
Related PRs
Merge order: #6341 (indexes) deployed first, then API, then front; OpenAPI before client/CLI releases. JavaScript CI uses the pinned feature spec during that rollout.