Add ALL and EXACTLY_ONE array filter operators for MongoDB and Postgres by lrathod · Pull Request #323 · hypertrace/document-store

lrathod · 2026-08-27T11:16:26Z

Summary

Adds two new ArrayOperator values for filtering on array-valued attributes:

ALL — array attribute must contain every value specified in the filter. Set-containment semantics: order and duplicates are irrelevant on both sides ([red, red] ALL [red] is true).
EXACTLY_ONE — array attribute must contain exactly one element, and that element must be one of the specified values. The cardinality check counts raw elements, not distinct values ([red, red] EXACTLY_ONE [red] is false).

Both work on top-level and nested array fields (e.g. props.colors, scope.environmentScope.environmentIds), on MongoDB and Postgres.

MongoDB

ALL → {"$expr": {"$setIsSubset": [<values>, <guarded array>]}}
EXACTLY_ONE → $and of $size == 1 and $in on $arrayElemAt [path, 0]
Non-array guard: <guarded array> is {"$cond": [{"$isArray": "$path"}, "$path", []]} — documents holding a missing/null/non-array scalar value simply do not match instead of erroring ($setIsSubset/$size reject non-array operands), matching the Postgres behaviour.

Postgres

Native array columns (flat collections): ALL → col @> ?; EXACTLY_ONE → array_length(col, 1) = 1 AND col && ?. No COALESCE — NULL arrays are excluded by WHERE semantics anyway, and the unwrapped column reference keeps the filter GIN-indexable (SARGable).
JSONB array paths (nested documents): ALL → (CASE WHEN jsonb_typeof(path) = 'array' THEN path ELSE '[]'::jsonb END) @> ?::jsonb; EXACTLY_ONE → jsonb_array_length(<guarded>) = 1 AND <guarded> <@ ?::jsonb (single bound param — with exactly one element, membership ≡ containment). The runtime jsonb_typeof guard is retained only for schemaless JSONB paths.
Compile-time element types: the array element type is resolved from the field expression's DataType (ArrayIdentifierExpression#getElementDataType()), falling back to inference from the filter values only when the field carries no type info.

Design note

Both parsers require the inner RelationalExpression to carry a constant value list; a non-constant RHS throws UnsupportedOperationException. This is intentional — ALL/EXACTLY_ONE are set-level operators, unlike ANY which supports arbitrary per-element sub-filters. Empty value lists are rejected at construction by ConstantExpression.

Future consideration (noted in the enum javadoc): an EXACTLY operator for set equality — array contains exactly the filter values, no more and no less.

Test coverage / EXACTLY_ONE semantics

MATCH_EXACTLY_ONE describes the stored array cardinality, not the length of the RHS:

Stored array must have exactly 1 element
That element must be in the RHS list (RHS can be 1 or many candidates)

Example with column tags:

Entity A: ["A", "B"]
Entity B: ["A"]
Entity C: ["B"]
Entity D: ["A", "B", "C"]

Query	Result
MATCH_ALL `["A", "B"]`	A and D (D matches because extras are allowed; ALL is subset / containment)
MATCH_EXACTLY_ONE `["A"]`	B only
MATCH_EXACTLY_ONE `["A", "B"]`	B and C (not A, not D — size ≠ 1)

Same translation on both stores: Mongo $size=1 + $in; Postgres native array_length=1 AND && / JSONB length + <@.

Test plan

MongoArrayFilterParserTest — operator structure, $isArray guards, nested paths, non-constant RHS rejection, no double $expr wrapping
PostgresQueryParserTest — ALL/EXACTLY_ONE × JSONB/native array, nested JSONB paths, compile-time type precedence over value inference, UNSPECIFIED fallback, non-constant RHS rejection
DocStoreQueryV1Test (nested ArrayMatchAllOneOperatorTest) — integration tests on both datastores: nested JSONB array fields, native array columns (typed + untyped via PostgresArrayTypeProvider), JSONB array column on flat collections, 3-level nested paths with missing intermediate objects, non-array scalar values, duplicates ([red, red]), and order-independence — run in CI
:document-store:build (compile + unit tests + spotless) passes locally

Made with Cursor

Add two new ArrayOperator values for filtering on array-valued attributes: - ALL: array attribute must contain every value specified in the filter - ONE: array attribute must contain exactly one element, and that element must be one of the specified values MongoDB: ALL uses $setIsSubset with an $ifNull guard; ONE combines $size == 1 with $in on the first element via $arrayElemAt. Postgres: native array columns use @> (ALL) and array_length + && (ONE); JSONB array paths use jsonb_typeof-guarded @> containment and jsonb_array_length respectively. Both parsers require the inner filter to carry a constant value list; non-constant RHS expressions throw UnsupportedOperationException since these are set-level operators, not per-element predicates like ANY. Co-authored-by: Cursor <cursoragent@cursor.com>

suddendust · 2026-08-31T11:45:32Z

+    return value instanceof List ? (List<?>) value : List.of(value);
+  }
+
+  private PostgresDataType resolvePostgresDataType(final List<?> values) {


Can we avoid a runtime check? ArrayIdentifierExpression contains DataType that can be extracted statically.

Done. The native-array branch now resolves the element type from the compile-time type info on the field expression (ArrayIdentifierExpression#getElementDataType() / IdentifierExpression#getDataType()) via PostgresDataType.fromDataType, and only falls back to inferring from the filter values when the field carries no type info (UNSPECIFIED). The runtime jsonb_typeof guard is now retained only for the JSONB/nested-array path, where the value is schemaless and can be JSON null or a non-array at runtime. Added unit tests covering both the compile-time precedence (declared long[] wins over Integer values) and the fallback.

sure, let me update,
for nested arrays, we'd still need runtime check

suddendust · 2026-08-31T11:49:23Z

What about operators for nested json arrays?

suddendust · 2026-08-31T11:53:07Z

Lets add some integration tests? You can add in DocStoreQueryV1Test.

suddendust · 2026-08-31T11:53:53Z

+   */
+  @ParameterizedTest
+  @ArgumentsSource(AllProvider.class)
+  void getDocumentsContainingAllGivenValues(final String dataStoreName)


Oh I see we already have ITs. Can we move them to DocStoreQueryV1Test?

sure, let me move

Done. Moved all ALL/ONE integration tests into DocStoreQueryV1Test as a nested ArrayMatchAllOneOperatorTest class and removed them from ArrayFiltersQueryIntegrationTest. They now run against the shared collections: nested JSONB path (props.colors) on both stores, native array columns (tags/flags) on the flat collection using the existing PostgresArrayTypeProvider (typed + untyped variants), and the JSONB array column on the flat collection.

…mantics - Resolve native array element type from the compile-time type info on the field expression (ArrayIdentifierExpression/IdentifierExpression DataType) instead of inferring from filter values; value inference is now only a fallback when no type info is present. The runtime jsonb_typeof guard is retained only for schemaless JSONB/nested array paths. - Add unit + integration tests covering ALL/ONE on nested array fields (e.g. props.colors, scope.environmentScope.environmentIds) for both MongoDB and Postgres. - Add integration test documenting that ALL is set-containment: duplicates in the document array ([red, red] ALL [red]) still match in both backends. Co-authored-by: Cursor <cursoragent@cursor.com>

Consolidate the ALL/ONE integration tests into DocStoreQueryV1Test as a nested ArrayMatchAllOneOperatorTest class, per review feedback: - Nested JSONB array path (props.colors) covered on both MongoDB and Postgres via the shared document collection - Native array columns (tags TEXT[], flags BOOLEAN[]) covered on the flat collection with both typed (compile-time DataType) and untyped (value-inference fallback) ArrayIdentifierExpression variants - JSONB array column (props.colors) covered on the flat collection - Duplicate-containing arrays ([red, red] ALL [red] -> true) covered via a dedicated collection, documenting set-containment semantics on real DBs Co-authored-by: Cursor <cursoragent@cursor.com>

lrathod · 2026-09-01T05:56:55Z

Addressed the review feedback in the latest commits:

Nested JSON arrays — ALL/ONE already go through the same array-source resolution infra as ANY (PostgresFieldIdentifierExpressionVisitor / MongoDollarPrefixingIdempotentParser), so nested paths like props.colors and scope.environmentScope.environmentIds work. Added coverage to prove it: unit tests in PostgresQueryParserTest (nested ALL + ONE) and MongoArrayFilterParserTest (nested ALL + ONE), plus integration tests in DocStoreQueryV1Test running nested-path ALL/ONE against both MongoDB and Postgres.

Integration tests — moved to DocStoreQueryV1Test (nested ArrayMatchAllOneOperatorTest), covering: nested JSONB array field (both stores), native array columns on the flat collection with typed and untyped ArrayIdentifierExpression (via PostgresArrayTypeProvider), and the JSONB array column on the flat collection.

Duplicates question — [red, red] ALL [red] returns true in both backends. `` treats both operands as sets (duplicates collapsed), and Postgres @> is element-wise containment (each RHS element must exist in LHS; duplicates on either side are irrelevant). So ALL follows set semantics — order and duplicates don't matter. Added an integration test (`testAllWithDuplicatesInDocumentArray`) that asserts a document with `tags: [red, red]` matches `ALL [red]` on both datastores. (For completeness: ONE is unaffected by this — `[red, red]` has length 2, so it never matches ONE.)

Negative coverage: - ALL/ONE reject a non-constant RHS with UnsupportedOperationException in both Mongo and Postgres parsers - Empty value lists are rejected at construction by ConstantExpression - Integration: non-array JSONB values do not match and do not error on Postgres, exercising the jsonb_typeof guard Semantics documentation via integration tests on both datastores: - ALL is order-independent: [red, blue] ALL [blue, red] matches - ALL/ONE on a three-level nested array field (props.metadata.colors), including docs with missing intermediate objects Co-authored-by: Cursor <cursoragent@cursor.com>

…ONE semantics - Native Postgres arrays: drop COALESCE - NULL arrays are excluded by WHERE semantics anyway, and the unwrapped column reference keeps the filter GIN-indexable (SARGable) - JSONB ONE: replace the per-value OR chain with a single <@ containment against the full filter list (with exactly one element, membership and containment are equivalent) - one bound param instead of N - Mongo: guard ALL/ONE with $cond/$isArray so documents holding a non-array scalar no longer error out ($setIsSubset/$size reject non-array operands), matching the Postgres jsonb_typeof behavior; subsumes $ifNull - Document that ONE counts raw elements, not distinct values ([red, red] ONE [red] is false), with an integration test on both stores; non-array scalar test now runs on Mongo too Co-authored-by: Cursor <cursoragent@cursor.com>

Aligns with the service-level MATCH_EXACTLY_ONE name and reads unambiguously ("exactly one element, in the given set"). Also notes a future EXACTLY (set-equality) operator in the enum javadoc. Co-authored-by: Cursor <cursoragent@cursor.com>

codecov · 2026-09-02T10:56:10Z

Codecov Report

❌ Patch coverage is 85.71429% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.19%. Comparing base (330cbc2) to head (6137a94).

Files with missing lines	Patch %	Lines
...1/vistors/PostgresFilterTypeExpressionVisitor.java	83.33%	7 Missing and 6 partials ⚠️
...ore/mongo/query/parser/MongoArrayFilterParser.java	90.62%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #323      +/-   ##
============================================
+ Coverage     81.06%   81.19%   +0.13%     
- Complexity     1617     1677      +60     
============================================
  Files           243      243              
  Lines          7656     7765     +109     
  Branches        755      769      +14     
============================================
+ Hits           6206     6305      +99     
- Misses          960      966       +6     
- Partials        490      494       +4

Flag	Coverage Δ
integration	`81.19% <85.71%> (+0.13%)`	⬆️
unit	`58.49% <84.82%> (+1.33%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

suddendust · 2026-09-02T11:06:36Z

+                      .build())
+              .build();
+      // Only id 3 has exactly one element, and it is Black
+      assertEquals(1, collection.count(oneBlackOrWhite));


Nit: Would be better to assert on the actual docIds for 100% confidence.

suddendust · 2026-09-02T11:10:57Z

+    }
+
+    /**
+     * A non-array value (here props.brand, a string) must simply not match instead of failing the


If this how we're handling other operators too? Can you validate what happens when the LHS type does not conform to the RHS for an existing operator?

We validated ALL/EXACTLY_ONE type mismatch against existing EQ and IN in this suite (string vs number).

Mongo EQ and IN: 0 rows, no throw.

Postgres EQ of "10" on numeric quantity: matches (JSON ->> is text, so "10" equals stored 10).

Postgres IN of ["10"] on the same field: 0 rows (jsonb containment is typed).

ALL/EXACTLY_ONE with numeric RHS against string array props.colors: 0 rows on both Mongo and Postgres — same family as Mongo EQ/IN and Postgres IN, not Postgres EQ's text coercion. No new error path; mismatch is empty result for this JSON/document case.

suddendust · 2026-09-02T11:11:47Z

+     */
+    @ParameterizedTest
+    @ArgumentsSource(AllProvider.class)
+    void testAllAndOneOnNonArrayValueDoNotMatch(String dataStoreName) {


Also, what happens in this case for top-level non-array fields?

for Postgres flat native column (item as text) scalar type non-array throws we say "operator does not exist: text @> text[]"

Document JSONB (mongo + postgres item) -> “top-level non-array → no match,”

suddendust · 2026-09-02T11:12:33Z

+                      .operator(ArrayOperator.ALL)
+                      .filter(
+                          RelationalExpression.of(
+                              IdentifierExpression.of("props.metadata.colors"),


Should we not use JsonIdentifierExpression here?

JsonIdentifierExpression on nested document collections — NestedPostgresColTransformer rejects it. Nested ITs still use IdentifierExpression.of("props.colors"). Flat JSONB does use JsonIdentifierExpression.of("props", "colors"). Honest reply: we used it where the transformer supports it; document nested path still uses dotted identifiers.

suddendust · 2026-09-02T11:17:23Z

+                      .filter(
+                          RelationalExpression.of(
+                              IdentifierExpression.of("props.colors"),
+                              IN,


What does IN mean here semantically?

IN is element membership, not a second array-IN.

ALL + IN [a,b]: every RHS value is in the stored array (extras on the array OK).
EXACTLY_ONE + IN [a,b]: stored array length is 1, and that element is in {a,b}.
So EXACTLY_ONE ["Code","Live Traffic"] = singleton Code or singleton Live Traffic, not the two-element array ["Code","Live Traffic"].

Added test and also add the same in PR desc as well

suddendust · 2026-09-02T11:30:50Z

  }
+
+  @Test
+  void testAllOperatorWithJsonbArrayField() {


So this is the legacy PG flow, with no support for first-class columns. That's why you'll see everything is accessed via -> accessor (for ex: jsonb_typeof(document->'tags')). What we need is the flat collection flow. You can parse queries for flat collections using:

PostgresQueryParser postgresQueryParser = new PostgresQueryParser(TEST_TABLE, PostgresQueryTransformer.transform(query), new FlatPostgresFieldTransformer());

Can you please add test cases for that? I am actually surprised that I am unable to find a test class that tests parsed queries for flat collections. If that is indeed the case, would you mind creating one? Thanks :)

But do keep these tests

[true,false] matches ids 5 and 8 (each a one-element flags array), not a parser bug; add [true]→1 to pin the singleton-true case. Co-authored-by: Cursor <cursoragent@cursor.com>

suddendust · 2026-09-03T09:08:22Z

+    @ParameterizedTest
+    @ArgumentsSource(AllProvider.class)
+    void testAllAndOneOnNonArrayValueDoNotMatch(String dataStoreName) {
+      Collection collection = getCollection(dataStoreName);


Let us run these tests on flat collections please: Collection flatCollection = getFlatCollection(dataStoreName);.

Collection collection = getCollection(dataStoreName);: This returns the legacy storage mode PG in and therefore it's using jsonb_typeof guard.

suddendust · 2026-09-03T09:27:57Z

+     */
+    @ParameterizedTest
+    @ArgumentsSource(AllProvider.class)
+    void testAllIsOrderIndependent(String dataStoreName) throws IOException {


Should we also validate this for flat?

suddendust · 2026-09-03T09:28:10Z

+     */
+    @ParameterizedTest
+    @ArgumentsSource(AllProvider.class)
+    void testAllWithDuplicatesInDocumentArray(String dataStoreName) throws IOException {


Same as above, validate for flat?

suddendust · 2026-09-03T09:28:29Z

+     */
+    @ParameterizedTest
+    @ArgumentsSource(AllProvider.class)
+    void testOneWithDuplicatesInDocumentArray(String dataStoreName) throws IOException {


suddendust · 2026-09-03T09:29:41Z

Can you add a test for parsed queries for flat collections? We might need that information for perf tuning.

lrathod requested review from avinashkolluru, kotharironak, puneet-traceable, skjindal93, suddendust and suresh-prakash as code owners August 27, 2026 11:16

suddendust reviewed Aug 31, 2026

View reviewed changes

lrathod and others added 2 commits September 1, 2026 11:16

lrathod and others added 3 commits September 1, 2026 11:39

Rename ArrayOperator.ONE to EXACTLY_ONE

ce97a5d

Aligns with the service-level MATCH_EXACTLY_ONE name and reads unambiguously ("exactly one element, in the given set"). Also notes a future EXACTLY (set-equality) operator in the enum javadoc. Co-authored-by: Cursor <cursoragent@cursor.com>

lrathod changed the title ~~Add ALL and ONE array filter operators for MongoDB and Postgres~~ Add ALL and EXACTLY_ONE array filter operators for MongoDB and Postgres Sep 1, 2026

suddendust reviewed Sep 2, 2026

View reviewed changes

Fix EXACTLY_ONE native-array test to expect both singleton docs.

6137a94

[true,false] matches ids 5 and 8 (each a one-element flags array), not a parser bug; add [true]→1 to pin the singleton-true case. Co-authored-by: Cursor <cursoragent@cursor.com>

suddendust reviewed Sep 3, 2026

View reviewed changes

Uh oh!

Conversation

lrathod commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

MongoDB

Postgres

Design note

Test coverage / EXACTLY_ONE semantics

Test plan

Uh oh!

suddendust Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

suddendust commented Aug 31, 2026

Uh oh!

suddendust commented Aug 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lrathod commented Sep 1, 2026

Uh oh!

codecov Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lrathod Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

suddendust commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lrathod commented Aug 27, 2026 •

edited

Loading

suddendust Aug 31, 2026 •

edited

Loading

codecov Bot commented Sep 2, 2026 •

edited

Loading

lrathod Sep 2, 2026 •

edited

Loading