Hybrid eager/deferred accumulation for string_agg GroupsAccumulator to reduce copying and memory usage

[kosiew]

Which issue does this PR close?

• Part of Consider deferred copying in string_agg GroupsAccumulator #21156

---

Rationale for this change

The current StringAggGroupsAccumulator eagerly copies every input string into per-group String buffers during update_batch. This approach is simple but can lead to significant memory overhead and unnecessary copying, especially for large payloads or high-cardinality groupings.

This PR introduces a hybrid strategy that defers copying when it is likely to be beneficial. Instead of immediately materializing strings, the accumulator can retain references to input batches and store lightweight (group_idx, row_idx) entries. Actual string concatenation is deferred until evaluate().

This approach aims to:

• Reduce memory duplication for large strings
• Improve performance for workloads with many groups and large payloads
• Maintain efficiency for small inputs via an eager fast path

---

What changes are included in this PR?

• Introduced a hybrid accumulation model:

  • Eager path: existing behavior for small inputs
  • Deferred path: stores references to input batches and row indices

• Added new internal structures:

  • batches: Vec<ArrayRef> to retain input arrays
  • batch_entries: Vec<Vec<(u32, u32)>> to track (group_idx, row_idx) pairs
  • num_groups to track total group count

• Introduced StringInputArray abstraction to unify handling of:

  • Utf8
  • LargeUtf8
  • Utf8View

• Implemented heuristics to decide when to defer:

  • DEFER_GROUP_THRESHOLD
  • DEFER_PAYLOAD_LEN_THRESHOLD

• Refactored append logic:

  • append_rows_typed for deferred indexing
  • append_batch_typed for eager materialization
  • append_batch_values_typed for reconstruction during evaluation

• Updated evaluate() to:

  • Materialize deferred data into output
  • Support partial emits (EmitTo::First)
  • Retain un-emitted state correctly

• Added state management improvements:

  • clear_state() to fully reset buffers
  • retain_after_emit() to compact deferred state after partial emits

• Extended memory accounting in size() to include:

  • retained batches
  • batch entry metadata

• Added test:

  • groups_mixed_eager_and_deferred_batches to validate correctness of hybrid behavior

---

Are these changes tested?

Yes.

• Added a new test covering mixed eager and deferred batches

• Existing tests continue to pass

• The new test verifies:

  • Correct concatenation across eager + deferred inputs
  • Correct handling of partial emits
  • Proper retention of remaining groups after emission

---

Are there any user-facing changes?

No.

• This change is internal to the execution engine
• No API or SQL behavior changes
• Expected improvements are limited to performance and memory usage characteristics

---

LLM-generated code disclosure

This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed and tested.

[kosiew]

Benchmark (#21437)

              Criterion Benchmark Summary (Statistically Significant Changes)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark                                                                                     ┃ Mean Change ┃  P-value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ aggregate_query_approx_percentile_cont_on_f32                                                 │      -4.93% │ 0.000000 │
│ aggregate_query_approx_percentile_cont_on_u64                                                 │      -4.68% │ 0.000000 │
│ aggregate_query_distinct_median                                                               │      -1.59% │ 0.000000 │
│ aggregate_query_group_by                                                                      │      -4.48% │ 0.000000 │
│ aggregate_query_group_by_u64 15 12                                                            │      -1.34% │ 0.000000 │
│ aggregate_query_group_by_u64_multiple_keys                                                    │      -3.66% │ 0.000000 │
│ aggregate_query_group_by_wide_u64_and_f32_without_aggregate_expressions                       │      -5.07% │ 0.000000 │
│ (aggregate_query_group_by_wide_u64_and_f32_without_aggregate_expr)                            │             │          │
│ aggregate_query_group_by_wide_u64_and_string_without_aggregate_expressions                    │      -4.94% │ 0.000000 │
│ (aggregate_query_group_by_wide_u64_and_string_without_aggregate_e)                            │             │          │
│ aggregate_query_group_by_with_filter                                                          │      -3.93% │ 0.000000 │
│ aggregate_query_no_group_by_count_distinct_wide                                               │      -4.73% │ 0.000000 │
│ aggregate_query_no_group_by_min_max_f64                                                       │      -3.28% │ 0.000000 │
│ array_agg_query_group_by_few_groups                                                           │      -3.85% │ 0.000000 │
│ array_agg_query_group_by_many_groups                                                          │      -3.06% │ 0.000000 │
│ array_agg_query_group_by_mid_groups                                                           │      -5.56% │ 0.000000 │
│ array_agg_struct_query_group_by_mid_groups                                                    │      -3.55% │ 0.000000 │
│ first_last_ignore_nulls                                                                       │      -1.65% │ 0.000000 │
│ first_last_many_columns                                                                       │      -1.76% │ 0.000000 │
│ first_last_one_column                                                                         │      -4.00% │ 0.000000 │
│ string_agg_payloads/few_groups/large_1024b (large_1024b)                                      │      -4.48% │ 0.000000 │
│ string_agg_payloads/few_groups/medium_64b (medium_64b)                                        │      -2.78% │ 0.000000 │
│ string_agg_payloads/few_groups/small_3b (small_3b)                                            │      -4.02% │ 0.000000 │
│ string_agg_payloads/many_groups/large_1024b (large_1024b)                                     │      -2.12% │ 0.000000 │
│ string_agg_payloads/many_groups/medium_64b (medium_64b)                                       │      -5.19% │ 0.000000 │
│ string_agg_payloads/many_groups/small_3b (small_3b)                                           │      -4.37% │ 0.000000 │
│ string_agg_payloads/mid_groups/large_1024b (large_1024b)                                      │      -4.01% │ 0.000000 │
│ string_agg_payloads/mid_groups/medium_64b (medium_64b)                                        │     -12.10% │ 0.000000 │
└───────────────────────────────────────────────────────────────────────────────────────────────┴─────────────┴──────────┘

Summary: 26 improvements, 0 regressions (p < 0.05)