HBASE-29974: Persist filter hints across scan circuit breaks

[shubham-roy]

JIRA

HBASE-29974

---

Problem / Motivation

HBase's scan pipeline applies several structural short-circuit checks in
UserScanQueryMatcher.matchColumn — time-range gate, column-set exclusion, and
version-limit exhaustion — before Filter.filterCell is ever called. When any
of these gates fire, the pipeline returns a plain SKIP or SEEK_NEXT_COL code
and the filter is bypassed entirely.

Similarly, RegionScannerImpl iterates through every cell in a rejected row
one-by-one via nextRow() after filterRowKey returns true, even though a
filter can deterministically compute the next valid
row boundary.

The consequence is that filter implementations that could provide meaningful
seek hints are never consulted at these decision points, forcing the scanner to
advance cell-by-cell rather than issuing a single requestSeek. For
range-oriented or sparse-column filters over large tables this results in
significant, avoidable read amplification.

---

Root Cause

The existing getNextCellHint / SEEK_NEXT_USING_HINT mechanism requires
filterCell to have been called first; there was no API contract or call-site
for filters to provide a forward hint when the cell was structurally excluded
before reaching filterCell. The three structural gates in matchColumn and the
filterRowKey rejection path all lacked hook points.

---

Solution

New public API (Filter)

Two new optional methods are added to the abstract Filter class, both with
backward-compatible null defaults in FilterBase and FilterWrapper:

Method	When called	Contract
getSkipHint(Cell skippedCell)	Cell rejected by time-range, column-set, or version gate before filterCell	Stateless — must not modify filter state
getHintForRejectedRow(Cell firstRowCell)	Row rejected by filterRowKey before any cell-level iteration	May use state set during filterRowKey

UserScanQueryMatcher (server-side)

• Introduces pendingSkipHint field and resolveSkipHint() helper.
• At each of the three structural short-circuit branches (time-range tsCmp > 0,
  time-range tsCmp < 0, column-set exclusion, version SKIP/SEEK_NEXT_COL),
  calls resolveSkipHint() and promotes the result to SEEK_NEXT_USING_HINT
  when a non-null hint is returned.
• getNextKeyHint drains pendingSkipHint first, before delegating to
  filter.getNextCellHint, ensuring the correct hint is returned for the cell
  that triggered the structural gate.

RegionScannerImpl (server-side)

• getHintForRejectedRow(Cell) validates and retrieves the filter hint after a
  filterRowKey rejection.
• nextRowViaHint(ScannerContext, Cell, ExtendedCell) replaces the nextRow()
  call with a single storeHeap.requestSeek(hint), wrapped with skippingRow
  mode to keep block-size accounting consistent.

---

Backward Compatibility

• No breaking changes. Both new Filter methods return null by default
  (defined on Filter directly, with matching no-ops in FilterBase and
  FilterWrapper), so all existing filter implementations continue to behave
  identically.
• DoNotRetryIOException is thrown (consistent with existing getNextCellHint
  validation) if a filter returns a non-ExtendedCell from either new method.

[virajjasani]

@shubham-roy could you rebase with latest master?

[virajjasani]

FYI @tkhurana

[virajjasani]

Error Type	Count	Module	Tests
Failures	1	hbase-server	org.apache.hadoop.hbase.io.hfile.bucket.TestPrefetchWithBucketCache.testPrefetchRunTriggersEvictions
Flakes	3	hbase-server	org.apache.hadoop.hbase.master.balancer.TestBalancerDecision.testBalancerDecisions org.apache.hadoop.hbase.master.region.TestMasterRegionWALCleaner.test org.apache.hadoop.hbase.security.access.TestZKPermissionWatcher.testPermissionsWatcher

[virajjasani]

❌	spotless	0m 17s	patch has 63 errors when running spotless:check, run spotless:apply to fix.

[shubham-roy]

@virajjasani , I have resolved merge conflicts and have also applied spotless command.

For the test failures:
org.apache.hadoop.hbase.io.hfile.bucket.TestPrefetchWithBucketCache.testPrefetchRunTriggersEvictions

The above passes in my local. Are we sure it is not a flapper?

Also, I believe the 3 count of flakes can be ignored, right?