feat: support irspack's feature-aware iALS

[marevol]

Adds support for irspack 0.5.0's feature-aware iALS. A new optional features: recipe block declares item- and/or user-side attribute tables; recotem encodes them into numeric matrices, feeds them to IALSRecommender through Optuna search and the final refit, persists the encoder state in the signed artifact, and uses it at serve time to recommend for users and seed items absent from training.

What it looks like

features:
  item:
    source: {type: bigquery, query: "SELECT item_id, genres, year FROM items"}
    id_column: item_id
    columns:
      - {name: genres, encoding: multi_label, delimiter: "|"}
      - {name: year,   encoding: numerical}
  user:
    source: {type: csv, path: ./users.csv}
    id_column: user_id
    columns:
      - {name: age_band, encoding: categorical, min_frequency: 5}

The block's presence enables feature-aware training — no separate flag. Feature tables come from the existing datasource registry (csv / parquet / bigquery / sql / plugins), which turned out to be role-agnostic: FetchContext carries no interaction semantics.

Cold start reaches irspack's feature API on all four verbs:

Case	Verb
A: features only	:recommend + user_features
B: features + ad-hoc history	:recommend-related + user_features
C: seed item absent from training	:recommend-related + item_features

Working example under examples/feature-aware/.

Notable design decisions

Feature-aware iALS is not a new irspack class — it is IALSRecommender with extra kwargs, so best_class cannot distinguish it. The existing 0.4↔0.5 skew guard already covers it: the IALSModelConfig.__setstate__ 7→10 arity change it refuses is this feature.

lambda_*_feature is tuned by Optuna (5e-2–1e6, log — the range upstream's own example exercises). irspack ships no default range and its constructor default of 0.0 is a hard error whenever features are present. This is the first exception to "hyperparameter ranges come from irspack"; docs/recipe-reference.md is amended accordingly. Note the range's upper bound is not a verified features-off bound — for a true on/off comparison, run two recipes.

The artifact FQCN allow-list is unchanged. The encoder state is plain Python containers only — dict / list / str / int / float, no numpy, no classes, no sklearn estimators, no pandas objects — specifically so artifact/signing.py never widens (docs/operations.md documents sklearn as a known unguarded unpickle axis). git diff main -- src/recotem/artifact/signing.py is empty. The str() coercions on vocabulary keys are load-bearing: they are the only thing keeping numpy scalars out of the state. Note the allow-list does not back them up — numpy.str_ pickles via numpy._core.multiarray.scalar + numpy.dtype, both allow-listed, so it would load if it ever leaked in (harmlessly, since it is a str subclass, but the state would no longer be plain). pandas.Index is genuinely refused. (Corrected from an earlier claim that numpy.str_ was not allow-listed.)

A new features header descriptor doubles as the payload-shape gate recotem lacked. FORMAT_VERSION describes the container only and recotem_version is never gated at serve time. Unknown/newer/malformed versions fail closed — a changed state shape would encode request features into the wrong vector space and return silently incorrect recommendations. A pre-feature serve stays ungated and needs no gating: it has no feature code, never reads the state, and its known-user recommendations remain correct.

Two hazards addressed structurally, not by care

Axis alignment. The search phase orders items by list(set(...)) — unsorted, and for string ids not stable across processes — while the final refit uses pd.Categorical (sorted). irspack raises on a row-count mismatch but accepts a misordered matrix silently: it trains and the Evaluator scores it with no error. encode() therefore takes index_order as a required argument and always reindexes; there is no API returning a matrix without naming its row order, and no caching. A standing in-suite guard re-runs the alignment test in subprocesses under three fixed PYTHONHASHSEED values.

Partial feature injection. lambda_*_feature without a matrix trains silently as plain iALS. search.py had two construction sites (the per_trial_timeout_seconds path and the default path); both now route through a single _construct helper whose asserts turn irspack's silent asymmetry into a loud one.

Compatibility

Verified with real signed artifacts through the real write/read/unpickle path:

Direction	Behavior
New payload → old code	Loads; __setstate__ absorbs unknown keys; recommend unaffected
New features header → old code	Tolerated; the header is a plain dict, not a pydantic model
Old artifact → new code	Class-level default resolves to None
New recipe → old load_recipe	Fails closed: RecipeError: features: Extra inputs are not permitted
Cold-start request → old serve	Not silently ignored (every request model is extra="forbid"): 422 on the single verbs; on the batch verbs, 200 with a per-element VALIDATION_ERROR (batch elements are validated per-element by design)

A non-feature artifact's header has the same key set as main's — the features key is absent, not null — so an older serve loads it fine. (It is not byte-identical: recipe_hash is a header field and every recipe's hash changes, per the note below.)

Security

features.*.source gets the same path-scheme allow-list, ::-chain rejection, embedded-credential rejection, and sha256-for-network rule as source, through the same shared helpers. Plugin-declared no_expand_fields reaches feature subtrees, so SQLSource's dsn_env keeps its env-expansion protection.

user_features carries PII by construction. Raw values are never logged (column names and counts only); log_redaction also redacts the keys as a mechanical backstop.

Known limits

• Feature dimension scales with catalog size, not interaction count — the vocabulary is built from the whole feature table so cold-start entities are representable. min_frequency is the only recipe-level lever; cost is cubic in the dimension and multiplies with training.parallelism. Sized in docs/operations.md.
• A large-but-not-solver-breaking numerical value degrades silently and is not counted. Clamping the standardized magnitude was deliberately deferred — it is a modelling decision, not a bugfix. A value extreme enough to break the solver returns 400 FEATURE_VALUE_UNUSABLE.
• An unknown feature column in a request is silently ignored — no error, no counter. An entirely bogus column encodes to the bias alone and returns the population prior as a normal 200. Counting it is additive and can land later; rejecting unknown keys cannot, once clients ship. Flagged for a decision.
• Recommending items absent from training (irspack's get_score_from_item_features) is a non-goal — it spans a different score axis and needs its own verb.

Every recipe's recipe_hash changes

features is a new optional field and _compute_recipe_hash dumps without exclude_none, so existing recipes now serialize {"features": null}. Nothing compares or gates on recipe_hash, but it is emitted for SIEM consumption, so operators pinning it will see a spurious change. Noted in CHANGELOG.md.

Pre-existing bugs found, left out of scope

• time_global silently drops a user: 1 - 0.3 - 0.0 == 0.7 exactly but 1 - 0.7 == 0.30000000000000004, so the ratio lands at 0.9999999999999998 and int() truncates a user into the test split, which recotem never reads. Verified features still align correctly under it.
• datasource/csv.py's _validate_required_columns is dead in production — the sole FetchContext construction passes no extra, so it always returns early.

Verification

pytest tests 2120 passed, 1 skipped; pytest -m slow 4 passed; tests/e2e/run.sh passes; ruff + ruff format clean.

Review fixes (commit adad2b6)

A multi-reviewer audit — followed by a second audit of the fixes themselves — found several defects, all addressed in the follow-up commit on this branch:

Correctness (client-facing):

• An oversized numeric feature value (a ≥309-digit JSON integer) raised OverflowError past except (TypeError, ValueError) and returned an uncaught HTTP 500; it now routes to the counted-unknown path (200).
• encode/encode_one now mirror the fitting parser (pd.to_numeric) across the whole value domain, so a value the fit dropped — underscored, non-ASCII, bytes/bytearray/Fraction — can no longer be encoded against statistics it never contributed to (it silently encoded as a large-sigma outlier before).
• The cold-start 400s now set explicit status labels, so client errors are no longer counted as status="error" (which pages on-call at the documented 10% threshold).

Silent-failure guards (training):

• Training refuses at 0% feature-table id overlap with the interaction axis, and when a feature block encodes to bias only — both otherwise sign an artifact advertising features for what is really plain iALS. (The dtype trap: one blank cell makes pandas infer float64, so 1 reads back as 1.0 and never matches the axis's "1".)
• A dead column (empty or constant vocabulary) now warns, at parity with the existing zero-variance numerical warning.
• The partial-feature-injection guards are now an unconditional raise — the original asserts were stripped under -O. (So the "single _construct helper whose asserts…" note above is now a raise, not an assert.)
• exclude_items is unified: the cold-start verbs post-filter like every pre-existing verb, instead of back-filling to a full page (which changed the returned item count when user_features was present).

Observability / limits:

• Unknown feature columns are counted per request per side (no column-name label, to bound cardinality); total cold-seed solves per batch request are capped.
• Source-validation errors name the recipe file again.

Docs/comments were corrected to match the code — including the three claims above (encoder-state contents, the numpy.str_ allow-list behavior, and the header key-set-vs-byte-identical wording), plus the index_order invariant (requiring the argument makes omission unrepresentable, not misalignment), the near-constant numerical floor, and the new feature_axis_error exit-4 runbook entry.

Out of scope (pre-existing, flagged for follow-up): training/lock.py's -O-strippable assert; ruff not selecting the S ruleset (so # noqa: S101 is decorative); a features: recipe pulling optuna into recotem serve (same as main's per_algorithm_trials); a broken anchor link in docs/plugin-authoring.md.

Review fixes, second round (commit 0a8313a)

A follow-up multi-agent audit (spec-completeness, correctness/regression, security, test-coverage, plus an adversarial pass) confirmed the substantive claims above and surfaced one MAJOR gap plus doc/test polish, addressed in this commit:

Security (MAJOR):

• The 8192-char cap covered only string values; feature-dict keys were length-unbounded and there was no request-body size limit at all, so an authenticated client could send GB-scale bodies that Starlette buffers and parses before validation. Now: every feature-dict key is bounded 1–256 chars (user_features keys, item_features outer seed-id keys typed _ItemStr, nested per-seed keys), and a new BodySizeLimitMiddleware caps the whole body at RECOTEM_MAX_BODY_BYTES (default 128 MiB — clears the ~72 MiB legitimate worst case; clamped [1 MiB, 2 GiB]) with 413 PAYLOAD_TOO_LARGE, enforced on both Content-Length and chunked bodies.

Docs:

• README now advertises the feature-aware iALS / cold-start capability.
• api-reference documents the silently-ignored unknown feature key (200, metric-only signal — clients must not rely on key validation), the 413 code, and all length/size bounds.
• recipe-reference extends the dtype-trap note to categorical/multi_label value columns (declare id-like columns as strings at the source).
• The compatibility-table cell above is corrected: old-serve rejection of cold-start requests is 422 on single verbs but per-element VALIDATION_ERROR inside a 200 on batch verbs. The dangling design-doc link was removed (the file was never committed).

Tests:

• The fuzz corpus now includes a features:-bearing seed YAML (mutations reach the new nested validation), plus arbitrary-value injection at the features: key.
• A 0-row feature table is pinned to feature_table_error (exit 4).
• New body-cap (Content-Length / chunked / boundary) and key-length tests.

Reviewed, deliberately unchanged: the recipe_hash shift (disclosed above; nothing gates on it — freezing the hash schema is a separate decision), the recotem validate stdout wording (intentional), the bounded id samples in the zero-overlap error (deliberate diagnostic tradeoff), and the cold-seed cap counting warm feature-carrying seeds (errs strict, never loose).