Add pet clustering with feedback, naming, and search#9665
Add pet clustering with feedback, naming, and search#9665Amrithesh-Kakkoth wants to merge 119 commits intoente-io:mainfrom
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 536ec02e71
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
3-phase Rust clustering pipeline (face cluster, body rescue, cross-merge) using average-linkage agglomerative clustering, validated against Python reference with 100% agreement (NMI=1.0). Includes Dart service layer, DB migrations, API bindings, and test binary for comparison.
Call PetClusteringService after face clustering in runAllML so pet clusters are formed automatically after indexing completes.
- LazySession was forcing CPU-only for ALL models; now only pet models use CPU-only policy (to avoid NNAPI/CoreML issues with FP16 models) - Restore preferNnapi and preferXnnpack to Platform.isAndroid - Restore GraphOptimizationLevel::Level3 (was downgraded to Level1) - Restore arena allocator on CPUExecutionProvider
Sessions stay loaded across images and get cleaned up when the isolate disposes via release_runtime(). Avoids expensive repeated ONNX session load/unload cycles on every image.
This was debug-only code that altered production indexing logic to re-process all files for pet-only mode. Removes the flag, the forceAll parameter threading through ml_util, and the ClipText skip hack.
- Inline embedding columns into CREATE TABLE (remove ALTER TABLE migrations) - Store landmarks inside detection JSON (remove separate column) - Remove migration tracking from PetVectorDB (tables are new) - Rename pet_bodies table to detected_objects with generic column names - Change species column from TEXT to INTEGER (store raw COCO class) - Revert shouldAutomaticDispose back to true - Add separate petMlVersion constant - Move embedding storage methods from MLService to MLDataDB
Add pet_indexed_files tracking table so files already indexed for faces+CLIP are re-queued when pet recognition is enabled. Also fix MLResult.ranML to include petsRan, which was causing pet-only analysis results to be silently discarded.
…sc bugs - Migrate ML runtime from Mutex to RwLock + OnceCell for concurrent inference (CLIP text no longer blocks image indexing) - Fix cat face detection: model outputs per-class scores, not a class ID; use argmax instead of truncating float to u8 - Fix species normalization: store 0/1 (dog/cat) not COCO 15/16 - Fix markPetIndexed guard to check shouldRunPets && rustPets - Add pet model paths to runtime config cache key - Batch pet body embeddings by species (matching face pattern) - Rename objectIDColumn to detectedObjectIDColumn to avoid ambiguity - Implement deletePetDataForFiles in DB layer - Fix design system usage in pet thumbnail widget
…ing column Co-Authored-By: Amrithesh <amritheshkakkoth10@gmail.com>
Amrithesh-Kakkoth
force-pushed
the
ml/pet-clustering
branch
from
536ec02 to
a3b74d4
Compare
… missing schema tables
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2def4bb422
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
mobile/apps/photos/lib/services/machine_learning/pet_ml/pet_clustering_service.dart
Outdated
Show resolved
Hide resolved
Extract shared clustering algorithm from pet module into reusable ml::cluster module. Add face-specific batch and incremental clustering functions with rejection support. Wire as fallback in ml_service when Dart linear fails and useRustForML is enabled. Also remove dead petIndexedFilesTable and redundant markPetIndexed calls.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1f9c592313
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…esh, species-filtered moves
|
@codex review |
|
@codex review |
Offline mode is view-only raw clustering output. Skip not-pet feedback table creation/deletion and rejected-assignment overrides in offline mode. Disable name editing, cluster view, merge, and selection bar in pet cluster UI when offline.
mobile/apps/photos/rust/Cargo.toml
Outdated
|
|
||
| [lib] | ||
| crate-type = ["cdylib", "staticlib"] | ||
| crate-type = ["cdylib", "staticlib", "rlib"] |
There was a problem hiding this comment.
Bump. check other old comments too
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0fe3d34071
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
mobile/apps/photos/lib/services/machine_learning/pet_ml/pet_service.dart
Outdated
Show resolved
Hide resolved
mobile/apps/photos/lib/services/machine_learning/pet_ml/pet_service.dart
Show resolved
Hide resolved
# Conflicts: # mobile/apps/photos/pubspec.yaml # mobile/apps/photos/pubspec_overrides.yaml
Add secondaryActionLabel and onSecondaryTap to SavePersonBanner to fix assertion crash on People & Pets page. Clean local cluster-to-pet mappings when deleting a pet.
|
No dependency changes detected. Learn more about Socket for GitHub. 👍 No dependency changes detected in pull request |
Remove petClusterSummaryTable and petBodyVectorIdMappingTable from migration scripts as neither table is read or written in application code. Update SqlDbBase.migrate to gracefully downgrade user_version instead of crashing when migration count decreases.
Add merge, ignore, cluster removal, and file selection actions for pets, matching the existing person interaction patterns. - Merge: banner secondary action opens named pet grid for merging - Ignore: popup menu option hides pet cluster via isHidden flag - Remove clusters: red button on secondary clusters in clusters page - File selection: long-press photos, "Not this pet" records rejection feedback and reassigns faces to singleton clusters - Add petCluster to GalleryType enum with matching switch cases - Add l10n strings: viewClusters, ignorePet, confirmation dialogs
Pet incremental clustering uses exemplars (sampled embeddings), not centroids. The centroid VDB was write-only — computed and stored but never read back for any clustering decision. Remove the centroid compute/write from clustering service, delete the VDB file, and clean up all references in db.dart and schema.dart.
Offline list is identical to default — the notPetFeedback table operations are already guarded by isOffline checks at runtime.
2 participants
Summary