Add upload status tracking and per-upload change detection (issue #205) - #208
Conversation
bf1f202 to
d123c69
Compare
d123c69 to
cc46f8f
Compare
|
I am trying to understand the workflow that will make use of this PR. My theory is the following:
Assuming that is the workflow we are targeting then I think there are some aspects of the proposed solution that we should consider
|
Basically yes.
Yes, I acknowledge it's limiting atm.
Imo they have different lifecycles, ProcessingService manages persisted ProcessingTrackerEntity for crash recovery, while UploadService is intentionally in-memory with short retention for real-time polling.
I agree it simplifies the h5m codebase to just return detection value IDs and let clients query details via ValueService. However, I think that shifts the complexity to the API consumer and makes the user experience worse. The intended workflow for a CI integration is:
With List detectionValueIds, the client would need to make N additional API calls to fetch each value, then look up the node for each to understand the detection type. For a CI system that just wants to know "did this upload trigger any regressions?", the Change summary gives enough context (node name, detection type, value ID) in one response without extra round-trips. I took another look at this pr and how it "works" with the current notification system regarding changes. I'm thinking that the "push/pull" change system has diverged a bit. I'm looking at changing this pr now such that they have the "same" backend system. |
b319450 to
5885110
Compare
willr3
left a comment
There was a problem hiding this comment.
I do not think we should have sql to fetch Values outside of the ValueService.
76d8eca to
1656252
Compare
1656252 to
0b6d82b
Compare
…uery (issue Hyperfoil#205) Separate upload processing status from value querying, following REST best practices (Will's feedback): Processing status endpoint: GET /api/processing/{id} → {id, state, error} Backed by WorkService.getTracker() with DB fallback (checks if root value exists to infer COMPLETED after tracker cleanup). Value descendants endpoint: GET /api/value/{id}/descendants?detection=true → List<Value> Returns detection node values scoped to an upload (root value). Uses in-memory cache (populated by ChangeDetectedEvent observer in ValueService) with DB fallback via recursive CTE query. Without ?detection=true, returns all descendant values. Eliminated components: - UploadService — replaced by ValueService cache + WorkService tracker. Processing state from WorkService.getTracker(), detection values from ValueService.getDetectionDescendants(). - UploadStatus record — processing status is a simple ProcessingStatus record on ValueResource. - FolderResource.getUploadStatus() — moved to ValueResource. Change record stays as internal event type: - Used by ChangeDetectedEvent → NotificationService → plugins - Not exposed in REST API — pull consumers get Value DTOs - Push consumers (webhooks) get enriched Change records Detection value cache in ValueService: - ConcurrentHashMap<Long, List<Value>> keyed by root value ID - Populated by @observes ChangeDetectedEvent observer - 10-minute retention, max 1000 entries - getDetectionDescendants() checks cache, falls back to DB
0b6d82b to
441ff94
Compare
Adds REST endpoint for polling upload processing status and retrieving change detection results triggered by specific uploads.
New endpoint:
GET /api/folder/upload/{uploadId}/status
Returns UploadStatus: state (PROCESSING/COMPLETED/FAILED), durationMs,
error message, and list of Change records with changeType enum.
Implementation: