Stop the orphan sweep from deleting workspace logos - #161
Merged
Conversation
The daily orphan sweep built its "referenced" set from DocumentRevision and Attachment rows only. Workspace logos are written through writeBlob, so their keys match ATTACHMENT_KEY_RE and the sweep enumerates them -- but they are owned by Workspace.logoStorageKey, not by an Attachment row. Any logo older than the one-hour grace period was therefore classified as crash residue and deleted. The read path then compounded it: on StorageNotFoundError it nulls the logo columns, so the user sees an empty logo field rather than a broken image, with nothing left in the DB to show one was ever uploaded. That is why this reads as "the upload randomly does not stick" instead of an error. Keep the self-healing clear, but log a warning first so the next occurrence of this class of bug is visible in logs. The sweep is a 24h setInterval registered at boot, which is why it only bites long-uptime deployments -- boxes that redeploy daily never fire it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Uploaded images were disappearing and users had to re-upload them. The cause is the daily orphan sweep deleting live blobs.
sweepOrphanObjectslistsobjects/,attachments/andworkspaces/, then keeps anything present in areferencedset built fromDocumentRevision.storageKeyandAttachment.storageKey. Workspace logos are written throughwriteBlob, so their keys matchATTACHMENT_KEY_REand the sweep enumerates them — but a logo is owned byWorkspace.logoStorageKey, not by anAttachmentrow. Any logo older than the one-hour grace period was classified as crash/rollback residue and removed.The read path then hid the evidence: on
StorageNotFoundErrorit nullslogoStorageKey/logoContentType/logoSha, so the user sees an empty logo field rather than a broken image, and the DB retains no sign a logo was ever set. That is why this presents as "the upload sometimes doesn't stick" rather than as an error.It is intermittent because the sweep is a
setInterval(..., 24h)registered at boot and.unref()'d — it only ever fires on a process that stays up a full day. Deployments that restart frequently never trigger it; long-uptime ones lose logos every 24 hours.Workspace.logoStorageKeyto the sweep'sreferencedset, with a comment stating the invariant: every owner of a content-addressed key must be enumerated there.Note for the cloud fork:
User.signatureStorageKeyhas the exact same defect and is the originally reported symptom. It is cloud-only (noapps/server/src/users/here), so it is fixed in the matching pageden-cloud PR.Numbers
Verification that the test catches the bug
With the sweep change reverted and the new test alone:
Passes with the fix applied.
Test plan
pnpm -r typecheckpnpm -r lintpnpm --filter @pageden/server testpnpm --filter @pageden/server test:integrationpnpm --dir apps/server exec tsx scripts/sweep-orphans.ts, and confirm the logo still renders afterward.Note on already-affected data
Logos already swept are unrecoverable — the blobs are deleted and the pointer columns nulled. Affected workspaces need one re-upload after this ships.