fix: reopen DuckLake catalog after restart and seed data dir ownership - #496
Conversation
Omit CREATE_IF_NOT_EXISTS when the local DuckLake catalog file already exists so same-volume recreate can ATTACH. Create /home/admin/data and /home/admin/log as admin in the image so empty Docker named volumes inherit uid 1000. Document object-table PRIMARY KEY rejection and named-volume ownership. Does not change v0.2.1 / latest.
leon-ape
left a comment
There was a problem hiding this comment.
Independent review of exact head 36ed564dcbbc6ccbfc5b40dee395acb35f0a53a9.
- Unique parent
93d2cf9db417038f5467b5753f4e078e24e4f9ba; GitHub verification valid. - Existing local catalog omits
CREATE_IF_NOT_EXISTS; missing catalog still sets it. TestTestDuckLakeAttachOmitsCreateIfCatalogExistslocks that. - Dockerfile creates
/home/admin/dataand/home/admin/logasadminafterUSER admin. - Docs record PRIMARY KEY rejection and named-volume ownership.
APPROVE binds only this OID. Does not accept image be2cd5bd or reopen task #78. Runtime remount/first-volume still need a new immutable image after merge. No v0.2.1 / latest change.
Restart failed because attachCatalogs attached ducklake.db as a normal DuckDB database, so the later ducklake: ATTACH returned driver_error. Skip the configured metadata file in AttachCatalog. Stat: only ErrNotExist selects CREATE_IF_NOT_EXISTS; permission/IO errors fail closed. CLI ATTACH of the r5 catalog succeeded both with and without CREATE_IF_NOT_EXISTS when the file was not already attached as a regular database.
leon-ape
left a comment
There was a problem hiding this comment.
Independent review of exact head 57058f32c47684a3b5f3c26e6a95806b9bf97483.
- Unique parent
36ed564dcbbc6ccbfc5b40dee395acb35f0a53a9; GitHub verification valid. duckLakeCatalogMissingonly treatsos.IsNotExistas missing; other Stat errors fail closed and do not emit ATTACH.AttachCatalogskips the configured DuckLake metadata file so it is not occupied as a regular database.- Tests: omit CREATE_IF_NOT_EXISTS when the file exists; skip metadata file; permission Stat does not create.
APPROVE binds only this OID. Does not promote be2cd5bd. Same-volume recreate and empty named-volume start still need a new immutable image after merge. No v0.2.1 / latest change.
Restart failed because attachCatalogs opened the metadata file as a regular DuckDB database. CREATE_IF_NOT_EXISTS is not the recreate cause: CLI ATTACH of the r5 catalog succeeded with and without that option when the file was not already attached. Replace chmod 000 Stat coverage with injected os.ErrPermission so the test is stable as root.
leon-ape
left a comment
There was a problem hiding this comment.
Independent review of exact head 4be256010814034131cf094b038047434dfc5933.
- Unique parent
57058f32c47684a3b5f3c26e6a95806b9bf97483; GitHub verification valid. - Comments and PR body attribute restart failure to attaching the metadata file as a regular database; CREATE_IF_NOT_EXISTS is not the recreate cause.
- Stat coverage injects
os.ErrPermissionviaduckLakeStat(stable as root) and expectsreason=permission_deniedwith no ATTACH SQL.
APPROVE binds only this OID. Does not promote be2cd5bd. Remount and empty named-volume start still need a new immutable image after merge. No v0.2.1 / latest change.
Summary
Fixes two 0.3-dev image blockers from task #88 (
be2cd5bdmust not be promoted):Same-volume recreate failed
ATTACHwithstage=attach reason=driver_error. Root cause:attachCatalogsATTACHes every*.dbin the data directory as a regular DuckDB database, so on restart the DuckLake metadata file is opened first and the laterducklake:ATTACH fails. Fix: skip the configured metadata path inAttachCatalog.CLI ATTACH of the r5 catalog copy succeeded both with and without
CREATE_IF_NOT_EXISTSwhen the file was not already attached as a regular database. Omitting that option on existing files is still kept (os.StatonlyErrNotExistcreates; permission/IO fail closed) but is not the recreate root cause.Empty Docker named volumes mounted at
/home/admin/datawereroot:root, so useradmin(uid 1000) could not createmyduck.db. The image now seeds/home/admin/dataand/home/admin/logas admin.Also documents object-table PRIMARY KEY rejection (1105, no leftover files).
Does not change
v0.2.1/latest. Does not touch PR #489.Test
go test ./catalog -count=1 -run 'TestDuckLakeAttachUsesServicePaths|TestDuckLakeAttachOmitsCreateIfCatalogExists|TestAttachCatalogSkipsDuckLakeMetadataFile|TestDuckLakeAttachStatNonExistErrorDoesNotCreate|TestDuckLakeAttachRejectsRemoteCatalogURI'