diff --git a/.changeset/connection-identity-dedup.md b/.changeset/connection-identity-dedup.md deleted file mode 100644 index b9d1d54..0000000 --- a/.changeset/connection-identity-dedup.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -'@walkthru-earth/objex': minor ---- - -Canonical connection identity and deduplication across every write path. - -- New `utils/connection-identity.ts` (exported from `src/lib/index.ts`): `connectionIdentityKey`, `isSameConnectionIdentity`, `normalizeEndpoint`, `normalizeProvider`, `ConnectionIdentityInput`. Identity is provider-aware: `azure` → `provider|endpoint|bucket`, `gcs` → `provider|bucket` (global namespace), `s3` with empty endpoint → `s3|bucket|region` (region is load-bearing for signing), all other S3-compatible providers → `provider|normalizedEndpoint|bucket`. `normalizeEndpoint` lowercases host, strips default ports (`:443`/`:80`) and trailing slashes, and preserves explicit non-default ports and pathnames so `http` vs `https`, `:443` vs empty, and trailing-slash drift collapse to one key. -- `connections` store: removed `findByBucketEndpoint` (bucket+endpoint string match, which produced silent duplicates for AWS same-bucket-different-region and custom S3-compat scheme/port drift, and was bypassed entirely by the manual Add Connection dialog). Every write path now dedups through `connectionIdentityKey`: - - `save(config)` returns `{ id, existed }`. On `existed: true`, the row is reused and credentials from the new config overwrite the old ones. - - `update(id, config)` throws the new `DuplicateConnectionError` when the new identity would collide with a different saved row, so edits can't silently produce phantom duplicates. - - `saveHostConnection(detected)` continues to be the auto-detect entry and returns the final id, either reused or newly inserted. - - New public `findByIdentity(input)` exposes the same key for callers that need to check without writing. -- `ConnectionDialog` surfaces both outcomes: amber "merged into existing" notice on dedup and destructive "already used by X" block on edit collision, with the offending connection's name. -- Build: svelte-check 0 errors, publint clean, no `$lib/` leaks in `dist/`. diff --git a/.changeset/deckgl-geotiff-0.6.md b/.changeset/deckgl-geotiff-0.6.md deleted file mode 100644 index a12f4af..0000000 --- a/.changeset/deckgl-geotiff-0.6.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -'@walkthru-earth/objex': minor -'@walkthru-earth/objex-utils': minor ---- - -Bump the `@developmentseed/deck.gl-geotiff` family to `0.6.0-alpha.1` and add two new viewers plus a dual-path Zarr renderer. No breaking changes to existing tabs, and CogViewer behavior is unchanged. - -### What's new - -- **StacMosaicViewer** (renamed from SentinelMosaicViewer, wrapped in a new `StacTabViewer`). `ViewerRouter` now detects STAC Items / FeatureCollections / Collections / Catalogs via a 256 KB adapter peek (`utils/stac.ts::classifyStac`) and mounts a tab wrapper with `Map` / `STAC Browser` / `JSON` buttons (URL hash `#map` / `#stac-browser` / `#code`, shareable). The user can always toggle back to the third-party stac-browser iframe. For Collection / Catalog inputs, `utils/stac-hydrate.ts::hydrateStacItems` walks `links[rel=item|child|next]` with a 12-way concurrency pool and a 2000-item cap, emitting progressive batches so the MosaicLayer starts rendering after ~1–2s. Each inner COG still runs through `selectCogPipeline`, so palette-indexed short-circuits, non-uint custom pipelines, LinearRescale, and `normalizeCogGeotiff` (overview strip + polar bbox clamp) all apply per scene. Shared `DecoderPool` and `createEpsgResolver` across every inner source. -- **MultiCogViewer.** STAC Item JSON routes here when `eo:bands.common_name` or MPC/Element 84/AWS asset-key heuristics identify at least the red/green/blue Sentinel-2 bands. Preset dropdown (True Color / False-Color IR / SWIR / Vegetation / Agriculture) drives the v0.6 `MultiCOGLayer.composite` prop, and a `FilterNoDataVal` + `LinearRescale` pipeline (0..0.3 default for L2A reflectance) mask scene edges and stretch contrast. -- **Zarr dual path.** `utils/zarr.ts::detectGeoZarr` inspects hierarchy attributes for the GeoZarr convention (`multiscales` + spatial + CRS). Matching stores render via `@developmentseed/deck.gl-zarr` `ZarrLayer` on `MapboxOverlay`; anything else falls through to the existing `@carbonplan/zarr-layer` path with its 10 k-tile guard and numcodecs codec aliases. -- **New utilities.** `utils/stac.ts` (STAC item/FeatureCollection shape checks, Sentinel band extraction, bbox helper). `utils/cog.ts` gains `buildMosaicSourceMeta`, `buildBandRenderPipeline` (composes `FilterNoDataVal` + `LinearRescale` in GPU-correct order). `utils/zarr.ts` gains `detectGeoZarr` and `zarrTileToImageData`. -- **CogControls `mode` prop.** Accepts `'single'` (default, full band + color-ramp UI) or `'multi'` (rescale slider only). MultiCogViewer uses the new mode; existing CogViewer is unchanged. - -### Package bumps - -`@developmentseed/deck.gl-geotiff`, `deck.gl-raster`, `geotiff`, `proj`, `epsg`: `^0.5.0 → ^0.6.0-alpha.1`. New deps: `@developmentseed/deck.gl-zarr@^0.6.0-alpha.1` (pulls in `@developmentseed/geozarr` transitively). `zarrita` bumped `^0.6.2 → ^0.7.1`, forced across the tree via `pnpm.overrides` so `@carbonplan/zarr-layer@^0.4.3` runs on the same major. - -### Patches - -`patches/@developmentseed__deck.gl-geotiff@0.5.0.patch` renamed and re-attached as `@0.6.0-alpha.1.patch`. Both hunks (proj4 `+over` antimeridian fix, `inferRenderPipeline` re-export) still apply unchanged, upstream tickets [#366](https://github.com/developmentseed/deck.gl-raster/issues/366) and [PR #374](https://github.com/developmentseed/deck.gl-raster/pull/374) remain open. - -New patch `patches/@carbonplan__zarr-layer@0.4.3.patch` replaces two calls to `zarr.tryWithConsolidated()` with `Promise.resolve(baseStore)`. The helper was removed in zarrita 0.7, and the override above forces 0.7 across the tree, which otherwise surfaced as a runtime `(void 0) is not a function` inside `_onAddAsync` when mounting the legacy ZarrLayer. Consolidated metadata (`.zmetadata`) is still fetched manually by the library's own `_loadV2`, so skipping the helper is behavior-preserving. - -### Vite config - -`optimizeDeps.include` extended with `@developmentseed/deck.gl-zarr` and its `geozarr` + `raster-reproject` leaves, plus `zarrita` itself. diff --git a/.changeset/stac-geoparquet-support.md b/.changeset/stac-geoparquet-support.md deleted file mode 100644 index 62a322b..0000000 --- a/.changeset/stac-geoparquet-support.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -'@walkthru-earth/objex': minor -'@walkthru-earth/objex-utils': minor ---- - -Add stac-geoparquet support. - -- `objex-utils`: new `stac-geoparquet` module with pure transforms that any consumer can use: `isStacGeoparquetSchema`, `stacRowToItem`, `flattenStacBbox`, `resolveStacAssetHref`, `pickStacPrimaryAsset`, plus `StacGeoparquetRow` / `StacBboxStruct` / `StacGeoparquetSchemaColumn` / `StacRowToItemOptions` types. -- `objex` Svelte lib: `ViewerRouter` detects stac-geoparquet via hyparquet schema sniff and routes matching `.parquet` / `.geoparquet` files to `StacTabViewer`. A new `query/stac-geoparquet.ts` helper uses the existing DuckDB engine (presigned URL, single worker, cancellable) to materialize a STAC FeatureCollection in one shot; `StacMosaicViewer` consumes it through the same `buildMosaicSourceMeta` + MosaicLayer path as JSON catalogs. `StacTabViewer` now shows a "Parquet" badge, relabels the last tab as "Table" (mounting `TableViewer`), and disables the `STAC Browser` iframe button with a tooltip since Radiant Earth stac-browser is JSON-only. The `stac-map` DevSeed iframe handles parquet on its own, so its button is unchanged. diff --git a/CHANGELOG.md b/CHANGELOG.md index 29cb3ba..e2362f1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,49 @@ # @walkthru-earth/objex +## 1.3.0 + +### Minor Changes + +- [`07e1570`](https://github.com/walkthru-earth/objex/commit/07e15707580ed26512ff1905848a3ef3853d99ff) Thanks [@yharby](https://github.com/yharby)! - Canonical connection identity and deduplication across every write path. + + - New `utils/connection-identity.ts` (exported from `src/lib/index.ts`): `connectionIdentityKey`, `isSameConnectionIdentity`, `normalizeEndpoint`, `normalizeProvider`, `ConnectionIdentityInput`. Identity is provider-aware: `azure` → `provider|endpoint|bucket`, `gcs` → `provider|bucket` (global namespace), `s3` with empty endpoint → `s3|bucket|region` (region is load-bearing for signing), all other S3-compatible providers → `provider|normalizedEndpoint|bucket`. `normalizeEndpoint` lowercases host, strips default ports (`:443`/`:80`) and trailing slashes, and preserves explicit non-default ports and pathnames so `http` vs `https`, `:443` vs empty, and trailing-slash drift collapse to one key. + - `connections` store: removed `findByBucketEndpoint` (bucket+endpoint string match, which produced silent duplicates for AWS same-bucket-different-region and custom S3-compat scheme/port drift, and was bypassed entirely by the manual Add Connection dialog). Every write path now dedups through `connectionIdentityKey`: + - `save(config)` returns `{ id, existed }`. On `existed: true`, the row is reused and credentials from the new config overwrite the old ones. + - `update(id, config)` throws the new `DuplicateConnectionError` when the new identity would collide with a different saved row, so edits can't silently produce phantom duplicates. + - `saveHostConnection(detected)` continues to be the auto-detect entry and returns the final id, either reused or newly inserted. + - New public `findByIdentity(input)` exposes the same key for callers that need to check without writing. + - `ConnectionDialog` surfaces both outcomes: amber "merged into existing" notice on dedup and destructive "already used by X" block on edit collision, with the offending connection's name. + - Build: svelte-check 0 errors, publint clean, no `$lib/` leaks in `dist/`. + +- [`676c792`](https://github.com/walkthru-earth/objex/commit/676c79298b3171333be8e0752002c434404dde43) Thanks [@yharby](https://github.com/yharby)! - Bump the `@developmentseed/deck.gl-geotiff` family to `0.6.0-alpha.1` and add two new viewers plus a dual-path Zarr renderer. No breaking changes to existing tabs, and CogViewer behavior is unchanged. + + ### What's new + + - **StacMosaicViewer** (renamed from SentinelMosaicViewer, wrapped in a new `StacTabViewer`). `ViewerRouter` now detects STAC Items / FeatureCollections / Collections / Catalogs via a 256 KB adapter peek (`utils/stac.ts::classifyStac`) and mounts a tab wrapper with `Map` / `STAC Browser` / `JSON` buttons (URL hash `#map` / `#stac-browser` / `#code`, shareable). The user can always toggle back to the third-party stac-browser iframe. For Collection / Catalog inputs, `utils/stac-hydrate.ts::hydrateStacItems` walks `links[rel=item|child|next]` with a 12-way concurrency pool and a 2000-item cap, emitting progressive batches so the MosaicLayer starts rendering after ~1–2s. Each inner COG still runs through `selectCogPipeline`, so palette-indexed short-circuits, non-uint custom pipelines, LinearRescale, and `normalizeCogGeotiff` (overview strip + polar bbox clamp) all apply per scene. Shared `DecoderPool` and `createEpsgResolver` across every inner source. + - **MultiCogViewer.** STAC Item JSON routes here when `eo:bands.common_name` or MPC/Element 84/AWS asset-key heuristics identify at least the red/green/blue Sentinel-2 bands. Preset dropdown (True Color / False-Color IR / SWIR / Vegetation / Agriculture) drives the v0.6 `MultiCOGLayer.composite` prop, and a `FilterNoDataVal` + `LinearRescale` pipeline (0..0.3 default for L2A reflectance) mask scene edges and stretch contrast. + - **Zarr dual path.** `utils/zarr.ts::detectGeoZarr` inspects hierarchy attributes for the GeoZarr convention (`multiscales` + spatial + CRS). Matching stores render via `@developmentseed/deck.gl-zarr` `ZarrLayer` on `MapboxOverlay`; anything else falls through to the existing `@carbonplan/zarr-layer` path with its 10 k-tile guard and numcodecs codec aliases. + - **New utilities.** `utils/stac.ts` (STAC item/FeatureCollection shape checks, Sentinel band extraction, bbox helper). `utils/cog.ts` gains `buildMosaicSourceMeta`, `buildBandRenderPipeline` (composes `FilterNoDataVal` + `LinearRescale` in GPU-correct order). `utils/zarr.ts` gains `detectGeoZarr` and `zarrTileToImageData`. + - **CogControls `mode` prop.** Accepts `'single'` (default, full band + color-ramp UI) or `'multi'` (rescale slider only). MultiCogViewer uses the new mode; existing CogViewer is unchanged. + + ### Package bumps + + `@developmentseed/deck.gl-geotiff`, `deck.gl-raster`, `geotiff`, `proj`, `epsg`: `^0.5.0 → ^0.6.0-alpha.1`. New deps: `@developmentseed/deck.gl-zarr@^0.6.0-alpha.1` (pulls in `@developmentseed/geozarr` transitively). `zarrita` bumped `^0.6.2 → ^0.7.1`, forced across the tree via `pnpm.overrides` so `@carbonplan/zarr-layer@^0.4.3` runs on the same major. + + ### Patches + + `patches/@developmentseed__deck.gl-geotiff@0.5.0.patch` renamed and re-attached as `@0.6.0-alpha.1.patch`. Both hunks (proj4 `+over` antimeridian fix, `inferRenderPipeline` re-export) still apply unchanged, upstream tickets [#366](https://github.com/developmentseed/deck.gl-raster/issues/366) and [PR #374](https://github.com/developmentseed/deck.gl-raster/pull/374) remain open. + + New patch `patches/@carbonplan__zarr-layer@0.4.3.patch` replaces two calls to `zarr.tryWithConsolidated()` with `Promise.resolve(baseStore)`. The helper was removed in zarrita 0.7, and the override above forces 0.7 across the tree, which otherwise surfaced as a runtime `(void 0) is not a function` inside `_onAddAsync` when mounting the legacy ZarrLayer. Consolidated metadata (`.zmetadata`) is still fetched manually by the library's own `_loadV2`, so skipping the helper is behavior-preserving. + + ### Vite config + + `optimizeDeps.include` extended with `@developmentseed/deck.gl-zarr` and its `geozarr` + `raster-reproject` leaves, plus `zarrita` itself. + +- [`439dfd7`](https://github.com/walkthru-earth/objex/commit/439dfd7049ad602a3871e9da6a5612e92c2e51cc) Thanks [@yharby](https://github.com/yharby)! - Add stac-geoparquet support. + + - `objex-utils`: new `stac-geoparquet` module with pure transforms that any consumer can use: `isStacGeoparquetSchema`, `stacRowToItem`, `flattenStacBbox`, `resolveStacAssetHref`, `pickStacPrimaryAsset`, plus `StacGeoparquetRow` / `StacBboxStruct` / `StacGeoparquetSchemaColumn` / `StacRowToItemOptions` types. + - `objex` Svelte lib: `ViewerRouter` detects stac-geoparquet via hyparquet schema sniff and routes matching `.parquet` / `.geoparquet` files to `StacTabViewer`. A new `query/stac-geoparquet.ts` helper uses the existing DuckDB engine (presigned URL, single worker, cancellable) to materialize a STAC FeatureCollection in one shot; `StacMosaicViewer` consumes it through the same `buildMosaicSourceMeta` + MosaicLayer path as JSON catalogs. `StacTabViewer` now shows a "Parquet" badge, relabels the last tab as "Table" (mounting `TableViewer`), and disables the `STAC Browser` iframe button with a tooltip since Radiant Earth stac-browser is JSON-only. The `stac-map` DevSeed iframe handles parquet on its own, so its button is unchanged. + ## 1.2.1 ### Patch Changes diff --git a/package.json b/package.json index bdde44e..77402d9 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@walkthru-earth/objex", - "version": "1.2.1", + "version": "1.3.0", "description": "Svelte 5 components and utilities for exploring geospatial object storage — S3, GCS, Azure, R2", "author": "Youssef Harby ", "license": "CC-BY-4.0", diff --git a/packages/objex-utils/CHANGELOG.md b/packages/objex-utils/CHANGELOG.md index 37abc22..5635a56 100644 --- a/packages/objex-utils/CHANGELOG.md +++ b/packages/objex-utils/CHANGELOG.md @@ -1,5 +1,38 @@ # @walkthru-earth/objex-utils +## 1.3.0 + +### Minor Changes + +- [`676c792`](https://github.com/walkthru-earth/objex/commit/676c79298b3171333be8e0752002c434404dde43) Thanks [@yharby](https://github.com/yharby)! - Bump the `@developmentseed/deck.gl-geotiff` family to `0.6.0-alpha.1` and add two new viewers plus a dual-path Zarr renderer. No breaking changes to existing tabs, and CogViewer behavior is unchanged. + + ### What's new + + - **StacMosaicViewer** (renamed from SentinelMosaicViewer, wrapped in a new `StacTabViewer`). `ViewerRouter` now detects STAC Items / FeatureCollections / Collections / Catalogs via a 256 KB adapter peek (`utils/stac.ts::classifyStac`) and mounts a tab wrapper with `Map` / `STAC Browser` / `JSON` buttons (URL hash `#map` / `#stac-browser` / `#code`, shareable). The user can always toggle back to the third-party stac-browser iframe. For Collection / Catalog inputs, `utils/stac-hydrate.ts::hydrateStacItems` walks `links[rel=item|child|next]` with a 12-way concurrency pool and a 2000-item cap, emitting progressive batches so the MosaicLayer starts rendering after ~1–2s. Each inner COG still runs through `selectCogPipeline`, so palette-indexed short-circuits, non-uint custom pipelines, LinearRescale, and `normalizeCogGeotiff` (overview strip + polar bbox clamp) all apply per scene. Shared `DecoderPool` and `createEpsgResolver` across every inner source. + - **MultiCogViewer.** STAC Item JSON routes here when `eo:bands.common_name` or MPC/Element 84/AWS asset-key heuristics identify at least the red/green/blue Sentinel-2 bands. Preset dropdown (True Color / False-Color IR / SWIR / Vegetation / Agriculture) drives the v0.6 `MultiCOGLayer.composite` prop, and a `FilterNoDataVal` + `LinearRescale` pipeline (0..0.3 default for L2A reflectance) mask scene edges and stretch contrast. + - **Zarr dual path.** `utils/zarr.ts::detectGeoZarr` inspects hierarchy attributes for the GeoZarr convention (`multiscales` + spatial + CRS). Matching stores render via `@developmentseed/deck.gl-zarr` `ZarrLayer` on `MapboxOverlay`; anything else falls through to the existing `@carbonplan/zarr-layer` path with its 10 k-tile guard and numcodecs codec aliases. + - **New utilities.** `utils/stac.ts` (STAC item/FeatureCollection shape checks, Sentinel band extraction, bbox helper). `utils/cog.ts` gains `buildMosaicSourceMeta`, `buildBandRenderPipeline` (composes `FilterNoDataVal` + `LinearRescale` in GPU-correct order). `utils/zarr.ts` gains `detectGeoZarr` and `zarrTileToImageData`. + - **CogControls `mode` prop.** Accepts `'single'` (default, full band + color-ramp UI) or `'multi'` (rescale slider only). MultiCogViewer uses the new mode; existing CogViewer is unchanged. + + ### Package bumps + + `@developmentseed/deck.gl-geotiff`, `deck.gl-raster`, `geotiff`, `proj`, `epsg`: `^0.5.0 → ^0.6.0-alpha.1`. New deps: `@developmentseed/deck.gl-zarr@^0.6.0-alpha.1` (pulls in `@developmentseed/geozarr` transitively). `zarrita` bumped `^0.6.2 → ^0.7.1`, forced across the tree via `pnpm.overrides` so `@carbonplan/zarr-layer@^0.4.3` runs on the same major. + + ### Patches + + `patches/@developmentseed__deck.gl-geotiff@0.5.0.patch` renamed and re-attached as `@0.6.0-alpha.1.patch`. Both hunks (proj4 `+over` antimeridian fix, `inferRenderPipeline` re-export) still apply unchanged, upstream tickets [#366](https://github.com/developmentseed/deck.gl-raster/issues/366) and [PR #374](https://github.com/developmentseed/deck.gl-raster/pull/374) remain open. + + New patch `patches/@carbonplan__zarr-layer@0.4.3.patch` replaces two calls to `zarr.tryWithConsolidated()` with `Promise.resolve(baseStore)`. The helper was removed in zarrita 0.7, and the override above forces 0.7 across the tree, which otherwise surfaced as a runtime `(void 0) is not a function` inside `_onAddAsync` when mounting the legacy ZarrLayer. Consolidated metadata (`.zmetadata`) is still fetched manually by the library's own `_loadV2`, so skipping the helper is behavior-preserving. + + ### Vite config + + `optimizeDeps.include` extended with `@developmentseed/deck.gl-zarr` and its `geozarr` + `raster-reproject` leaves, plus `zarrita` itself. + +- [`439dfd7`](https://github.com/walkthru-earth/objex/commit/439dfd7049ad602a3871e9da6a5612e92c2e51cc) Thanks [@yharby](https://github.com/yharby)! - Add stac-geoparquet support. + + - `objex-utils`: new `stac-geoparquet` module with pure transforms that any consumer can use: `isStacGeoparquetSchema`, `stacRowToItem`, `flattenStacBbox`, `resolveStacAssetHref`, `pickStacPrimaryAsset`, plus `StacGeoparquetRow` / `StacBboxStruct` / `StacGeoparquetSchemaColumn` / `StacRowToItemOptions` types. + - `objex` Svelte lib: `ViewerRouter` detects stac-geoparquet via hyparquet schema sniff and routes matching `.parquet` / `.geoparquet` files to `StacTabViewer`. A new `query/stac-geoparquet.ts` helper uses the existing DuckDB engine (presigned URL, single worker, cancellable) to materialize a STAC FeatureCollection in one shot; `StacMosaicViewer` consumes it through the same `buildMosaicSourceMeta` + MosaicLayer path as JSON catalogs. `StacTabViewer` now shows a "Parquet" badge, relabels the last tab as "Table" (mounting `TableViewer`), and disables the `STAC Browser` iframe button with a tooltip since Radiant Earth stac-browser is JSON-only. The `stac-map` DevSeed iframe handles parquet on its own, so its button is unchanged. + ## 1.2.1 ### Patch Changes diff --git a/packages/objex-utils/package.json b/packages/objex-utils/package.json index ad37a26..63b0e57 100644 --- a/packages/objex-utils/package.json +++ b/packages/objex-utils/package.json @@ -1,6 +1,6 @@ { "name": "@walkthru-earth/objex-utils", - "version": "1.2.1", + "version": "1.3.0", "description": "Pure TypeScript utilities from objex — WKB parser, GeoArrow builder, storage URL parser, file type registry", "author": "Youssef Harby ", "license": "CC-BY-4.0",