Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,21 @@

# Changelog

## [v0.3.2] – 2026-07-06

### Fixed
- **FITS cutout WCS re-tangenting bug**: cutout WCS now reproduces the parent tile mapping exactly — `CRVAL`, `CTYPE` and the CD/PC orientation are inherited from the parent unchanged and only `CRPIX` is shifted to the extraction origin — instead of re-tangenting the projection at each source. The previous approach left the cutout frame rotated by the meridian convergence between tile centre and source, producing a positional error that was ~0 at the cutout centre and grew toward the edges (~1″ at a few-arcmin FOV for sources far from the tile centre, worse near high \|Dec\| / tile corners). Only the FITS WCS header was affected; pixel data and Zarr/streaming outputs were not

### Added
- **`UNIT` and `CONSVFLX` FITS header keywords** on individual cutout outputs: `UNIT` records the pixel unit (`OriginalUnit`, `Jy`, `approx Jy`, or `approx OriginalUnit`) depending on whether flux conversion and flux-conserved resizing were applied, and `CONSVFLX` records whether flux-conserved resizing was used

### Changed
- **UI "Raw cutout" checkbox** relabelled to "Raw cutout (in Jy):" to make the output unit explicit
- **`fitsbolt` pinned to `==0.3.0`** (from `==0.2.0`)

### Documentation
- **README**: added a Flux Conversion section (pixels converted to Jansky by default via the `MAGZERO` keyword, disable with `config.apply_flux_conversion = False`), documented the `UNIT` / `CONSVFLX` header keywords, and clarified the "Raw cutouts" terminology

## [v0.3.1] – 2026-06-04

### Fixed
Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
![Cutana Demo](assets/cutana_demo_2x.gif)

> **Note:** Cutana is currently optimised for **Euclid Q1/IDR1 data**. Some defaults and assumptions are Euclid-specific:
> - Flux conversion expects the `MAGZERO` header keyword (configurable via `config.flux_conversion_keywords.AB_zeropoint`)
> - Flux conversion expects the `MAGZERO` header keyword (configurable via `config.flux_conversion_keywords.AB_zeropoint`) to convert to Jy
> - Filter detection patterns are tuned for Euclid bands (VIS, NIR-Y, NIR-H, NIR-J)
> - FITS structure assumes one file per channel/filter
>
Expand Down Expand Up @@ -100,8 +100,11 @@ TILE_102018666_12346,45.124,12.457,256,"['/path/to/tile_vis.fits','/path/to/tile

**ZARR Format** (recommended): All cutouts stored in a efficient archives, ideal for large datasets and analysis workflows. Cutana uses the [Zarr format](https://zarr.readthedocs.io/en/stable/) for high-performance storage and the [images_to_zarr](https://github.com/gomezzz/images_to_zarr/) library for conversion. (See the Output section below for sample code to access)

**FITS Format**: Individual FITS files per source, best for compatibility with existing astronomical software. Mandatory format for `do_only_cutout_extraction`,
which skips all processing aside from the flux converison, which can be disabled.
**FITS Format**: Individual FITS files per source, best for compatibility with existing astronomical software. Mandatory format for `do_only_cutout_extraction` (Raw cutouts),
which skips all processing aside from the flux conversion, which can be disabled.

### Flux Conversion
To align cutout units, pixels are converted to Janskys by default using the `MAGZERO` header keyword (configurable via `config.flux_conversion_keywords.AB_zeropoint`). Disable this flux conversion via `config.apply_flux_conversion = False`.

## WCS (World Coordinate System) Handling

Expand All @@ -112,6 +115,8 @@ FITS cutouts **preserve full WCS information** with accurate astrometric calibra
- **Reference coordinate centering**: WCS reference pixel (`CRPIX`) is set to the cutout center, with reference coordinates (`CRVAL`) pointing to the source position
- **Format compatibility**: Supports CD matrix, CDELT, and PC+CDELT WCS formats from original FITS files
- **Sky area preservation**: Total sky coverage remains constant while pixel scale adjusts for resize operations
- The header denotes the image unit (`UNIT`), `OriginalUnit`,`Jy`,`approx OriginalUnit`, `approx Jy`.
- `CONSVFLX` denotes if flux conserved resizing was applied. If not units are "approx"

### Zarr Output
**Important**: Zarr archives **do not contain WCS information**. The WCS data is not recorded in the image metadata stored within the Zarr files. The central point and image size (in pixels or arcseconds, depending on what is provided) is recorded.
Expand Down
2 changes: 1 addition & 1 deletion cutana/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
# Application entry points (Orchestrator, UI) will enable logging when needed
logger.disable("cutana")

__version__ = "0.3.1"
__version__ = "0.3.2"
__author__ = "ESA Datalabs"

# Import main classes for easy access
Expand Down
69 changes: 51 additions & 18 deletions cutana/cutout_extraction.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,17 @@ def extract_cutouts_vectorized_from_extension(
padding_factor: Factor to scale the extraction area (1.0 = no padding)

Returns:
Tuple of (cutout_list, success_mask, pixel_offset_x, pixel_offset_y) where:
Tuple of (cutout_list, success_mask, pixel_offset_x, pixel_offset_y,
origin_x, origin_y) where:
- cutout_list: List of cutout arrays (or None for failures)
- success_mask: Boolean array indicating successful extractions
- pixel_offset_x: Array of sub-pixel X offsets (positive = target toward right)
- pixel_offset_y: Array of sub-pixel Y offsets (positive = target toward top)
- origin_x: Array of 0-based parent-pixel X origins of cutout pixel 0 (the
clip- and centre-pad-corrected window start). Pre-resize cutout pixel ``p``
maps to parent pixel ``origin + p``; the FITS writer uses this to build the
cutout WCS without recomputing the (already-vectorised) extraction geometry.
- origin_y: As ``origin_x`` for the Y axis.
"""
n_sources = len(ra_array)
logger.debug(f"Starting vectorized cutout extraction for {n_sources} sources")
Expand All @@ -125,8 +131,10 @@ def extract_cutouts_vectorized_from_extension(
return (
[None] * n_sources,
np.zeros(n_sources, dtype=bool),
np.zeros(n_sources, dtype=np.float64),
np.zeros(n_sources, dtype=np.float64),
np.zeros(n_sources, dtype=np.float64), # sub-pixel offset x
np.zeros(n_sources, dtype=np.float64), # sub-pixel offset y
np.zeros(n_sources, dtype=np.int32), # integer origin x
np.zeros(n_sources, dtype=np.int32), # integer origin y
)

img_height, img_width = image_data.shape
Expand All @@ -147,8 +155,10 @@ def extract_cutouts_vectorized_from_extension(
return (
[None] * n_sources,
np.zeros(n_sources, dtype=bool),
np.zeros(n_sources, dtype=np.float64),
np.zeros(n_sources, dtype=np.float64),
np.zeros(n_sources, dtype=np.float64), # sub-pixel offset x
np.zeros(n_sources, dtype=np.float64), # sub-pixel offset y
np.zeros(n_sources, dtype=np.int32), # integer origin x
np.zeros(n_sources, dtype=np.int32), # integer origin y
)

# Step 3: Vectorized bound computation
Expand Down Expand Up @@ -191,6 +201,13 @@ def extract_cutouts_vectorized_from_extension(
y_mins_clipped = np.maximum(0, y_mins)
y_maxs_clipped = np.minimum(img_height, y_maxs)

# Parent-pixel origin of cutout pixel 0 (0-based, integer). For an on-tile window
# this is the clipped window start; edge-clipped windows are centre-padded below,
# which shifts the origin left/down by the integer pad offset. Threaded out so the
# FITS writer can build the cutout WCS without recomputing this geometry.
origin_x_array = x_mins_clipped.astype(np.int32)
origin_y_array = y_mins_clipped.astype(np.int32)

# Check for valid regions (vectorized)
valid_mask = (x_maxs_clipped > x_mins_clipped) & (y_maxs_clipped > y_mins_clipped)

Expand Down Expand Up @@ -263,6 +280,11 @@ def extract_cutouts_vectorized_from_extension(
pixel_offset_x[i] -= pad_x_start
pixel_offset_y[i] -= pad_y_start

# The data now starts pad pixels into the window, so cutout pixel 0
# maps to a parent pixel pad_start before the clipped window start.
origin_x_array[i] -= pad_x_start
origin_y_array[i] -= pad_y_start

raw_cutout = padded_extraction

# apply flux conversion here
Expand Down Expand Up @@ -292,7 +314,7 @@ def extract_cutouts_vectorized_from_extension(
successful_count = np.sum(success_mask)
logger.debug(f"Vectorized extraction completed: {successful_count}/{n_sources} successful")

return cutouts, success_mask, pixel_offset_x, pixel_offset_y
return cutouts, success_mask, pixel_offset_x, pixel_offset_y, origin_x_array, origin_y_array


def extract_cutouts_batch_vectorized(
Expand Down Expand Up @@ -382,17 +404,22 @@ def extract_cutouts_batch_vectorized(
logger.debug(f"Processing extension {ext_name} for {n_sources} sources")

# Extract cutouts for all sources in this extension using vectorized method
cutout_list, success_mask, offset_x_array, offset_y_array = (
extract_cutouts_vectorized_from_extension(
hdul[ext_name],
wcs_dict[ext_name],
ra_array,
dec_array,
size_pixels_array,
source_ids,
padding_factor,
config,
)
(
cutout_list,
success_mask,
offset_x_array,
offset_y_array,
origin_x_array,
origin_y_array,
) = extract_cutouts_vectorized_from_extension(
hdul[ext_name],
wcs_dict[ext_name],
ra_array,
dec_array,
size_pixels_array,
source_ids,
padding_factor,
config,
)

# Organize results by source ID
Expand All @@ -401,10 +428,16 @@ def extract_cutouts_batch_vectorized(
if source_id not in combined_cutouts:
combined_cutouts[source_id] = {}
combined_wcs[source_id] = {}
# Store pixel offsets (same for all extensions since coords are the same)
# Store per-source geometry (same for all extensions since the
# coordinates and window are identical): the sub-pixel offsets and
# the integer extraction origin/size the FITS writer needs to build
# the cutout WCS without recomputing world_to_pixel and the bounds.
combined_offsets[source_id] = {
"x": float(offset_x_array[i]),
"y": float(offset_y_array[i]),
"origin_x": int(origin_x_array[i]),
"origin_y": int(origin_y_array[i]),
"extraction_size": int(size_pixels_array[i] * padding_factor),
}

combined_cutouts[source_id][ext_name] = cutout
Expand Down
9 changes: 9 additions & 0 deletions cutana/cutout_process_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,12 @@ def _process_sources_batch_vectorized_with_fits_set(
source_offsets = all_source_offsets.get(source_id, {"x": 0.0, "y": 0.0})
extraction_offset_x = source_offsets.get("x", 0.0)
extraction_offset_y = source_offsets.get("y", 0.0)
# Integer extraction origin/size (parent-tile pixels, pre-resize), computed
# vectorised at extraction time. Passed through so the FITS writer can build
# the cutout WCS without recomputing world_to_pixel and the window bounds.
extraction_origin_x = source_offsets.get("origin_x")
extraction_origin_y = source_offsets.get("origin_y")
extraction_size = source_offsets.get("extraction_size")

# When resizing is applied, scale offsets and pixel scale by the same
# resize_factor so both stay consistent with the final output coords.
Expand Down Expand Up @@ -541,6 +547,9 @@ def _process_sources_batch_vectorized_with_fits_set(
"processing_timestamp": batch_timestamp,
"rescaled_offset_x": rescaled_offset_x,
"rescaled_offset_y": rescaled_offset_y,
"extraction_origin_x": extraction_origin_x,
"extraction_origin_y": extraction_origin_y,
"extraction_size": extraction_size,
}
)
wcs_list.append(all_source_wcs.get(source_id, {}))
Expand Down
Loading
Loading