-
Notifications
You must be signed in to change notification settings - Fork 1
Feature full rules #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 50 commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
7babf71
Initial setup of rules: download data, cutout, process slope
sjpfenninger 49ad79c
Use 60m GEDTM30 for slope via COG wrapper
sjpfenninger 75d91b0
Move bathymetry download to COG; cleanup file naming
sjpfenninger b5a36c7
Use 30 arcsec GHSL
sjpfenninger 4597abf
First attempt rules for onshore
LinhHo 335ba06
Update calling script for snakemake
LinhHo 3d614cd
Resample with dictionary from config
LinhHo 21eec60
Cannot save netcdf AtttributeError
LinhHo b8671a9
Coordinates error?
LinhHo 84321a2
Update messages in rules
LinhHo 5dea4f2
Fix mixing rasterio and xarray dataset
LinhHo deace67
Get full workflow to run
sjpfenninger cc93098
Cleanup and merge scripts
sjpfenninger a2fa551
Add wind offshore
LinhHo 08e39f1
Add wind offshore
LinhHo 2c7acc7
Remove land sea mask
LinhHo bee7ac5
Add reports
sjpfenninger 25b86d2
Improve file naming consistency
sjpfenninger 9e1e131
Move more of the geo-processing into resample.py; Add summary report
sjpfenninger a95ae05
Further clean up processing steps; Report by shape
sjpfenninger 55808e1
Process bathymetry at the start; minor cleanup
sjpfenninger 8440e6e
Make shape a wildcard
sjpfenninger 98e959f
Make UTM-based buffering configurable
sjpfenninger 9b39da0
Simplify and generalise the area potential processing; Cleanup
sjpfenninger b88df0d
User must manually download WDPA
sjpfenninger c061cc5
Add integration test
sjpfenninger f061c68
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 684fa8b
Emulate `unzip` in Python
sjpfenninger aa58edf
Attempt to fix Windows tests
sjpfenninger 966e463
Add docstrings and logfiles
sjpfenninger 42c4e11
Fix savefig call
sjpfenninger 8e81fd9
Add INTERFACE.yaml
sjpfenninger 47136ee
Template updates, README cleanup, add pixi.lock
sjpfenninger 287004f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 6e91b28
Add missing pixi task update
sjpfenninger ea72ceb
Update integration test Snakefile
sjpfenninger 50b6dce
Separate download from unzip in integration test, scale down to 1 core
sjpfenninger 834ff15
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 4cb8bf6
One core only to avoid parallelism on Windows
sjpfenninger 48c520f
Improve unzip_like and remove bash-based tempdirs
sjpfenninger 4ca80ae
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] b031d79
Move shell-based raster clipping to Python script
sjpfenninger 862611c
Remove 'set -x'
sjpfenninger d36bbd0
Land cover types configurable, memory improvements
sjpfenninger 8879741
Update test config; increase figure resolution
sjpfenninger 5d04201
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] d18037d
Write CRS to TIFFs before saving
sjpfenninger 5718c8a
Allow workflow to break larger shapes into subunits; other improvements
sjpfenninger 42c3285
Improve memory use in area_potential_report rule
sjpfenninger 36b6091
Replace gdal_merge with gdalwarp, clean up report
sjpfenninger f7fca2a
Run `copier recopy`
sjpfenninger 899642e
Changes in response to review
sjpfenninger ebcca38
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 51010fa
Make tiny_files optional
sjpfenninger 5009d12
Zero out rather than NaN areas with no potential
sjpfenninger d7e054a
Add basic documentation
sjpfenninger 8fa413f
Add schema validation for shapes
sjpfenninger 5fc043c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] d4c59be
Improve schema
sjpfenninger File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,20 +1,15 @@ | ||
| # Module Input-Output structure for automated doc. generation | ||
| resources: | ||
| user: | ||
| user_message.md: "Example file emulating external files requested from a user." | ||
| "shapes/{shape}.parquet": "Region geometries in parquet format. These should conform to the schema defined in https://github.com/calliope-project/module_geo_boundaries/blob/main/workflow/internal/shape.schema.yaml" | ||
| "wdpa.gdb": "WDPA protected areas database from https://www.protectedplanet.net/ in GeoDB format (choose 'File Geodatabase' when downloading)." | ||
| automatic: | ||
| dummy_readme.md: "Example file emulating downloads determined by internal settings." | ||
| "{shape}.resampled_inputs.nc": "Resampled and rasterized input data layers for the regions, including land cover, slope, settlement, bathymetry, and protected areas." | ||
| "{shape}.resampled_inputs.png": "Visualization of the resampled input data for the regions." | ||
| results: | ||
| combined_text.md: "Example file emulating module results." | ||
|
|
||
| # Wildcard example: | ||
| # resources: | ||
| # user: | ||
| # shapes_{resolution}.geojson: region geometries. | ||
| # automatic: | ||
| # technology_data.parquet: dataset with technology characteristics. | ||
| # results: | ||
| # '{resolution}/tech_capacity.parquet': description of output data. | ||
| # '{resolution}/results_image.png': description of output image. | ||
| # wildcards: | ||
| # resolution: description of the wildcard’s purpose. | ||
| "results/{shape}/area_potential_{tech}.tif": "Area potential GeoTIFF raster for the specified technology and region geometries." | ||
| "results/{shape}/area_potential_report.csv": "CSV summary report of area potential for all techs, for the given region geometries." | ||
| "results/{shape}/area_potential_report.html": "HTML summary report of area potential for all techs, for the given region geometries." | ||
| wildcards: | ||
| shape: "Name of the shape to be processed, e.g., 'world', 'europe', 'MEX'." | ||
| tech: "Name of the technology, e.g., 'pv_rooftop' or 'wind_offshore'. Available technologies are defined in the module configuration." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,119 @@ | ||
| # A minimal example of how to configure this module | ||
| dummy_text: This is a user input. | ||
| tiny_files: False | ||
|
|
||
| # Options for buffering: either a string of the form "epsg:xxxx" or "UTM"/"utm" | ||
| # - "UTM": project each shape to the UTM zone of its centroid for buffering | ||
| # - "epsg:xxxx": use the specified CRS for all buffering | ||
| # A good option is "epsg:8857" (WGS 84 / Equal Earth Greenwich) for global coverage | ||
| buffer_crs: "epsg:8857" | ||
|
|
||
| split_by: country_id # likely country_id or shape_id | ||
|
|
||
| land_cover_types: | ||
| POST_FLOODING: FARM | ||
| RAINFED_CROPLANDS: FARM | ||
| MOSAIC_CROPLAND: FARM | ||
| MOSAIC_VEGETATION: FARM | ||
| CLOSED_TO_OPEN_BROADLEAVED_FOREST: FOREST | ||
| CLOSED_BROADLEAVED_FOREST: FOREST | ||
| OPEN_BROADLEAVED_FOREST: FOREST | ||
| CLOSED_NEEDLELEAVED_FOREST: FOREST | ||
| OPEN_NEEDLELEAVED_FOREST: FOREST | ||
| CLOSED_TO_OPEN_MIXED_FOREST: FOREST | ||
| MOSAIC_FOREST: FOREST | ||
| CLOSED_TO_OPEN_REGULARLY_FLOODED_FOREST: FOREST | ||
| CLOSED_REGULARLY_FLOODED_FOREST: FOREST | ||
| MOSAIC_GRASSLAND: OTHER | ||
| CLOSED_TO_OPEN_SHRUBLAND: OTHER | ||
| CLOSED_TO_OPEN_HERBS: OTHER | ||
| SPARSE_VEGETATION: OTHER | ||
| CLOSED_TO_OPEN_REGULARLY_FLOODED_GRASSLAND: OTHER | ||
| BARE_AREAS: OTHER | ||
| ARTIFICIAL_SURFACES_AND_URBAN_AREAS: URBAN | ||
| WATER_BODIES: WATER | ||
| PERMANENT_SNOW: NOT_SUITABLE | ||
| NO_DATA: NOT_SUITABLE | ||
|
|
||
| techs: | ||
| pv_rooftop: | ||
| initial_area: settlement_area | ||
| continuous_layers: | ||
| settlement_share: | ||
| min: 0.01 | ||
| max: 1 | ||
| share: 0.8 | ||
| binary_layers: | ||
| regions_maritime: 0 | ||
| regions_land: 1 | ||
| protected: 0 | ||
| # Include all land cover types; selection is done by settlement_share | ||
| landcover_FARM: 1 | ||
| landcover_FOREST: 1 | ||
|
sjpfenninger marked this conversation as resolved.
|
||
| landcover_URBAN: 1 | ||
| landcover_OTHER: 1 | ||
| landcover_NOT_SUITABLE: 0 | ||
| landcover_WATER: 0 | ||
| pv_open_field: | ||
| initial_area: pixel_area | ||
| continuous_layers: | ||
| slope: | ||
| min: 0 | ||
| max: 3 | ||
| settlement_share: | ||
| min: 0 | ||
| max: 0.01 | ||
| binary_layers: | ||
| regions_maritime: 0 | ||
| regions_land: 1 | ||
| protected: 0 | ||
| landcover_FARM: 0.1 | ||
| landcover_FOREST: 0 | ||
| landcover_URBAN: 0 | ||
| landcover_OTHER: 0.2 | ||
| landcover_NOT_SUITABLE: 0 | ||
| landcover_WATER: 0 | ||
| wind_onshore: | ||
| initial_area: pixel_area | ||
| continuous_layers: | ||
| slope: | ||
| min: 0 | ||
| max: 20 | ||
| settlement_share: | ||
| min: 0 | ||
| max: 0.01 | ||
| binary_layers: | ||
| regions_maritime: 0 | ||
| regions_land: 1 | ||
| protected: 0 | ||
| landcover_FARM: 0.2 | ||
| landcover_FOREST: 0.05 | ||
| landcover_URBAN: 0 | ||
| landcover_OTHER: 0.3 | ||
| landcover_NOT_SUITABLE: 0 | ||
| landcover_WATER: 0 | ||
| wind_offshore: | ||
| initial_area: pixel_area | ||
| continuous_layers: | ||
| bathymetry: | ||
| min: -50 | ||
| max: 0 | ||
| share: 0.8 | ||
| binary_layers: | ||
| regions_land: 0 | ||
| regions_maritime: 1 | ||
| protected: 0 | ||
| shapes_buffer: | ||
| land: 10000 # meters | ||
|
|
||
| # Optional: override settings for specific subunits (countries, regions, etc.) | ||
| # This allows you to set specific parameters that differ from the defaults, | ||
| # or apply settings that are not defined in the defaults (defaults = `techs` section). | ||
| # The subunit keys should match the subunit IDs used in column selected in `split_by`. | ||
| overrides: | ||
| PRT: # Inside a subunit, any setting from `techs` can be overridden | ||
| wind_offshore: | ||
| shapes_buffer: | ||
| land: 2000 # meters | ||
| NLD: | ||
| wind_offshore: | ||
| shapes_buffer: | ||
| land: 22000 # 22 km = 12 nautical miles | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.