Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
ec5a93b
Allow overwriting docs URL; select correct default config in test (#2…
brynpickering Feb 20, 2026
a1b452f
Link release badge to documentation (#2089)
lindnemi Feb 20, 2026
7b97305
fix: don't mutate config in cluster_network (#2091)
lkstrp Feb 24, 2026
e250075
build: use /workspace as dev-env container workdir (#2095)
lkstrp Feb 24, 2026
abb2068
fix: lower the lower bound for power grouping years (#2097)
tgilon Feb 24, 2026
90b464d
chore: remove duplicated OCGT in test config files (#2094)
tgilon Feb 24, 2026
025c72f
chore: remove duplicated OCGT in test config files (#2094)
tgilon Feb 24, 2026
18f8c81
build: install pixi env persistent with auto-symlink entrypoint (#2098)
lkstrp Feb 24, 2026
b012517
refactor: use scripts path provider consistently (#2093)
tgilon Feb 25, 2026
4a3906e
feat: setup pypsa-bot github app (#2099)
lkstrp Feb 25, 2026
34204dd
fix: validator config (#2102)
lkstrp Feb 26, 2026
5f62639
Update industry reference year to 2023 (#2103)
lindnemi Feb 27, 2026
ad10771
[github-actions.ci] Update locked envs (#2110)
github-actions[bot] Mar 2, 2026
21bc4fa
chore(deps): bump actions/upload-artifact in the github-actions group…
dependabot[bot] Mar 2, 2026
4fe739d
chore(deps): bump docker/login-action in the github-actions group (#2…
dependabot[bot] Mar 6, 2026
40c8954
Update installation.rst (#2116)
jack-gregory Mar 6, 2026
4d83a9b
chore(deps): bump dorny/paths-filter in the github-actions group (#2119)
dependabot[bot] Mar 16, 2026
960ff53
doc: update the PR template (#2118)
tgilon Mar 16, 2026
3f76942
Update solve_network.py (#2107)
toniseibold Mar 16, 2026
f85d295
[github-actions.ci] Update locked envs (#2120)
github-actions[bot] Mar 16, 2026
db2e5a0
chore(deps): bump actions/create-github-app-token (#2122)
dependabot[bot] Mar 18, 2026
7ab303c
feat: improve load shedding with carrier specific options and load si…
daniel-rdt Mar 18, 2026
25f8ee4
ci: validation runs via pypsa-app (#2123)
lkstrp Mar 19, 2026
c5b534d
test: trusted user (#2125)
daniel-rdt Mar 23, 2026
192857b
fix: clamp s/p_nom_max in brownfield to prevent floating-point infeas…
thogin Mar 25, 2026
e8b2730
ci: exclude packages newer than 7 days in lockfile update workflow (#…
lkstrp Mar 30, 2026
e2620cf
fix: remove Dask client.shutdown() in build_renewable_profiles (#2131)
thogin Mar 30, 2026
cb9ebde
chore(deps): bump prefix-dev/setup-pixi in the github-actions group (…
dependabot[bot] Apr 1, 2026
c8fd89c
perf: use chunks="auto" in a single thread (#2137)
coroa Apr 8, 2026
22c92a8
[pre-commit.ci] pre-commit autoupdate (#2134)
pre-commit-ci[bot] Apr 13, 2026
abea573
fix: remove asynchronous model for distributed client (#2149)
coroa Apr 14, 2026
f5ec6e6
fix: blacklist atlite 0.5.0 (#2150)
coroa Apr 14, 2026
d03edca
paste scenarios file from last report
lindnemi Apr 15, 2026
0ce157c
remove old scenarios
lindnemi Apr 15, 2026
4587e16
oriente towards new scenario definitions
lindnemi Apr 15, 2026
91e3371
define new scenario and get it running
lindnemi Apr 15, 2026
227d3a6
rename
lindnemi Apr 15, 2026
f6d5fee
[github-actions.ci] Update locked envs (#2133)
github-actions[bot] Apr 16, 2026
e44b3f3
avoid accidental mutations of input data
lindnemi Apr 20, 2026
617b410
update to new naming
lindnemi Apr 20, 2026
0b67d1c
clean-up scenarios automated
lindnemi Apr 20, 2026
5e77f75
define a 85 percent iin 2040 scenario
lindnemi Apr 20, 2026
918debe
feat: improve config validator for scenario management (#2155)
daniel-rdt Apr 23, 2026
fda902c
Merge branch 'main' into scenarios-2026
lindnemi Apr 27, 2026
afcec04
Heuristically split existing solar capacities 50:50 between rootop an…
lindnemi Apr 28, 2026
401b07e
take the improved defaults from config
lindnemi Apr 28, 2026
3be745a
update szenario config
lindnemi Apr 29, 2026
c6c30a7
adm clustering with post discretization
lindnemi Apr 29, 2026
ea07410
Merge remote-tracking branch 'upstream/master' into merge-master-apri…
lindnemi Apr 29, 2026
2c55375
update lockfile
lindnemi Apr 29, 2026
51dece2
fix pydantic errors
lindnemi Apr 29, 2026
91bc024
Merge branch 'merge-master-april-26' into scenarios-2026
lindnemi Apr 29, 2026
025214a
revert back to focus weight clusters
lindnemi Apr 30, 2026
9053558
more generous line threshold during post discretization
lindnemi Apr 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
Before asking for a review for this PR make sure to complete the following checklist:

- [ ] Workflow with target rule `ariadne_all` completes without errors
- [ ] The logic of `export_ariadne_variables` has been adapted to the changes
- [ ] One or several figures that validate the changes in the PR have been posted as a comment
- [ ] A brief description of the changes has been added to `Changelog.md`
- [ ] The latest `main` has been merged into the PR
- [ ] The config has a new prefix of the format `YYYYMMDDdescriptive_title`
## Changes proposed in this Pull Request


## Checklist

**Required:**
- [ ] Changes are tested locally and behave as expected.
- [ ] Code and workflow changes are documented.
- [ ] A brief description of the changes has been added to `Changelog.md`.

**If applicable:**
- [ ] Changes in configuration options are reflected in `scripts/lib/validation`.
- [ ] For new data sources or versions, [these instructions](https://pypsa-eur.readthedocs.io/en/latest/data_sources.html) have been followed.
- [ ] New rules are documented in the appropriate `doc/*.rst` files.
30 changes: 30 additions & 0 deletions .github/pypsa-bot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
tasks:
sync-locks:
triggers:
- "pixi.toml"
- "pixi.lock"
stage:
- "pixi.lock"
- "envs/"

generate-config:
triggers:
- "scripts/lib/validation/config/**"
stage:
- "config/"
- "doc/"
- "*.yaml"
- "*.yml"

app:
trusted_users:
- daniel-rdt
runs:
- name: "config.validator.yaml"
configfile: "config/test/config.validator.yaml"
triggers: [main, pr]
cache:
key: "validator-{YYYY}-{MM}"
dirs: ["data"]
import_networks:
- "results/validation/networks/base_s_50___2050.nc"
4 changes: 2 additions & 2 deletions .github/workflows/push-images.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ jobs:
with:
fetch-depth: 0

- name: 'Get last commit that changed pixi env'
- name: 'Get last commit that changed pixi env or Dockerfile'
run: |
hash_last_changed=$(git log -1 --pretty=format:%H -- pixi.toml pixi.lock)
hash_last_changed=$(git log -1 --pretty=format:%H -- pixi.toml pixi.lock docker/dev-env/Dockerfile)
echo "hash_last_changed=$hash_last_changed" >> $GITHUB_ENV

- name: 'Login to GitHub Container Registry'
Expand Down
43 changes: 11 additions & 32 deletions .github/workflows/update-lockfile.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
name: Update locked envs
on:
push:
paths:
- pixi.toml
schedule:
- cron: "0 8 1,16 * *" # Bi-weekly
workflow_dispatch:

jobs:
update-locked-environment:
update-lockfiles:
if: ${{ github.ref == 'refs/heads/master' }}
name: Update lockfiles
runs-on: ubuntu-latest
Expand All @@ -23,37 +20,19 @@ jobs:
with:
pixi-version: v0.59.0

- name: Update lockfile
- name: Full resolve (exclude packages newer than 7 days)
# Exclude recently published packages to avoid pulling in broken or
# yanked releases before upstream has had time to react
run: |
EXCLUDE_DATE=$(date -u -d '7 days ago' +%Y-%m-%dT00:00:00Z)
sed -i "/^\[workspace\]/a exclude-newer = \"$EXCLUDE_DATE\"" pixi.toml
rm pixi.lock
pixi install --all
git checkout pixi.toml

- name: Create new conda lock files
run: |
pixi workspace export conda-explicit-spec -e default envs
for f in envs/*_conda_spec*; do mv "$f" "${f/_conda_spec/.pin}"; done

- name: Create new conda environment file
run: |
pixi workspace export conda-environment -e default envs/environment.yaml -n pypsa-eur

- name: Upload artifacts
uses: actions/upload-artifact@v7
with:
name: lockfiles
path: |
pixi.lock
envs/

create-pull-request:
needs: update-locked-environment
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Download all artifacts
uses: actions/download-artifact@v8
with:
name: lockfiles
- name: Export conda environment files
if: ${{ !vars.PYPSA_BOT_ID }}
run: pixi run sync-locks

- name: Create Pull Request
uses: peter-evans/create-pull-request@v8
Expand All @@ -64,5 +43,5 @@ jobs:
body: |
Automatically generated PR to update the pixi lockfile.

**Note: Do not merge without manual test execution. Either update the branch to trigger tests, or use `workflow_dispatch` to run tests manually. Unlike standard PRs, tests will not run automatically.**
**Note: Do not merge without manual test execution, unless pypsa-bot has already exported the environment files and triggered tests.**
commit-message: "Update locked environment files for all platforms"
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# SPDX-FileCopyrightText: Contributors to PyPSA-Eur <https://github.com/pypsa/pypsa-eur>
#
# SPDX-License-Identifier: CC0-1.0
exclude: "^LICENSES|^config/schema\\.json$|^config/config\\.default\\.yaml$"
exclude: "^LICENSES|^config/schema\\.default[\\.]*.*\\.json$|^config/config\\.default[\\.]*.*\\.yaml$"

ci:
autoupdate_schedule: quarterly
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,9 @@ SPDX-FileCopyrightText: Contributors to PyPSA-Eur <https://github.com/pypsa/pyps
SPDX-License-Identifier: CC-BY-4.0
-->

![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/pypsa/pypsa-eur?include_prereleases)
[![Test workflows](https://github.com/pypsa/pypsa-eur/actions/workflows/test.yaml/badge.svg)](https://github.com/pypsa/pypsa-eur/actions/workflows/test.yaml)
[![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/pypsa/pypsa-eur?include_prereleases)](https://github.com/PyPSA/pypsa-eur/releases)
[![Documentation](https://readthedocs.org/projects/pypsa-eur/badge/?version=latest)](https://pypsa-eur.readthedocs.io/en/latest/?badge=latest)
[![Test workflows](https://github.com/pypsa/pypsa-eur/actions/workflows/test.yaml/badge.svg)](https://github.com/pypsa/pypsa-eur/actions/workflows/test.yaml)
![Size](https://img.shields.io/github/repo-size/pypsa/pypsa-eur)
[![Zenodo PyPSA-Eur](https://zenodo.org/badge/DOI/10.5281/zenodo.3520874.svg)](https://doi.org/10.5281/zenodo.3520874)
[![Zenodo PyPSA-Eur-Sec](https://zenodo.org/badge/DOI/10.5281/zenodo.3938042.svg)](https://doi.org/10.5281/zenodo.3938042)
Expand Down
14 changes: 13 additions & 1 deletion Snakefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@
#
# SPDX-License-Identifier: MIT

import copy
from pathlib import Path
import yaml
import sys
from os.path import normpath, exists, join
from shutil import copyfile, move, rmtree, unpack_archive
from dotenv import load_dotenv
from snakemake.utils import min_version
from snakemake.utils import min_version, update_config

load_dotenv()

Expand All @@ -33,6 +34,17 @@ validate_config(config)

run = config["run"]
scenarios = get_scenarios(run)

for scenario_name, scenario_overrides in scenarios.items():
merged = copy.deepcopy(config)
update_config(merged, scenario_overrides)
try:
validate_config(merged)
except Exception as e:
raise ValueError(
f"Scenario '{scenario_name}' failed config validation: {e}"
) from e

RDIR = get_rdir(run)
shadow_config = get_shadow(run)

Expand Down
95 changes: 43 additions & 52 deletions config/config.de.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,13 @@

# docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#run
run:
prefix: 20260422_small_changes
prefix: 20260415_report_2026
name:
# - ExPol
- KN2045_Mix
# - KN2045_Mix_LowDecentralDiscount
# - KN2045_Mix_HighDecentralDiscount
# - KN2045_Mix_H2plantsEU
# - KN2045_Elek
# - KN2045_H2
# - KN2045_NFniedrig
# - KN2045_NFhoch
- KN2045_Bal_v5
- KN2045_Elec_plus_v5
scenarios:
enable: true
manual_file: config/scenarios.manual.yaml
manual_file: config/scenarios.report.2026.yaml
file: config/scenarios.automated.yaml
shared_resources:
policy: base #stops recalculating
Expand All @@ -35,8 +28,8 @@ pypsa-de:
general: REMIND-EU v1.1
transport: Aladin v1
industry: FORECAST v1.0
use_internal_db: false
reference_scenario: KN2045_Mix # KN2045_Bal_v4
use_internal_db: true
reference_scenario: KN2045_Bal_v5
region: Deutschland
ageb_for_mobility: true # In 2020 use AGEB data for final energy demand and KBA for vehicles
uba_for_mobility: # Available for 2025–2035; uses MWMS scenario from UBA Projektionsbericht 2025
Expand All @@ -45,7 +38,6 @@ pypsa-de:
scale_non_energy: false # Scale non-energy demand directly proportional to energy demand
enable: # Allowed values are "false" or a subset of [2025, 2030, 2035]
- 2025
- 2030
limit_cross_border_flows_ac: # relevant if only one node per country is used
2020: 0.4
2025: 0.4
Expand All @@ -64,7 +56,7 @@ scenario:
ll:
- vopt
clusters:
- 27 #current options: 27, 49
- 49 #current options: 27, 49
opts:
- ''
sector_opts:
Expand Down Expand Up @@ -206,39 +198,39 @@ clustering:
# fw["ES"] = 1
# print(fw.div(fw.sum()).subtract(5e-5).round(4).to_dict().__repr__().replace(",","\n"))
focus_weights:
# 27 nodes: 8 for Germany, 3 for Italy, 2 each for Denmark, UK and Spain, 1 per each of other 10 "Stromnachbarn"
'DE': 0.2965
'AT': 0.0370
'BE': 0.0370
'CH': 0.0370
'CZ': 0.0370
'DK': 0.0741
'FR': 0.0741
'GB': 0.0741
'LU': 0.0370
'NL': 0.0370
'NO': 0.0370
'PL': 0.0370
'SE': 0.0370
'ES': 0.0741
'IT': 0.0741
# # 27 nodes: 8 for Germany, 3 for Italy, 2 each for Denmark, UK and Spain, 1 per each of other 10 "Stromnachbarn"
# 'DE': 0.2965
# 'AT': 0.0370
# 'BE': 0.0370
# 'CH': 0.0370
# 'CZ': 0.0370
# 'DK': 0.0741
# 'FR': 0.0741
# 'GB': 0.0741
# 'LU': 0.0370
# 'NL': 0.0370
# 'NO': 0.0370
# 'PL': 0.0370
# 'SE': 0.0370
# 'ES': 0.0741
# 'IT': 0.0741
# high spatial resolution: change clusters to 49
# 49 nodes: 30 for Germany, 3 for Italy, 2 each for Denmark, UK and Spain, 1 per each of other 10 "Stromnachbarn"
# 'DE': 0.6124
# 'AT': 0.0204
# 'BE': 0.0204
# 'CH': 0.0204
# 'CZ': 0.0204
# 'DK': 0.0408
# 'FR': 0.0408
# 'GB': 0.0408
# 'LU': 0.0204
# 'NL': 0.0204
# 'NO': 0.0204
# 'PL': 0.0204
# 'SE': 0.0204
# 'ES': 0.0408
# 'IT': 0.0408
'DE': 0.6124
'AT': 0.0204
'BE': 0.0204
'CH': 0.0204
'CZ': 0.0204
'DK': 0.0408
'FR': 0.0408
'GB': 0.0408
'LU': 0.0204
'NL': 0.0204
'NO': 0.0204
'PL': 0.0204
'SE': 0.0204
'ES': 0.0408
'IT': 0.0408
temporal:
resolution_sector: 365H

Expand All @@ -247,8 +239,8 @@ co2_budget:
2020: 0.720 # average emissions of 2019 to 2021 relative to 1990, excl LULUCF, EEA data, European Environment Agency. (2023a). Annual European Union greenhouse gas inventory 1990–2021 and inventory report 2023 - CRF Table. https://unfccc.int/documents/627830
2025: 0.648 # With additional measures (WAM) projection, CO2 excl LULUCF, European Environment Agency. (2023e). Member States’ greenhouse gas (GHG) emission projections 2023. https://www.eea.europa.eu/en/datahub/datahubitem-view/4b8d94a4-aed7-4e67-a54c-0623a50f48e8
2030: 0.450 # 55% reduction by 2030 (Ff55)
2035: 0.250
2040: 0.100 # 90% by 2040
2035: 0.300 # ~middle of 66.25% to 72.5%. https://www.consilium.europa.eu/en/press/press-releases/2025/11/05/paris-agreement-the-eu-submits-its-updated-ndc-with-an-indicative-target-for-2035-to-the-un-ahead-of-cop30/
2040: 0.150 # 85% reduction by 2040 + 5% offsets (not modelled, but assuming cheaper than domestic reducitons)
2045: 0.050
2050: 0.000 # climate-neutral by 2050

Expand Down Expand Up @@ -425,14 +417,13 @@ solving:
transmission_losses: true
custom_extra_functionality: "../scripts/pypsa-de/additional_functionality.py"
assign_all_duals: true
load_shedding: false
skip_iterations: true # settings for post-discretization: false
skip_iterations: false # settings for post-discretization: false
min_iterations: 1 # settings for post-discretization: 1
max_iterations: 1 # settings for post-discretization: 1
post_discretization:
enable: false
enable: true
line_unit_size: 1698
line_threshold: 0.3
line_threshold: 0.25
link_unit_size:
DC: 1000
gas pipeline: 1500
Expand Down
15 changes: 12 additions & 3 deletions config/config.default.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ electricity:
# docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#atlite
atlite:
default_cutout: "europe-2013-sarah3-era5"
nprocesses: 16
nprocesses: 1
show_progress: false
plot_availability_matrix: false
cutouts:
Expand Down Expand Up @@ -582,7 +582,7 @@ solar_thermal:
# docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#existing_capacities
existing_capacities:
grouping_years_power:
- 1920
- 1900
- 1950
- 1955
- 1960
Expand Down Expand Up @@ -1147,7 +1147,16 @@ adjustments:
solving:
options:
clip_p_max_pu: 0.01
load_shedding: false
load_shedding:
enable: false
default_cost: 100000
all_carriers: true
carriers: {}
load_sinks:
enable: false
default_cost: 100000
all_carriers: false
carriers: {}
curtailment_mode: false
noisy_costs: true
skip_iterations: true
Expand Down
3 changes: 2 additions & 1 deletion config/examples/config.validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ clustering:

solving:
options:
load_shedding: true
load_shedding:
enable: true
rolling_horizon: false
horizon: 1000
overlap: 48
Loading