Skip to content

[WIP] Tech/dbt add marts #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nicolasmerinian wants to merge 5 commits into
base: main
Choose a base branch
from
Open

[WIP] Tech/dbt add marts #31

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
abc86d9
tech/dbt-add-marts: created marts because marts should be exposed ins…
nicolasmerinian Feb 19, 2026
3f15816
tech/dbt-add-marts: deleted unused queries
nicolasmerinian Feb 19, 2026
29e4123
tech/dbt-add-marts: replace tables in queries by mart tables
nicolasmerinian Feb 19, 2026
413fd3e
tech/dbt-add-marts: created a file to track the tables that don't res…
nicolasmerinian Feb 19, 2026
19c6857
tech/dbt-add-marts: created a mapping so that all of the actual table…
nicolasmerinian Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'projet_cc'
version: '1.0.0'
version: '1.0.1'
config-version: 2

profile: 'projet_cc'
Expand All @@ -13,3 +13,4 @@ clean-targets: ["target", "dbt_packages"]
models:
projet_cc:
+materialized: view
marts:

Copilot AI Feb 19, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

marts: is added under models.projet_cc but has no value/config. In YAML this becomes null, and dbt model configs expect a mapping for subdirectories; this can lead to dbt parsing/runtime errors or silently not applying intended marts configuration. Either remove this key or give it an explicit config block (e.g., set +schema / +materialized, or at least {} if it’s meant as a placeholder).

Suggested change
marts:
marts: {}

Copilot uses AI. Check for mistakes.
30 changes: 30 additions & 0 deletions models/marts/_marts_models.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@

version: 2

models:
- name: mart_mensq_temperatures_sup_20deg
description: >
Table métier utilisée par le dashboard Streamlit.
Contient les mois avec température moyenne supérieure à 20°C.

columns:
- name: annee
tests:
- not_null

- name: mois
tests:
- not_null
Comment on lines +8 to +17

Copilot AI Feb 19, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The schema tests/documentation for mart_mensq_temperatures_sup_20deg don’t match the model it references. mart_mensq_temperatures_sup_20deg is a passthrough of int_mensq_temperatures_sup_20deg, which outputs ANNEE, Z_GEO, nb_stations, moy_nuits_ge_20 (no mois). As written, the mois not_null test will error because the column doesn’t exist, and the description claims the table contains “mois” data. Update the column list/tests/description to reflect the actual columns (or adjust the underlying model to produce mois if that’s the intent).

Suggested change
Contient les mois avec température moyenne supérieure à 20°C.
columns:
- name: annee
tests:
- not_null
- name: mois
tests:
- not_null
Contient, par année et zone géographique, des indicateurs sur les nuits avec température minimale supérieure ou égale à 20°C.
columns:
- name: annee
tests:
- not_null
- name: z_geo
tests:
- not_null
- name: nb_stations
tests:
- not_null
- name: moy_nuits_ge_20
tests:
- not_null

Copilot uses AI. Check for mistakes.
- name: mart_mensq_pluviometrie_sup_100mm
description: >
Table métier utilisée par le dashboard Streamlit.
Contient les mois avec pluviométrie supérieure à 100mm.

columns:
- name: annee
tests:
- not_null

- name: mois
tests:
- not_null
4 changes: 4 additions & 0 deletions models/marts/mart_mensq_pluviometrie_sup_100mm.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
--

select *
from {{ ref('int_mensq_pluviometrie_sup_100mm') }}
4 changes: 4 additions & 0 deletions models/marts/mart_mensq_temperatures_sup_20deg.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
--

select *
from {{ ref('int_mensq_temperatures_sup_20deg') }}
47 changes: 1 addition & 46 deletions src/data_layer/bigquery.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,56 +20,11 @@ def get_bq_client():
def run_query(sql: str):
client = get_bq_client()
return client.query(sql).to_dataframe()

# Exemple de fonction qui fait un truc
def get_todo1():
return run_query(f"""
WITH CTE AS (
SELECT
ANNEE AS annee,
MOIS AS mois,
-- On garde AAAAMM pour le tri ou l'affichage chronologique
AAAAMM AS date_key,

-- Somme des précipitations pour la période donnée
AVG(RR) AS Cumul_Mensuel_Pluie_Total,

-- 'NBRR', c'est le nombre de jours de pluie

-- RRAB précipitation maximale tombée en 24 heures au cours du mois (Average)
AVG(RRAB) AS Cumul_MAxi_par_mois,

-- Nombre de jours > 100mm
AVG(NBJRR100) AS Nb_Jours_Sup_100mm

FROM `cc-reunion.data_meteofrance.stg_mensq_pluviometrie`
-- Indispensable pour fusionner les données de toutes les stations par mois
GROUP BY
ANNEE,
MOIS,
AAAAMM

-- Tri par ordre chronologique (ce qui réglera aussi ton problème de tri dans le graph)
ORDER BY
AAAAMM ASC
)

SELECT *
FROM CTE
WHERE Nb_Jours_Sup_100mm > 1
""")

def get_data():
return run_query("""
SELECT ANNEE, moy_nuits_ge_20
FROM `cc-reunion.data_meteofrance.int_mensq_temperatures_sup_20deg`
ORDER BY ANNEE ASC
""")

def get_nb_moy_nuits_sup_20deg():
return run_query("""
SELECT ANNEE, AVG(moy_nuits_ge_20) as nb_moy_nuits_sup_20deg
FROM `cc-reunion.data_meteofrance.int_mensq_temperatures_sup_20deg`
FROM `cc-reunion.data_meteofrance.mart_mensq_temperatures_sup_20deg`
GROUP BY ANNEE
ORDER BY ANNEE ASC
""")
Expand Down