Authentification + améliorations

.github/workflows/deploy-docs.yml

@@ -0,0 +1,55 @@
+name: Deploy Documentation
+
+on:
+  push:
+    branches:
+      - main
+
+# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow only one concurrent deployment
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install uv
+          uv sync --all-extras
+          pip install -e .
+
+      - name: Build documentation
+        run: uv run mkdocs build
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: ./site
+
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

README.md

@@ -1,65 +1,91 @@
 # datEAUbase
 
-This repo contains the reference implementation of the dat*EAU*base relational database and data model for use in water resource recovery facilities (WRRFs). The purpose of dat*EAU*base is to allow WRRF data to be stored *along with their context* to ensure that they are correctly interpreted in data mining, modelling and decision support activities.
+This repo contains the reference implementation of the dat*EAU*base relational database and data model for use in water resource recovery facilities (WRRFs). The purpose of dat*EAU*base is to allow WRRF data to be stored *along with their context* to ensure that they are correctly interpreted in data mining, modelling and decision support activities. The data model is described in [Plana et al. (2019)](https://iwaponline.com/wqrj/article/54/1/1/64706/Towards-a-water-quality-database-for-raw-and).
 
-## Repo contents
+## Quick Start (Docker)
 
-- `/sql_generation_scripts`: This folder contains the SQL scripts that generate the tables, key constraints and fields defined by the dat*EAU*base data model. The scripts are versioned starting at v0 which corresponds to the version published in [Plana et al. (2018)](https://iwaponline.com/wqrj/article/54/1/1/64706/Towards-a-water-quality-database-for-raw-and). The scripts are refferred to as *"as designed"* (e.g., coming from a documented specification) or *"as built"* (e.g., reflecting the state of the pilEAUte's datEAUbase instance at a given date.) The scripts also vary based on the target database environment (e.g., MSSQL, MYSQL, etc.)
+Run the full stack (database + API + web app) with a single command:
 
-- `/docs`: The folder containing the source files for the official documentation of the datEAUbase project. The files found in `/docs` are processed by the python package `mkdocs` to build the documentation site. 
+```bash
+# 1. Copy and edit env file for Docker
+cp .env.docker.example .env.docker
+# Edit .env.docker: set DB_PASSWORD to a strong password
 
-- `/tests`: Contains a battery of tests to ensure that the documentation generation code responsible for creating the documentation website and the SQL generation scripts are working adequately.
+# 2. Start the database
+docker-compose up -d db
 
-- `/src`: The folder that contains the "source of truth" document for the datEAUbase: the dictionary table. The documentation and SQL statements are generated from the data encoded in this table.
+# 3. Initialize the schema (first time only)
+docker-compose --profile init up db-init
 
-## How to use this repo
+# 4. Start the API and web app
+docker-compose up -d api app
 
-### Install `uv`
+# 5. Open the app
+# Web app:  http://localhost:8501
+# API docs: http://localhost:8000/docs
+```
 
-`uv` is a tool for managing dependencies, virtual encironments and package distribution for Python. Install it with the following commands:
+## Local Development
 
-**macOS and Linux:**
+### Prerequisites
 
-```bash
-curl -LsSf https://astral.sh/uv/install.sh | sh
-```
+- Python 3.12+
+- [uv](https://docs.astral.sh/uv/) — fast Python package manager
+- Docker (for the MSSQL database)
+- ODBC Driver 18 for SQL Server ([install guide](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server))
 
-**Windows:**
+### Setup
 
-```powershell
-powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
-```
+```bash
+# Install all dependencies (API + app + dev tools)
+uv sync --extra api --extra app --extra dev
 
-Alternatively, you can use package managers:
+# Copy and configure your env file
+cp .env.example .env.local
+# Edit .env.local: set DB_HOST=localhost, DB_PORT=14330, DB_NAME=..., etc.
 
-- macOS: `brew install uv`
-- Windows: `winget install --id=astral-sh.uv -e`
+# Start the database container
+docker-compose up -d db
 
-### Install locally for development
+# Initialize the schema (first time only)
+docker-compose --profile init up db-init
 
-```uv sync .```
+# Start the API server
+uv run uvicorn api.main:app --reload
 
-```uv pip install -e .```
+# In a new terminal, start the Streamlit app
+uv run streamlit run app/Home.py
+```
+
+The app will be available at <http://localhost:8501> and the API at <http://localhost:8000/docs>.
 
-To install with development dependencies:
+### Run Tests
 
 ```bash
-uv sync --all-extras
+uv run pytest                    # all tests
+uv run pytest tests/unit/        # unit tests only (no DB required)
+uv run pytest -m db              # integration tests (requires running DB)
 ```
 
-or
+### Build the Docs
 
 ```bash
-uv pip install -e ".[dev]"
+uv run mkdocs serve   # local preview at http://localhost:8000
+uv run mkdocs build   # build static site to site/
 ```
 
-### Run the tests
-
-```uv run pytest```
+## Repository Structure
 
-### Build the docs website and serve it locally
-
-```uv run mkdocs serve```
+- `api/` — FastAPI REST API (endpoints, services, repositories, schemas)
+- `app/` — Streamlit multipage web application
+- `sql/` — Database initialization and seed SQL scripts
+- `migrations/` — Schema migration scripts (v1.0.0 → v2.1.0)
+- `schema_dictionary/` — YAML definitions for all tables and columns
+- `src/open_dateaubase/` — Core Python library (data models, lineage, processing)
+- `tests/` — Unit, API contract, and integration tests
+- `docs/` — MkDocs documentation source
+- `sql_generation_scripts/` — Versioned CREATE TABLE scripts
 
 ## License
-dat*EAU*base is published under the MIT license.
+
+dat*EAU*base is published under the CC-BY 4.0 license.

api/config.py

@@ -0,0 +1,38 @@
+"""API configuration loaded from environment variables."""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+from dotenv import load_dotenv
+
+_env_local = Path(__file__).parent.parent / ".env.local"
+_env = Path(__file__).parent.parent / ".env"
+if _env_local.exists():
+    load_dotenv(_env_local)
+else:
+    load_dotenv(_env)
+
+
+class Settings:
+    db_host: str = os.getenv("DB_HOST", "")
+    db_port: int = int(os.getenv("DB_PORT", "1433"))
+    db_name: str = os.getenv("DB_NAME", "")
+    db_user: str = os.getenv("DB_USER", "")
+    db_password: str = os.getenv("DB_PASSWORD", "")
+    db_driver: str = os.getenv("DB_DRIVER", "ODBC Driver 18 for SQL Server")
+
+    api_title: str = "open_datEAUbase API"
+    api_description: str = (
+        "REST API for the open_datEAUbase water quality database. "
+        "Provides access to time series data, metadata, campaigns, equipment, "
+        "ingestion routes, and processing lineage."
+    )
+    api_version: str = "1.0.0"
+    schema_version: str = "2.1.1"
+
+    upload_dir: str = "./uploads/images"
+
+
+settings = Settings()

api/database.py

@@ -0,0 +1,53 @@
+"""Database connection management for the open_datEAUbase API."""
+
+from __future__ import annotations
+
+import pyodbc
+from fastapi import HTTPException
+
+from .config import settings
+
+
+def get_connection() -> pyodbc.Connection:
+    if not all([settings.db_host, settings.db_name, settings.db_user, settings.db_password]):
+        missing = [
+            k
+            for k, v in {
+                "DB_HOST": settings.db_host,
+                "DB_NAME": settings.db_name,
+                "DB_USER": settings.db_user,
+                "DB_PASSWORD": settings.db_password,
+            }.items()
+            if not v
+        ]
+        raise HTTPException(
+            status_code=503,
+            detail=f"Database not configured. Missing env vars: {', '.join(missing)}",
+        )
+
+    conn_str = (
+        f"DRIVER={{{settings.db_driver}}};"
+        f"SERVER={settings.db_host},{settings.db_port};"
+        f"DATABASE={settings.db_name};"
+        f"UID={settings.db_user};"
+        f"PWD={settings.db_password};"
+        "Encrypt=no;"
+        "TrustServerCertificate=yes;"
+        "Connection Timeout=10;"
+    )
+    try:
+        return pyodbc.connect(conn_str)
+    except pyodbc.Error as exc:
+        raise HTTPException(
+            status_code=503,
+            detail=f"Cannot connect to database: {exc}",
+        ) from exc
+
+
+def get_db():
+    """FastAPI dependency that yields a connection and closes it after the request."""
+    conn = get_connection()
+    try:
+        yield conn
+    finally:
+        conn.close()

api/main.py

@@ -0,0 +1,33 @@
+"""FastAPI application entry point for the open_datEAUbase API."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from fastapi import FastAPI
+
+from .config import settings
+from .v1.router import router as v1_router
+
+Path(settings.upload_dir).mkdir(parents=True, exist_ok=True)
+
+app = FastAPI(
+    title=settings.api_title,
+    description=settings.api_description,
+    version=settings.api_version,
+    docs_url="/docs",
+    redoc_url="/redoc",
+    openapi_url="/openapi.json",
+)
+
+app.include_router(v1_router, prefix="/api/v1")
+
+
+@app.get("/", tags=["root"])
+def root():
+    return {
+        "message": f"{settings.api_title} is running.",
+        "docs": "/docs",
+        "health": "/api/v1/health",
+        "schema_version": settings.schema_version,
+    }

api/v1/dependencies.py

@@ -0,0 +1,20 @@
+"""Shared FastAPI dependencies.
+
+Import ``audit_repo`` and ``current_user`` from here so future endpoints
+don't need to duplicate the dependency wiring.
+"""
+
+from __future__ import annotations
+
+from fastapi import Depends
+
+from api.database import get_db
+from .endpoints.auth import get_current_user
+from .repositories.audit_repository import AuditRepository
+
+
+def get_audit_repo(conn=Depends(get_db)) -> AuditRepository:
+    return AuditRepository(conn)
+
+
+__all__ = ["get_audit_repo", "get_current_user"]

api/v1/endpoints/audit.py

@@ -0,0 +1,44 @@
+"""Audit log API endpoints."""
+
+from __future__ import annotations
+
+from datetime import datetime
+from typing import Optional
+
+from fastapi import APIRouter, Depends, Query
+
+from api.database import get_db
+from ..endpoints.auth import get_current_user
+from ..repositories.audit_repository import AuditRepository
+from ..schemas.audit import AuditLogEntry, AuditLogPage
+
+router = APIRouter()
+
+
+def get_audit_repo(conn=Depends(get_db)) -> AuditRepository:
+    return AuditRepository(conn)
+
+
+@router.get("/logs", response_model=AuditLogPage)
+def list_audit_logs(
+    user_id: Optional[int] = Query(None, description="Filter by user ID"),
+    action: Optional[str] = Query(None, description="Filter by action (CREATE, UPDATE, DELETE, LOGIN, SIGNUP)"),
+    resource_type: Optional[str] = Query(None, description="Filter by resource type"),
+    from_dt: Optional[datetime] = Query(None, description="Start of time range (ISO 8601)"),
+    to_dt: Optional[datetime] = Query(None, description="End of time range (ISO 8601)"),
+    limit: int = Query(50, ge=1, le=500),
+    offset: int = Query(0, ge=0),
+    _current_user=Depends(get_current_user),
+    audit_repo: AuditRepository = Depends(get_audit_repo),
+):
+    """Return a paginated list of audit log entries. Requires authentication."""
+    filters = dict(
+        user_id=user_id,
+        action=action,
+        resource_type=resource_type,
+        from_dt=from_dt,
+        to_dt=to_dt,
+    )
+    items = audit_repo.list_logs(limit=limit, offset=offset, **filters)
+    total = audit_repo.count_logs(**filters)
+    return AuditLogPage(total=total, limit=limit, offset=offset, items=items)