Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Base dos Dados backend — a Django + PostgreSQL API serving Brazil's largest public data platform. Exposes a GraphQL API (Graphene-Django) and a chatbot REST API (DRF + LangChain/LangGraph). Background tasks via Huey+Redis; full-text search via Elasticsearch+Haystack; payments via Stripe/dj-stripe.

## Commands

```bash
# Install
make install # poetry install
make install_precommit # install pre-commit hooks

# Development
make run_local # makemigrations + migrate + runserver 0.0.0.0:8080
make run_docker # docker-compose up --build --force-recreate --detach

# Database
make migrations # python manage.py makemigrations
make migrate # python manage.py migrate
make loadfixture # python manage.py loadfixture fixture.json
make superuser

# Lint
make lint # poetry run lint (Ruff check + format)

# Tests
poetry run pytest # all tests
poetry run pytest backend/apps/api/v1/tests/ # specific directory
poetry run pytest -k "test_name" # single test by name

# Docker helpers
make shell_docker # bash into api container
make logs_docker # tail logs
make stop_docker / make clean_docker
```

## Architecture

### Settings
- `backend/settings/base.py` — shared config, imported by all envs
- `backend/settings/local.py` — local dev (PostgreSQL via env vars, SMTP)
- `backend/settings/remote.py` — production/staging
- `pytest.ini` uses `DJANGO_SETTINGS_MODULE=backend.settings` (resolves to `base.py`)

### Django Apps (`backend/apps/`)

| App | Purpose |
|-----|---------|
| `account` | Custom user model (`Account`), roles (`BDRole`/`BDGroup`), registration tokens |
| `account_auth` | JWT authentication and authorization rules |
| `account_payment` | Stripe/dj-stripe integration |
| `api/v1` | Core data catalog: Dataset, Table, Column, ObservationLevel, Coverage, Area, Entity, etc. |
| `core` | Shared models (Metadata, TaskExecution), utility management commands |
| `chatbot` | Stub app; actual chatbot logic lives in the separate `chatbot/` package |
| `user_notifications` | Table-update notification subscriptions |

### GraphQL Auto-Generation

The GraphQL schema is **code-generated** from Django models via `backend/custom/graphql_auto.py`.

To expose a model in GraphQL:
1. Set `graphql_visible = True` on the model class (from `BaseModel`).
2. Control field exposure with `graphql_fields_whitelist`/`graphql_fields_blacklist`.
3. Control filtering with `graphql_filter_fields_whitelist` / `graphql_nested_filter_fields_whitelist`.
4. Set `graphql_query_decorator` (default: `anyone_required`) and `graphql_mutation_decorator` (default: `staff_member_required`) for auth.

The master schema is assembled in `backend/apps/schema.py`:
```python
schema = build_schema(
applications=["account", "v1"],
extra_queries=[APIQuery, PaymentQuery, UserNotificationQuery],
extra_mutations=[AccountMutation, APIMutation, PaymentMutation, ...],
)
```

Manual queries/mutations (not auto-generated) live in each app's `graphql.py`.

### BaseModel (`backend/custom/model.py`)

All domain models inherit from `BaseModel` (abstract). Key attributes:
- UUID primary key, `created_at`/`updated_at` timestamps
- GraphQL visibility and field filtering class attributes
- `admin_url` property

### Key Patterns

- **M2M mutations**: use `commit=False` + `save_m2m()` to ensure M2M fields are saved (see recent commit history).
- **Ordered models**: `api/v1` uses `django-ordered-model` for column/table ordering; management commands `reorder_columns` and `reorder_tables` handle bulk reordering.
- **Model translation**: `django-modeltranslation` provides pt/en/es translations on selected fields.
- **Search**: Haystack + custom `AsciifoldingElasticSearchEngine`; indexes auto-update via signals.
- **Fixtures**: `loadfixture`/`dumpfixture` management commands wrap Django's fixture system with chunking support (`fixtures_chunks/`).

### GraphQL Endpoint

`/graphql/` — authenticated via `Bearer <JWT>` header. JWT obtained via `ObtainJSONWebToken` mutation.
64 changes: 62 additions & 2 deletions backend/apps/api/v1/graphql.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# -*- coding: utf-8 -*-

from graphene import UUID, Boolean, Float, List, ObjectType, String
from graphene import UUID, Boolean, Float, List, Mutation, ObjectType, String
from graphene_django import DjangoObjectType

from backend.apps.api.v1.models import Table, TableNeighbor
from backend.apps.api.v1.models import Column, ObservationLevel, Table, TableNeighbor
from backend.apps.api.v1.sql_generator import OneBigTableQueryGenerator
from backend.custom.graphql_base import PlainTextNode

Expand Down Expand Up @@ -39,6 +39,42 @@ def resolve_score(root, info):
return root.score


class ReorderTables(Mutation):
"""Set display order for tables within a dataset.
ids: ordered list of table UUIDs (index 0 = first)."""

class Arguments:
ids = List(UUID, required=True)

ok = Boolean()
errors = List(String)

def mutate(root, info, ids):
if not info.context.user.is_staff:
return ReorderTables(ok=False, errors=["Permission denied"])
for i, table_id in enumerate(ids):
Table.objects.filter(pk=table_id).update(order=i)
return ReorderTables(ok=True, errors=[])


class ReorderObservationLevels(Mutation):
"""Set display order for observation levels within a table.
ids: ordered list of ObservationLevel UUIDs (index 0 = first)."""

class Arguments:
ids = List(UUID, required=True)

ok = Boolean()
errors = List(String)

def mutate(root, info, ids):
if not info.context.user.is_staff:
return ReorderObservationLevels(ok=False, errors=["Permission denied"])
for i, ol_id in enumerate(ids):
ObservationLevel.objects.filter(pk=ol_id).update(order=i)
return ReorderObservationLevels(ok=True, errors=[])


class Query(ObjectType):
get_table_neighbor = List(
TableNeighborNode,
Expand All @@ -63,3 +99,27 @@ def resolve_get_table_one_big_table_query(
table, columns, include_table_translation
)
return sql_query


class ReorderColumns(Mutation):
"""Set display order for columns within a table.
ids: ordered list of Column UUIDs (index 0 = first)."""

class Arguments:
ids = List(UUID, required=True)

ok = Boolean()
errors = List(String)

def mutate(root, info, ids):
if not info.context.user.is_staff:
return ReorderColumns(ok=False, errors=["Permission denied"])
for i, col_id in enumerate(ids):
Column.objects.filter(pk=col_id).update(order=i)
return ReorderColumns(ok=True, errors=[])


class APIMutation:
reorder_tables = ReorderTables.Field()
reorder_observation_levels = ReorderObservationLevels.Field()
reorder_columns = ReorderColumns.Field()
2 changes: 2 additions & 0 deletions backend/apps/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from backend.apps.account.graphql import AccountMutation
from backend.apps.account_payment.graphql import Mutation as PaymentMutation
from backend.apps.account_payment.graphql import Query as PaymentQuery
from backend.apps.api.v1.graphql import APIMutation
from backend.apps.api.v1.graphql import Query as APIQuery
from backend.apps.user_notifications.graphql import (
DeactivateAllTableUpdateNotification,
Expand All @@ -16,6 +17,7 @@
extra_queries=[APIQuery, PaymentQuery, UserNotificationQuery],
extra_mutations=[
AccountMutation,
APIMutation,
PaymentMutation,
TableUpdateNotification,
DeactivateTableUpdateNotification,
Expand Down
14 changes: 13 additions & 1 deletion backend/custom/graphql_auto.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from django.apps import apps
from django.core.exceptions import ValidationError
from django.db import models
from django.forms import ModelForm, modelform_factory
from django.forms import ModelForm, ModelMultipleChoiceField, modelform_factory
from django.forms import fields as forms_fields
from graphene import (
ID,
Expand Down Expand Up @@ -429,6 +429,13 @@ def __init_subclass_with_meta__(
_meta=_meta, input_fields=input_fields, **options
)

@classmethod
def perform_mutate(cls, form, info):
obj = form.save(commit=False)
obj.save()
form.save_m2m()
return cls(errors=[], **{cls._meta.return_field_name: obj})

@classmethod
def get_form_kwargs(cls, root, info, **input):
# Get file data
Expand Down Expand Up @@ -495,3 +502,8 @@ def generate_form_fields(model: BaseModel):
@convert_django_field.register(models.FileField)
def convert_file_to_url(field, registry=None):
return FileFieldScalar(description=field.help_text, required=not field.null)


@convert_form_field.register(ModelMultipleChoiceField)
def convert_form_field_to_list_of_id(field):
return List(ID, description=field.help_text, required=field.required)
Loading