Skip to content

[Stack 10/17] Fix test DB connection: use DATABASE_URL with dotenv#2517

Open
jucor wants to merge 1 commit intospr/edge/ea747196from
spr/edge/b9062b50
Open

[Stack 10/17] Fix test DB connection: use DATABASE_URL with dotenv#2517
jucor wants to merge 1 commit intospr/edge/ea747196from
spr/edge/b9062b50

Conversation

@jucor
Copy link
Copy Markdown
Collaborator

@jucor jucor commented Mar 30, 2026

Summary

  • Absorbed PR [CLOSED — absorbed into #2443] DB connect_timeout #2434 (DB connect_timeout): adds connect_timeout=5 to all DB connections so tests fail fast instead of hanging when Postgres is unavailable. Reverts timeout from production code (a 5s timeout could break real deployments), keeping it only in test code.
  • Adds require_dynamodb() and require_s3() helpers to conftest that probe services with short timeouts and pytest.fail() immediately — applied to all tests that previously hung when Docker services were down.
  • Adds pytest-timestamper for per-test timing visibility.
  • Fixes test_postgres_real_data.py to use DATABASE_URL (consistent with all other delphi Python code) via python-dotenv.

Refs #2442

Test plan

  • test_conversation_from_postgres passes
  • test_pakistan_conversation_batch passes (9 min, 400K votes)
  • Full delphi test suite: 294 passed, no regressions

🤖 Generated with Claude Code

Squashed commits

  • Add connect_timeout to all DB connections, fail tests on DB unavailable
  • Fail tests fast when services are unavailable instead of hanging
  • Fix conftest docstring: require_dynamodb/require_s3, not require_service
  • Skip (not fail) test_minio_access when MinIO is unavailable
  • Fail test_conversation_from_postgres fast when DynamoDB is unavailable
  • Fail test_run_math_pipeline_e2e fast when DynamoDB is unavailable
  • Add pytest-timestamper and fix test_501 hang on Docker unavailable
  • Defer heavy import in test_umap_narrative_pipeline (defensive)
  • Address Copilot review: fix timing_stats_compared None bug, fix return type
  • Fix test DB connection: use DATABASE_URL with dotenv
  • Fix CI: use find_dotenv() and fall back to DATABASE_* vars
  • Restore get_or_compute_conversation fixture and test reordering hook
  • Remove dead xdist_group markers from make_dataset_params

commit-id:b9062b50


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

@jucor jucor changed the title Fix test DB connection: use DATABASE_URL with dotenv [Stack 10/17] Fix test DB connection: use DATABASE_URL with dotenv Mar 30, 2026
@jucor jucor force-pushed the spr/edge/b9062b50 branch 2 times, most recently from 8dc9313 to abee548 Compare March 30, 2026 22:47
@jucor jucor force-pushed the spr/edge/ea747196 branch from 07c707d to e046e08 Compare March 30, 2026 22:47
## Summary


- **Absorbed PR #2434** (DB connect_timeout): adds `connect_timeout=5` to all DB connections so tests fail fast instead of hanging when Postgres is unavailable. Reverts timeout from production code (a 5s timeout could break real deployments), keeping it only in test code.
- Adds `require_dynamodb()` and `require_s3()` helpers to conftest that probe services with short timeouts and `pytest.fail()` immediately — applied to all tests that previously hung when Docker services were down.
- Adds `pytest-timestamper` for per-test timing visibility.
- Fixes `test_postgres_real_data.py` to use `DATABASE_URL` (consistent with all other delphi Python code) via `python-dotenv`.

Refs #2442

## Test plan

- [x] `test_conversation_from_postgres` passes
- [x] `test_pakistan_conversation_batch` passes (9 min, 400K votes)
- [x] Full delphi test suite: 294 passed, no regressions

🤖 Generated with [Claude Code](https://claude.com/claude-code)


## Squashed commits

- Add connect_timeout to all DB connections, fail tests on DB unavailable
- Fail tests fast when services are unavailable instead of hanging
- Fix conftest docstring: require_dynamodb/require_s3, not require_service
- Skip (not fail) test_minio_access when MinIO is unavailable
- Fail test_conversation_from_postgres fast when DynamoDB is unavailable
- Fail test_run_math_pipeline_e2e fast when DynamoDB is unavailable
- Add pytest-timestamper and fix test_501 hang on Docker unavailable
- Defer heavy import in test_umap_narrative_pipeline (defensive)
- Address Copilot review: fix timing_stats_compared None bug, fix return type
- Fix test DB connection: use DATABASE_URL with dotenv
- Fix CI: use find_dotenv() and fall back to DATABASE_* vars
- Restore get_or_compute_conversation fixture and test reordering hook
- Remove dead xdist_group markers from make_dataset_params

commit-id:b9062b50
@jucor jucor force-pushed the spr/edge/b9062b50 branch from abee548 to 517908a Compare March 31, 2026 00:35
@github-actions
Copy link
Copy Markdown

Delphi Coverage Report

File Stmts Miss Cover
init.py 2 0 100%
benchmarks/bench_pca.py 76 76 0%
benchmarks/bench_repness.py 81 81 0%
benchmarks/bench_update_votes.py 38 38 0%
benchmarks/benchmark_utils.py 34 34 0%
components/init.py 1 0 100%
components/config.py 165 133 19%
conversation/init.py 2 0 100%
conversation/conversation.py 1107 320 71%
conversation/manager.py 131 42 68%
database/init.py 1 0 100%
database/dynamodb.py 387 234 40%
database/postgres.py 305 205 33%
pca_kmeans_rep/init.py 5 0 100%
pca_kmeans_rep/clusters.py 257 22 91%
pca_kmeans_rep/corr.py 98 17 83%
pca_kmeans_rep/pca.py 52 16 69%
pca_kmeans_rep/repness.py 297 43 86%
pca_kmeans_rep/stats.py 107 22 79%
regression/init.py 4 0 100%
regression/clojure_comparer.py 188 17 91%
regression/comparer.py 887 720 19%
regression/datasets.py 135 27 80%
regression/recorder.py 36 27 25%
regression/utils.py 138 94 32%
run_math_pipeline.py 260 114 56%
umap_narrative/500_generate_embedding_umap_cluster.py 210 109 48%
umap_narrative/501_calculate_comment_extremity.py 112 53 53%
umap_narrative/502_calculate_priorities.py 135 135 0%
umap_narrative/700_datamapplot_for_layer.py 502 502 0%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 432 432 0%
umap_narrative/801_narrative_report_batch.py 785 785 0%
umap_narrative/802_process_batch_results.py 265 265 0%
umap_narrative/803_check_batch_status.py 175 175 0%
umap_narrative/llm_factory_constructor/init.py 2 2 0%
umap_narrative/llm_factory_constructor/model_provider.py 157 157 0%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 270 270 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 160 9 94%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 3 0 100%
umap_narrative/polismath_commentgraph/utils/converter.py 283 237 16%
umap_narrative/polismath_commentgraph/utils/group_data.py 354 336 5%
umap_narrative/polismath_commentgraph/utils/storage.py 584 518 11%
umap_narrative/reset_conversation.py 159 50 69%
umap_narrative/run_pipeline.py 453 312 31%
utils/general.py 62 41 34%
Total 10877 7647 30%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant