An async HTTP proxy built with NestJS that queues outgoing requests, retries failed ones with exponential backoff and jitter, and records every attempt.
| Tool | Version |
|---|---|
| Node.js | 18+ |
| pnpm | 8+ |
| Docker | any recent |
# 1. Install dependencies
pnpm install
# 2. Copy env file (defaults work with the docker-compose postgres)
cp .env.example .env
# 3. Start postgres
docker compose up -d
# 4. Apply migrations (creates tables + index)
pnpm migration:run
# 5. Start the server (worker boots with it)
pnpm start:devnode demo.mjsThe script starts its own local mock HTTP server, submits three requests that demonstrate each code path, and prints a live timeline with backoff gaps.
pnpm test # unit tests (BackoffService)
pnpm test:e2e # end-to-end lifecycle testscurl -s -X POST http://localhost:3000/request \
-H 'Content-Type: application/json' \
-d '{
"url": "https://httpbin.org/status/200",
"method": "GET",
"maxRetries": 5,
"backoffMs": 1000
}' | jq .Response 201:
{ "id": "e1b2c3d4-...", "status": "pending" }| Field | Type | Default | Notes |
|---|---|---|---|
url |
string | required | must be a valid URL |
method |
string | required | GET POST PUT PATCH DELETE |
body |
string | — | optional request body |
maxRetries |
integer | 5 | min 1 |
backoffMs |
integer | 1000 | min 100, base for exponential formula |
curl -s http://localhost:3000/requests/e1b2c3d4-... | jq .Response 200:
{
"id": "e1b2c3d4-...",
"url": "https://httpbin.org/status/500",
"method": "GET",
"status": "retrying",
"attemptCount": 2,
"maxRetries": 5,
"backoffMs": 1000,
"nextRetryAt": "2024-01-01T00:00:04.000Z",
"lastError": "HTTP 500",
"result": null,
"attempts": [
{ "attemptNumber": 1, "statusCode": 500, "waitedMs": 12, "madeAt": "..." },
{ "attemptNumber": 2, "statusCode": 500, "waitedMs": 18, "madeAt": "..." }
]
}Returns 404 if the id does not exist.
# all requests
curl -s http://localhost:3000/requests | jq .
# only failed ones
curl -s 'http://localhost:3000/requests?status=failed' | jq .Valid status values: pending retrying completed failed
waitMs = backoffMs × 2^attemptNumber × jitter
jitter = random in [0.8, 1.2]Example with backoffMs = 500:
| After attempt | Base wait | With jitter range |
|---|---|---|
| 1 | 1 000 ms | 800 – 1 200 ms |
| 2 | 2 000 ms | 1 600 – 2 400 ms |
| 3 | 4 000 ms | 3 200 – 4 800 ms |
| 4 | 8 000 ms | 6 400 – 9 600 ms |
When an external service starts failing, retrying immediately at full speed makes the problem worse. If you have hundreds of queued requests all hammering a struggling server every second, you're turning a recoverable issue into a prolonged outage. Exponential backoff gives the server breathing room; each successive wait is double the last, so the retry attempts tails off naturally as time goes on.
Pure exponential backoff has a hidden flaw: if many clients all started failing at the same moment (a brief server restart, say), they'll all retry on the same schedule. You get waves, a quiet gap, then a synchronized large load, then another quiet gap, then another wave. Jitter breaks that synchronization. By multiplying the wait time by a random number in [0.8, 1.2], each client ends up on a slightly different schedule, spreading the retry load smoothly over time instead of concentrating it.
A 5xx means the server had a problem — it was overloaded, it crashed, it's being deployed. The request itself was fine; the server just couldn't handle it right now. Retrying makes sense because the server might recover.
A 4xx means the request is the problem — the URL doesn't exist (404), the method isn't allowed (405), the payload is invalid (422). Retrying the same bad request will always get the same 4xx back. Retrying is pointless and wastes resources. The right response is to mark it failed immediately and surface the error so the caller can fix the request.
Below is a request that failed three times with HTTP 500, then succeeded on the fourth attempt. The attempts array shows the full history with timing data.
The gaps between madeAt timestamps show the backoff doubling (1 s → 2 s → 4 s) with jitter making each gap slightly different from an exact double.
TypeORM query builder column quoting. The entity uses camelCase property names (nextRetryAt), but PostgreSQL stores them as quoted identifiers. Getting the QueryBuilder andWhere('r.nextRetryAt <= NOW()') to produce correct SQL took a few iterations — TypeORM translates property names to column names automatically, but it wasn't obvious.
pnpm build script approval. pnpm v11 introduced a security gate where packages with install scripts need explicit approval before any command runs. This blocked pnpm build, pnpm test, and everything else until I added "pnpm": { "onlyBuiltDependencies": ["@nestjs/core"] } to package.json.
Transaction boundary. My first pass updated the request and inserted the attempt in two separate repository.save() calls. If the process crashed between them, the attempt count would be wrong (incremented but no attempt row). Moving both into a single DataSource.transaction() made the state consistent.
Background workers in NestJS. Using OnApplicationBootstrap and OnApplicationShutdown to own the lifecycle of a setInterval loop. The pattern of a busy flag to prevent overlapping ticks is something I'll reuse anywhere you have a periodic job that might occasionally run long.
Atomic transactions as a correctness invariant. The attempt insert and the request status update have to be atomic. If they're separate, you can read an inconsistent state — attemptCount = 2 but only one row in attempts. I now think about "what partial state can exist if this crashes halfway?" as a first-class design question.
Exponential backoff with jitter as a production pattern. I knew what retries were. I didn't know about the thundering herd problem or why jitter specifically helps. The AWS architecture blog post on this was eye-opening; the graph simulation showing synchronized retries vs jittered retries helped visualise the pattern.
TypeORM migrations as the contract. Keeping synchronize: false and running explicit migrations means the schema never silently drifts. The CLI generates correct SQL from entity diffs, and the data-source.ts file keeps CLI usage consistent with what the app actually connects to.
- Exponential Backoff and Jitter — AWS Architecture Blog — the clearest explanation of why jitter is needed and the simulation that makes it obvious
- NestJS documentation — Custom providers, lifecycle hooks, schedule
- TypeORM documentation — DataSource, migrations, QueryBuilder
- TypeORM migration guide
@nestjs/axiosdocs —firstValueFrompattern,validateStatusoption- PostgreSQL docs — TRUNCATE, partial indexes, composite indexes
- Claude (Anthropic) — used to scaffold the NestJS project structure, talk through the worker loop architecture, and debug the E2E test race condition
Before this project, I thought about HTTP failures as binary; it either worked or it didn't, and "add a retry" was a complete solution. Now I think differently about three specific things.
Retry logic has policy, not just a count. Whether to retry, how long to wait, whether the request itself is the problem (4xx) or the server is the problem (5xx) — these are separate decisions that belong in a defined policy, not scattered try/catch blocks. Building the decision tree explicitly made that visible.
Distributed systems have partial states. The crash-between-two-writes problem is real at any scale. Any time I write to two places as part of one logical operation, I'll now ask "what does an observer see if we stop halfway?" and wrap it in a transaction if the answer is "something wrong."
Backpressure is an important concern. A worker that polls unboundedly, or retries immediately, can take a struggling downstream service and make it permanently unavailable. The batch limit (10 per tick), the backoff, and the jitter are all about being a polite client — backing off when things are bad rather than amplifying the problem. I'll think about this in any system that produces work faster than it can be consumed.
FlowBrand exposes two long-running async endpoints that the frontend polls for completion:
| Endpoint | Submitted by | Terminal states |
|---|---|---|
GET /api/funnels/generate/status/:funnelId |
POST /api/funnels/generate |
active, failed |
GET /api/funnels/upload/progress/:uploadId |
POST /api/funnels/upload |
ready, failed |
Both endpoints are currently polled on a fixed client interval (every 2 seconds). When many browser tabs are open simultaneously, the server receives a perfectly synchronised wave of requests at the same rate regardless of whether anything has changed. During a slow LLM response or a large document extraction, those tabs keep hammering at full rate; exactly when the server is already under load.
The response body should include a nextPollMs hint. The client waits that
duration before the next request instead of using a hardcoded interval.
GET /api/funnels/generate/status/:funnelId response (proposed):
{
"data": {
"funnelId": "34f3f095-...",
"status": "generating",
"nextPollMs": 4000
}
}The backend calculates the hint based on elapsed time since the funnel entered
generating state — short early on (likely still queued), longer as time passes
(LLM is running):
0–5 s since created → nextPollMs: 2000
5–15 s → nextPollMs: 4000
15–30 s → nextPollMs: 8000
30 s+ → nextPollMs: 16000 (cap)
status = active/failed → omit field (no next poll)
GET /api/funnels/upload/progress/:uploadId response (proposed):
{
"data": {
"uploadId": "...",
"status": "parsing",
"percentComplete": 40,
"nextPollMs": 2000
}
}Here percentComplete already exists; the hint can shrink as progress
accelerates:
percentComplete 0–25 → nextPollMs: 3000
percentComplete 26–75 → nextPollMs: 2000
percentComplete 76–99 → nextPollMs: 1000
status = ready/failed → omit field
Add ±20 % random jitter on the client side to prevent all tabs from re-synchronising after a deploy or cache miss:
const jitter = nextPollMs * (0.8 + Math.random() * 0.4); // ±20 %
setTimeout(poll, jitter);- Funnel generation hands off to Gemini/Groq via a Bull queue. The LLM call can take 10–40 s. A fixed 3 s interval means 10–13 redundant requests per user per generation. With backoff, that drops to 3–5.
- Document extraction runs in a Bull worker and updates
percent_completeon theuploaded_documentsrow. Polling faster than the worker writes is pure waste; the hint aligns the poll rate with the actual write cadence.
The right long-term move is GET /api/funnels/generate/status/:funnelId
returning 202 Accepted with a Retry-After header while the job is pending,
and 200 only on terminal states — but the nextPollMs body field is a
zero-breaking-change step in that direction.