Skip to content

joshaustintech/ralphdb

Repository files navigation

ralphdb

A Redis-compatible server clone that targets full RESP3 support with RESP2 backwards compatibility.

Requirements

  • Rust toolchain (stable). Install via rustup.
  • Redis CLI tools (for redis-benchmark and redis-cli).

Install examples:

# macOS (Homebrew)
brew install rustup redis

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y curl build-essential redis-tools
curl https://sh.rustup.rs -sSf | sh

Build

cargo build --release

Run

# Default listen address and port
cargo run --release

# Or run the built binary directly
./target/release/ralphdb

Wiggum Loop (Local)

Use wiggum.sh to run an iterative Codex loop that stops only when the repo is truly green.

# Interactive max-iteration prompt
./wiggum.sh

# Explicit max iterations
./wiggum.sh --max-iters 80

What the default green check enforces:

  • Build with warnings denied.
  • Test suite with warnings denied.
  • Clippy with warnings denied.
  • TODO.md marker reports no remaining work.

Default settings:

  • SLEEP_SECS=90 (can be overridden).
  • REQUIRE_TODO_GREEN=1 (enabled).
  • TODO_FILE=TODO.md
  • TODO_MARKER_KEY=WIGGUM_REMAINING_WORK

The TODO gate expects one marker line in TODO.md:

WIGGUM_REMAINING_WORK=yes

or

WIGGUM_REMAINING_WORK=no

Behavior:

  • yes: loop is not green, even if compile/test/clippy pass.
  • no: TODO gate passes.
  • If marker is missing, TODO gate fails.

Useful overrides:

# Custom green condition command
./wiggum.sh --max-iters 40 --green-cmd "cargo test --all-targets"

# Disable TODO gating for a run
REQUIRE_TODO_GREEN=0 ./wiggum.sh --max-iters 20

# Use alternate todo file
TODO_FILE=/tmp/TODO.md ./wiggum.sh --max-iters 10

Configuration

Set these environment variables to override defaults:

  • RALPHDB_HOST (default: 127.0.0.1)
  • RALPHDB_PORT (default: 6379)
  • RALPHDB_THREADS (default: number of logical CPUs, at least 1)
  • RALPHDB_IDLE_TIMEOUT_SECS (default: 300, set to 0 to disable idle-timeout)

Real-World Usage

# Basic usage with redis-cli
redis-cli -p 6379 PING
redis-cli -p 6379 SET hello world
redis-cli -p 6379 GET hello
redis-cli -p 6379 CONFIG GET server.*

# RESP3 negotiation
redis-cli -p 6379 HELLO 3
redis-cli -p 6379 MSET key1 value1 key2 value2
redis-cli -p 6379 MGET key1 key2
redis-cli -p 6379 INFO

Features

  • RESP3 handshake via HELLO 3 with RESP2 fallback when needed.
  • Core command surface: PING, ECHO, SET, GET, DEL, EXISTS, INCR, DECR, MGET, MSET, EXPIRE, and TTL.
  • In-memory store with optional expirations and atomic counter support using a sharded DashMap.
  • RESP3 parsing now understands maps, sets, attributes, push frames, verbatim strings, and big numbers and enforces payload/collection caps to avoid DoS injections.
  • Configurable thread pool via RALPHDB_THREADS for multi-core command handling.
  • Idle connections close automatically after the configured RALPHDB_IDLE_TIMEOUT_SECS (300 seconds by default) to avoid resource leaks.
  • Optional command: INFO [section] returns server metadata text (currently server/default) in both RESP2/RESP3 modes.
  • CONFIG GET <pattern> exposes a documented key set (server.name, server.version, server.bind, server.port, server.threads) in that stable order and understands simple wildcard patterns such as server.*; unmatched patterns now return an empty array.
  • CLIENT SETNAME stores a per-connection name and CLIENT GETNAME returns that value (null if unset); both subcommands work for RESP2 and RESP3 sessions. CLIENT ID returns the numeric connection identifier while CLIENT LIST emits a RESP3 attribute (client → push → map with id, name, protocol) that is sent before the legacy summary string so metadata precedes the payload consumers read; RESP2 clients still receive the traditional bulk-string summary.
  • RESP3 scalar arguments (booleans, doubles, big numbers, and verbatim strings) are coerced into byte arrays after HELLO 3, so existing commands accept them without special handling.

INFO command

INFO returns a bulk-string that lists the server version, role, mode, and the stable id emitted via HELLO 3. Only the server/default sections are recognized; any other section yields ERR unsupported INFO section '<name>', so clients always see deterministic text.

Protocol Hardening

  • Bulk strings are capped at 32 MiB and collections at 1 million entries to keep frame parsing predictable and resilient.
  • Null bulk arguments are rejected and PING/MSET enforce their Redis-compatible arity expectations.
  • Expired keys are treated as missing so repeated EXPIRE commands return 0 and TTL immediately reflects removal.

Benchmarks (redis-benchmark)

Run representative workloads against a locally running server:

redis-benchmark -h 127.0.0.1 -p 6379 -t set,get -n 100 -c 1
redis-benchmark -h 127.0.0.1 -p 6379 -t set,get -n 200 -c 1 -P 10
redis-benchmark -h 127.0.0.1 -p 6379 -t incr -n 200 -c 1
redis-cli -p 6379 MSET bench:k1 v1 bench:k2 v2
redis-benchmark -h 127.0.0.1 -p 6379 -n 200 -c 1 MGET bench:k1 bench:k2
redis-benchmark -h 127.0.0.1 -p 6379 -n 200 -c 1 MSET bench:k1 v1 bench:k2 v2

Sample results from the runs above:

  • redis-benchmark -h 127.0.0.1 -p 6379 -t set,get -n 100 -c 1
    SET: 12,500 requests per second (avg latency 0.071 ms, max 0.119 ms)
    GET: 12,500 requests per second (avg latency 0.075 ms, max 0.119 ms)
  • redis-benchmark -h 127.0.0.1 -p 6379 -t set,get -n 200 -c 1 -P 10
    SET: 66,666.66 requests per second (avg latency 0.075 ms, max 0.119 ms)
    GET: 99,999.99 requests per second (avg latency 0.065 ms, max 0.087 ms)
  • redis-benchmark -h 127.0.0.1 -p 6379 -t incr -n 200 -c 1
    INCR: 8,333.33 requests per second (avg latency 0.108 ms, p95 0.143 ms, max 0.151 ms)
  • redis-benchmark -h 127.0.0.1 -p 6379 -n 200 -c 1 MGET bench:k1 bench:k2
    MGET: 22,222.22 requests per second (avg latency 0.036 ms, p95 0.071 ms, max 0.151 ms)
  • redis-benchmark -h 127.0.0.1 -p 6379 -n 200 -c 1 MSET bench:k1 v1 bench:k2 v2
    MSET: 40,000.00 requests per second (avg latency 0.020 ms, p95 0.023 ms, max 0.119 ms)

RESP3 (-3) coverage for the same command set:

redis-benchmark -3 -h 127.0.0.1 -p 6379 -t set,get -n 200 -c 1
redis-benchmark -3 -h 127.0.0.1 -p 6379 -t incr -n 200 -c 1
redis-cli -p 6379 MSET bench:k1 v1 bench:k2 v2
redis-benchmark -3 -h 127.0.0.1 -p 6379 -n 200 -c 1 MGET bench:k1 bench:k2
redis-benchmark -3 -h 127.0.0.1 -p 6379 -n 200 -c 1 MSET bench:k1 v1 bench:k2 v2

Sample RESP3 results (captured on 2026-03-03):

  • redis-benchmark -3 -h 127.0.0.1 -p 6379 -t set,get -n 200 -c 1
    SET: 10,000.00 requests per second (avg latency 0.092 ms, p95 0.119 ms, max 0.127 ms)
    GET: 22,222.22 requests per second (avg latency 0.037 ms, p95 0.087 ms, max 0.103 ms)
  • redis-benchmark -3 -h 127.0.0.1 -p 6379 -t incr -n 200 -c 1
    INCR: 50,000.00 requests per second (avg latency 0.013 ms, p95 0.023 ms, max 0.151 ms)
  • redis-benchmark -3 -h 127.0.0.1 -p 6379 -n 200 -c 1 MGET bench:k1 bench:k2
    MGET: 28,571.43 requests per second (avg latency 0.030 ms, p95 0.055 ms, max 0.111 ms)
  • redis-benchmark -3 -h 127.0.0.1 -p 6379 -n 200 -c 1 MSET bench:k1 v1 bench:k2 v2
    MSET: 50,000.00 requests per second (avg latency 0.014 ms, p95 0.023 ms, max 0.175 ms)

Validation note: redis-cli -p 6379 CONFIG GET "server.*" returns the documented entries (server.name, server.version, server.bind, server.port, server.threads) while benchmarking in both RESP2 and RESP3 workflows.

Multi-Client Profile Matrix

Use the scripted profile to capture a consistent concurrency and pipelining mix across RESP2 and RESP3:

# Run against a local server (defaults: HOST=127.0.0.1, PORT=6379)
scripts/benchmark_profile.sh baseline
scripts/benchmark_profile.sh candidate

Default profile settings:

  • Mixes: 1:1, 8:1, 32:1, 32:8 (clients:pipeline)
  • Commands: SET/GET/INCR, MGET, MSET
  • Protocols: RESP2 and RESP3
  • Repeats: 3
  • Requests per run: 10,000

Tune with environment variables when needed:

REQUESTS=50000 REPEATS=5 MIXES="1:1 16:1 64:4" scripts/benchmark_profile.sh stress

Each run writes raw redis-benchmark output to benchmark-results/<timestamp>-<label>/.

To document stability and latency deltas in a PR:

  1. Run a baseline profile on the reference revision.
  2. Run a candidate profile on your branch using the same machine/load conditions.
  3. Compare throughput and latency (avg, p95, max) for each matching file pair and report percentage deltas.

Latest baseline/candidate snapshot (captured on 2026-03-03 with REQUESTS=5000, REPEATS=1, MIXES="1:1 8:1"):

  • Baseline: benchmark-results/20260303-024854-baseline
  • Candidate: benchmark-results/20260303-024856-candidate
  • Throughput delta range: -8.70% to +12.75%
  • Avg-latency delta range: -13.64% to +11.76%
  • p95-latency delta range: -25.81% to +34.78%
  • Max-latency delta range: -35.63% to +90.57%

Per-workload delta summary (candidate vs baseline):

Workload file Baseline rps Candidate rps rps delta Baseline avg ms Candidate avg ms avg delta Baseline p95 ms Candidate p95 ms p95 delta Baseline max ms Candidate max ms max delta
resp2-basic-c1-p1-r1.txt 42464.78 45635.99 +7.47% 0.020 0.018 -10.00% 0.026 0.026 +0.00% 0.162 0.127 -21.44%
resp2-basic-c8-p1-r1.txt 94362.02 93800.85 -0.59% 0.062 0.061 -1.61% 0.095 0.095 +0.00% 0.156 0.196 +25.59%
resp2-mget-c1-p1-r1.txt 43859.65 44642.86 +1.79% 0.019 0.019 +0.00% 0.023 0.023 +0.00% 0.095 0.151 +58.95%
resp2-mget-c8-p1-r1.txt 90909.09 92592.59 +1.85% 0.064 0.063 -1.56% 0.095 0.095 +0.00% 0.175 0.215 +22.86%
resp2-mset-c1-p1-r1.txt 44247.79 44247.79 +0.00% 0.019 0.019 +0.00% 0.023 0.023 +0.00% 0.087 0.103 +18.39%
resp2-mset-c8-p1-r1.txt 94339.62 92592.59 -1.85% 0.062 0.062 +0.00% 0.095 0.095 +0.00% 0.191 0.175 -8.38%
resp3-basic-c1-p1-r1.txt 45531.35 44798.29 -1.61% 0.018 0.018 +3.77% 0.031 0.026 -17.20% 0.106 0.114 +7.57%
resp3-basic-c8-p1-r1.txt 93174.93 90968.16 -2.37% 0.062 0.063 +1.61% 0.095 0.100 +5.61% 0.154 0.202 +31.24%
resp3-mget-c1-p1-r1.txt 45871.56 44247.79 -3.54% 0.018 0.018 +0.00% 0.023 0.031 +34.78% 0.103 0.167 +62.14%
resp3-mget-c8-p1-r1.txt 94339.62 89285.71 -5.36% 0.059 0.064 +8.47% 0.087 0.095 +9.20% 0.199 0.191 -4.02%
resp3-mset-c1-p1-r1.txt 44247.79 43859.65 -0.88% 0.019 0.018 -5.26% 0.023 0.031 +34.78% 0.103 0.095 -7.77%
resp3-mset-c8-p1-r1.txt 87719.30 92592.59 +5.56% 0.067 0.062 -7.46% 0.103 0.095 -7.77% 0.215 0.207 -3.72%

Tests

cargo test

Tests cover the refreshed protocol parser (frame limits, RESP3 types) plus stricter command/expiry semantics (PING/MSET arity, EXPIRE on evicted keys, null argument rejection).

RESP3 Compatibility

  • RESP2 clients work out of the box; RESP3-specific capabilities activate when HELLO 3 is negotiated.
  • The protocol module parses RESP2/RESP3 frames and encodes replies that remain readable to either protocol version.

HELLO 3 Metadata

HELLO 3 replies with a RESP3 map describing the server state. The following keys are guaranteed:

Key Meaning
server The server name (ralphdb).
version The current package version (from CARGO_PKG_VERSION).
proto The negotiated protocol (3).
id A stable server identifier (the crate name, so it does not change per connection).
mode Always standalone in this build.
role Always primary.
modules Empty array for future module support.

Clients can rely on id staying the same across connections and restarts (it mirrors the crate name), while other fields convey runtime metadata.

RESP2 vs RESP3 Compatibility Matrix

Behavior RESP2 (default) RESP3 (after HELLO 3)
Scalar arguments Only bulk strings and integers are accepted; RESP3 scalars (boolean, double, big number, verbatim) are rejected. Scalars are coerced into byte strings (true/false, decimal doubles, textual big numbers, verbatim payloads) so existing commands keep working without special casts.
Null replies Null values are encoded as bulk string nil ($-1). Nulls emit the RESP3 literal (_) once the session is upgraded, but existing commands still return the same semantics.
Protocol metadata HELLO is optional and defaults to RESP2; no capability negotiation happens. HELLO 3 upgrades the connection and replies with a stable metadata map that exposes server, version, proto, id, mode, role, and an empty modules array.
Client naming CLIENT SETNAME works and GETNAME returns a null bulk string when unset. Same behavior, but GETNAME replies with _ once RESP3 is active (so clients can tell which mode they are in without parsing the value).

Project Plan

See RESP3_PLAN.md for instructions and milestones to reach full RESP3 compatibility.

About

Demo of a Redis clone built using the Ralph Wiggum loop

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors