tombstone

Tombstone is a small Java database engine for auditable state changes. It is built around an explicit pipeline:

event -> transition -> projection

An event is an immutable fact, a transition is the materialized explanation of how that fact changes state, and a projection is the resulting state. Tombstone persists the relationship directly instead of hiding transitions inside handlers.

Start with docs/INDEX.md for the layered documentation.

Embedded usage

EventType orderPlaced = EventType.of(DomainEventType.ORDER_PLACED);
SchemaRegistry schemas = SchemaRegistry.with(
    EventSchema.of(orderPlaced, PayloadSchema.utf8Text(), PayloadSchema.utf8Text()));
TombstoneConfig config = TombstoneConfig.embedded(Path.of("data"))
    .withSchemaRegistry(schemas);
try (Tombstone db = TombstoneEngine.open(config)) {
  AppendResult appended = db.append(new AppendCommand(
      StreamId.of("orders"),
      ExpectedVersion.noStream(),
      orderPlaced,
      "event".getBytes(StandardCharsets.UTF_8),
      "transition".getBytes(StandardCharsets.UTF_8)));

  ReadEventsResult events = db.readEvents(new ReadEventsCommand(StreamId.of("orders")));
}

Clustered usage

The first clustered mode is a deliberately simple single-leader model. A leader appends locally, signs the record, and sends the signed record to followers. Followers verify the HMAC, key id, checksum, hash chain, and stream sequence before accepting the record.

Cluster members must share a compatible schema registry. Unknown event types or payloads that do not satisfy the registered event and transition schemas are rejected before append, recovery, replication acceptance, or projection replay.

ClusterNode leader = ClusterNode.leader("leader", leaderEngine);
ClusterNode follower = ClusterNode.follower("follower", followerEngine);
leader.membership().add(follower);
leader.append(command);

Query DSL

Tombstone also includes a small SQL-like DSL that runs natively through the Java API:

TombstoneDsl dsl = TombstoneDsl.using(db);
dsl.execute("APPEND STREAM orders EXPECT ANY TYPE ORDER_PLACED EVENT 'order-1' TRANSITION 'placed'");
dsl.execute("READ STREAM orders LIMIT 10");
dsl.execute("PROJECT STREAM orders INTO orders_view INITIAL '' SAVE");

See docs/QUERY_DSL.md for the grammar and execution model.

Developer docs

• docs/INDEX.md: documentation index and reading paths.
• docs/INTRODUCTION.md: project introduction and mental model.
• docs/USING_ENGINE.md: shortest path to using the engine.
• docs/ARCHITECTURE.md: layered architecture and system flow.
• docs/DECISIONS.md: architecture decisions and tradeoffs.
• docs/DEVELOPER_GUIDE.md: embedding Tombstone, schemas, projections, DSL, clustering, and testing.
• docs/OBSERVABILITY.md: metrics, operation events, health checks, and adapter points.
• docs/LOGGING.md: standard Java logging, levels, redaction, and routing.
• docs/PERFORMANCE.md: ring-buffer outbox and direct-buffer storage write path.
• docs/PROFESSIONAL_HARDENING.md: durability modes, exception taxonomy, and hardening roadmap.
• docs/GRPC_TRANSPORT.md: networked cluster replication over gRPC.
• docs/RAFT_CONSENSUS.md: Raft election, durable metadata, gRPC RPC, quorum append acknowledgement, and threading model.

Security model

Records and projection snapshots are signed with HMAC-SHA256 using an in-memory Keyring. Checksums detect accidental frame corruption. Hash chains detect removed or reordered stream records. HMACs authenticate record content and security metadata. Secret keys are never persisted by Tombstone.

Guarantees

• Append-only file frames.
• Per-stream expected-version checks.
• Per-stream hash chains.
• Strict schema registry checks before append, recovery, replication acceptance, and projection replay.
• HMAC verification before records and projections are returned.
• Followers verify replicated records independently.
• Cluster replication supports in-memory and gRPC transports, including TLS/mTLS, bearer auth, retry policy, and streaming batches.
• Raft coordination provides elections, durable term/vote/commit metadata, snapshots, dynamic peers, pre-vote, leadership transfer, gRPC RPC, follower catch-up state, and majority append acknowledgements.
• Projections are produced by applying transitions.
• Replication uses a bounded in-process ring-buffer outbox.
• Segment appends use FileChannel with reusable direct-buffer staging.
• Append durability is configurable with a durable default.
• Plug-and-play observability provides metrics, operation events, and health checks.

Non-goals

Tombstone does not implement SQL compatibility, secondary indexes, production-complete Raft joint consensus, automatic failover orchestration, automatic conflict resolution, distributed transactions, persistent secret-key storage, persistent schema registry storage, certificate rotation, service discovery, or load balancing.