diff --git a/docs/website/contents/explanations/hard_forks_and_node_to_node_versioning.md b/docs/website/contents/explanations/hard_forks_and_node_to_node_versioning.md new file mode 100644 index 0000000000..88bd0390c3 --- /dev/null +++ b/docs/website/contents/explanations/hard_forks_and_node_to_node_versioning.md @@ -0,0 +1,185 @@ +# Hard forks and node-to-node versioning + +Part of: [System Overview](index.md) + +Does a hard-fork release require bumping the node-to-node version? +**Short answer: no.** + +A [hard fork](../references/glossary.md#hard-forks) is triggered by the on-chain ledger protocol major version, which governance sets. +It is not triggered by the network handshake version. +Adding the new era to the node can reuse the existing [node-to-node](../references/glossary.md#node-to-node-protocol) version. + +## Five different "versions" + +The question is easy to get wrong, because several separate things all get called a "version". Three of them drive the argument below: + +1. **[On-chain protocol version](../references/miscellaneous/era_transition_governance.md)** (ledger `major.minor`, e.g. major 9 = Conway). + Bumping the major version *is* the hard fork. + Governance controls it. +2. **Negotiated `NodeToNodeVersion`** (eg the integer 14/15/16 sent in the handshake). + Controls the wire codecs and which mini-protocols are available. +3. **`BlockNodeToNodeVersion` / `CardanoNodeToNodeVersion`** (the [HFC](../references/glossary.md#hard-fork-combinator) block-level version). + Controls how a Cardano block is serialised across eras. + +Two more carry the "version" name but do not drive this argument: + +4. **Header protocol version** (the `ProtVer major.minor` carried in each block header). + The producer sets it to the highest protocol version its node understands (its `cardanoProtocolVersion`), as an upgrade-readiness signal; it is not the version the block was built under. +5. **Negotiated `NodeToClientVersion`** (the handshake version for the node-to-*client* protocols used by wallets, the CLI, and db-sync). + The node-to-client counterpart of (2). + +The consensus code states outright that (2) and (3) are independent of (1). +See [`NetworkProtocolVersion.hs#L25-L33`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/Node/NetworkProtocolVersion.hs#L25-L33): + +```haskell +-- | Protocol versioning +-- +-- IMPORTANT Note that this is entirely independent of the +-- 'Ouroboros.Consensus.Shelley.Node.TPraos.shelleyProtVer' field et al. +-- +-- Its primary purpose is to control the details of on-the-wire codecs. +``` + +## Where eras live: the block codec version + +Why care about the block codec? +A hard fork puts a new kind of block on the wire. +Nodes send blocks, headers, and transactions to each other over the node-to-node mini-protocols, and the block codec version decides how each one is encoded. +It is the version argument to the encode and decode functions. +See [`Serialisation.hs#L68-L72`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/Node/Serialisation.hs#L68-L72): + +```haskell +-- | Serialise a type @a@ so that it can be sent across network via a +-- node-to-node protocol. +class SerialiseNodeToNode blk a where + encodeNodeToNode :: CodecConfig blk -> BlockNodeToNodeVersion blk -> a -> Encoding + decodeNodeToNode :: CodecConfig blk -> BlockNodeToNodeVersion blk -> forall s. Decoder s a +``` + +Adding an era changes how that era's blocks, headers, and transactions serialise. +On the node-to-node side, that serialisation is controlled by `BlockNodeToNodeVersion blk`. +The rest of this section shows the value is one codec per era, not one codec for all. + +`BlockNodeToNodeVersion (CardanoBlock c)` reduces to `HardForkNodeToNodeVersion (CardanoEras c)`. +[`HardForkNodeToNodeVersion`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/HardFork/Combinator/NetworkVersion.hs#L47-L63) has two constructors: `HardForkNodeToNodeDisabled` (the HFC off, used only before Shelley) and `HardForkNodeToNodeEnabled` ([`NetworkVersion.hs#L60-L63`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/HardFork/Combinator/NetworkVersion.hs#L60-L63)). +For the Cardano block, `HardForkNodeToNodeDisabled` appears only in `CardanoNodeToNodeVersion1`, which the node no longer advertises (the supported-versions map below lists only `CardanoNodeToNodeVersion2`), so it is unused on the Cardano side (though the single-era `ByronHFC` and `ShelleyHFC` instances still construct it). +In particular, in [`Node.hs#L280-L293`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus-cardano/src/ouroboros-consensus-cardano/Ouroboros/Consensus/Cardano/Node.hs#L280-L293) we have: + +```haskell +pattern CardanoNodeToNodeVersion2 :: BlockNodeToNodeVersion (CardanoBlock c) +pattern CardanoNodeToNodeVersion2 = + HardForkNodeToNodeEnabled + HardForkSpecificNodeToNodeVersion1 -- era-tag version + ( WrapNodeToNodeVersion ByronNodeToNodeVersion2 -- ByronBlock + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (TPraos c) ShelleyEra + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (TPraos c) AllegraEra + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (TPraos c) MaryEra + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (TPraos c) AlonzoEra + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (Praos c) BabbageEra + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (Praos c) ConwayEra + :* WrapNodeToNodeVersion ShelleyNodeToNodeVersion1 -- ShelleyBlock (Praos c) DijkstraEra + :* Nil ) +``` + +One entry per era, so a hard fork appends an entry and leaves the existing ones unchanged. +It creates no new `CardanoNodeToNodeVersion`, and every existing era still serialises exactly as before. + +## The supported-versions map + +The node declares which node-to-node versions it speaks in `supportedNodeToNodeVersions`. +Its type already keeps two things apart: the negotiated handshake version and the block codec version. +See [`NetworkProtocolVersion.hs#L50-L57`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus/src/ouroboros-consensus/Ouroboros/Consensus/Node/NetworkProtocolVersion.hs#L50-L57): + +```haskell +class HasNetworkProtocolVersion blk => SupportedNetworkProtocolVersion blk where + -- | Enumerate all supported node-to-node versions + supportedNodeToNodeVersions :: + Proxy blk -> Map NodeToNodeVersion (BlockNodeToNodeVersion blk) +``` + +The `Map` type already separates the two. +Its key is `NodeToNodeVersion`, the integer negotiated in the handshake; its value is `BlockNodeToNodeVersion blk`, the block codec version. +So the handshake version and the block codec are, by construction, two separate things. + +The Cardano instance fills the map like this. +See [`Node.hs#L437-L442`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus-cardano/src/ouroboros-consensus-cardano/Ouroboros/Consensus/Cardano/Node.hs#L437-L442): + +```haskell +supportedNodeToNodeVersions _ = + Map.fromList $ + [ (NodeToNodeV_14, CardanoNodeToNodeVersion2) + , (NodeToNodeV_15, CardanoNodeToNodeVersion2) + , (NodeToNodeV_16, CardanoNodeToNodeVersion2) + ] +``` + +We can see that bumping the handshake integer from 14 to 15 to 16 did not change the block codec. + +## The answer, and the mainnet caveat + +Does a hard fork *require* a node-to-node bump? That depends on what "require" means. +One is about the wire format: does the new era need a node-to-node bump to serialise and deserialise? +The other is operational: does a release bump the version anyway, to force upgrades around the fork? +The two have different answers. + +**For the new era's serialisation: no, as long as the change only appends.** +Since [`ed49cd11b`](https://github.com/IntersectMBO/ouroboros-consensus/commit/ed49cd11b2318a201bba6ef3fe0ce514461da015), the block codec version cannot disable an era, and every node-to-node payload self-identifies its era. +Adding an era appends one always-on entry, as Dijkstra did to `CardanoNodeToNodeVersion2` above, and leaves every existing era's per-era codec untouched. +A not-yet-active era is held back by the max-major protocol-version check, not by the node-to-node version. +That check is in the consensus protocol's envelope validation. +See [`Praos.hs#L111-L122`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus-cardano/src/shelley/Ouroboros/Consensus/Shelley/Protocol/Praos.hs#L111-L122): + +```haskell +envelopeChecks cfg lv hdr = do + unless (m <= maxpv) $ throwError (ObsoleteNode m maxpv) + ... + where + (MaxMajorProtVer maxpv) = praosMaxMajorPV pp + (ProtVer m _) = lvProtocolVersion lv +``` + +`m` is the protocol major version currently in force on-chain, read from the ledger view (`lvProtocolVersion lv`); `maxpv` is the highest major version the node supports. +If that on-chain version exceeds `maxpv`, the header is rejected as `ObsoleteNode`. +That maximum comes from the protocol version the node's software and config declare, not from the handshake. +See [`Node.hs#L640-L641`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus-cardano/src/ouroboros-consensus-cardano/Ouroboros/Consensus/Cardano/Node.hs#L640-L641): + +```haskell +maxMajorProtVer :: MaxMajorProtVer +maxMajorProtVer = MaxMajorProtVer $ pvMajor cardanoProtocolVersion +``` + +So no new node-to-node version is needed to carry the new era. + +The exception is changing how an *existing* era serialises on the wire. +Each era is pinned to a per-era codec version, threaded into the `encodeNodeToNode` / `decodeNodeToNode` methods that serialise that era's block, header, transaction, and transaction id. +The Shelley-based eras all use `ShelleyNodeToNodeVersion1`, but a per-era version can grow: Byron already has two, `ByronNodeToNodeVersion1` and `ByronNodeToNodeVersion2` (headers without vs with a size hint). +Byron's header codec branches on the version to pick the layout. +See [`Serialisation.hs#L89-L105`](https://github.com/IntersectMBO/ouroboros-consensus/blob/bd119d907b7701aac5f7825e6645614a7473e9bf/ouroboros-consensus-cardano/src/byron/Ouroboros/Consensus/Byron/Node/Serialisation.hs#L89-L105): + +```haskell +instance SerialiseNodeToNode ByronBlock (Header ByronBlock) where + encodeNodeToNode ccfg = \case + ByronNodeToNodeVersion1 -> + wrapCBORinCBOR $ + encodeUnsizedHeader . fst . splitSizeHint + ByronNodeToNodeVersion2 -> + encodeDisk ccfg . unnest + ... +``` + +Bumping an existing era's codec while still supporting peers that speak the old one would need a second `CardanoNodeToNodeVersion` holding the new per-era version, mapped to a new negotiated `NodeToNodeVersion`, so peers can negotiate which codec to use. +That is a node-to-node bump; adding an era does not do it. + +**As an upgrade gate around a hard fork: sometimes, by choice.** +Conway's most recent hard fork, Plomin, made `NodeToNodeV_14` mandatory on 2025-01-29, per the comment on that constructor in the [`NodeToNodeVersion` enum](https://github.com/IntersectMBO/ouroboros-network/blob/e8d59d8a219563760fc21ba5bc86fab77d886742/cardano-diffusion/api/lib/Cardano/Network/NodeToNode/Version.hs#L71-L72) (in `ouroboros-network`). +That bump forces nodes to upgrade before the fork. +It is a coordination step, not an era-codec requirement: V_14 itself lists no wire-format change; the changes shipped around then were in V_13, and were all PeerSharing-related. + +So a hard-fork release does not need a node-to-node bump for the new era's sake. +A release may still bump the version and make it mandatory as a coordination step, which is a separate operational decision. + +## Further reading + +- [Era transition governance](../references/miscellaneous/era_transition_governance.md): how on-chain governance ends an era by incrementing the protocol major version, with the era-to-version table. +- [Adding an era](../howtos/adding_an_era.md): the checklist for adding a new era to the node. +- [Key type families and classes](../references/key_type_families_and_classes.md): `HasNetworkProtocolVersion` and the block-version type families used here. diff --git a/docs/website/contents/howtos/adding_an_era.md b/docs/website/contents/howtos/adding_an_era.md index 840c3237b8..f98e0fdb68 100644 --- a/docs/website/contents/howtos/adding_an_era.md +++ b/docs/website/contents/howtos/adding_an_era.md @@ -67,10 +67,19 @@ be adding is the Alonzo era, which comes after the Mary era. instance. * In `Ouroboros.Consensus.Cardano.Node`, update the `SerialiseHFC` instance by - following the existing patterns. Add a new `CardanoNodeToNodeVersion` and - `CardanoNodeToClientVersion` that enable the `AlonzoEra`, update the existing - ones so that they disable the new era. Be sure to include the new versions in - the two methods of the `SupportedNetworkProtocolVersion` instance. Extend + following the existing patterns. Append the new era to `CardanoNodeToNodeVersion2`, reusing + `ShelleyNodeToNodeVersion1`, and do not add a new `CardanoNodeToNodeVersion`. + The node-to-node side has no per-era disable: `WrapNodeToNodeVersion` is a + single-constructor wrapper (the enable/disable type was removed in + `ed49cd11b`), so every era there is always on (see [Hard forks and + node-to-node versioning](../explanations/hard_forks_and_node_to_node_versioning.md)). + On the node-to-client side, add the new era as `EraNodeToClientEnabled` to + every existing `CardanoNodeToClientVersion`; that side still has + `EraNodeToClientDisabled`, but current practice leaves every era enabled. + Adding an era does not itself require a new `CardanoNodeToClientVersion`: new + client versions are added separately, when node-to-client capabilities change + (a new query or a codec change) and bump `ShelleyNodeToClientVersion`, and any + such version is listed in the `supportedNodeToClientVersions` method. Extend `protocolInfoCardano` with the new era by following the type errors and adding the missing parameters (including `ProtocolParamsTransition`). Don't forget to derive `maxMajorProtVer` from the new final era. Update diff --git a/docs/website/contents/references/glossary.md b/docs/website/contents/references/glossary.md index d1c628cfab..5fd117e083 100644 --- a/docs/website/contents/references/glossary.md +++ b/docs/website/contents/references/glossary.md @@ -466,6 +466,12 @@ Nodes communicate with other nodes via a set of two-party protocols in which one The prefix of the node's selection that excludes the youngest `k` blocks. +## ;Node-to-node protocol + +The set of [mini protocols](#mini-protocol) (ChainSync, BlockFetch, TxSubmission, KeepAlive, PeerSharing) that Cardano nodes use to exchange blocks, headers, and transactions, keep connections alive, and discover peers. +Its wire format is negotiated at connection time via a `NodeToNodeVersion`. +See [Hard forks and node-to-node versioning](../explanations/hard_forks_and_node_to_node_versioning.md). + ## ;Nonce The [ledger state](#ledger-state) maintains a nonce, updated by each block's header, independent of the block body. The ledger takes a snapshot of this nonce once per epoch. The snapshot taken during one epoch is used by VRFs in the next epoch. diff --git a/docs/website/sidebars.js b/docs/website/sidebars.js index df4c7b62f3..707068211b 100644 --- a/docs/website/sidebars.js +++ b/docs/website/sidebars.js @@ -29,6 +29,7 @@ const sidebars = { 'explanations/ticking', 'explanations/queries', 'explanations/node_tasks', + 'explanations/hard_forks_and_node_to_node_versioning', ] } ],