From bf5c01df0a6b58d490f472c53f9b228121a15e1c Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Tue, 17 Mar 2026 03:21:45 -0600 Subject: [PATCH 1/7] add serialization documentation --- .vitepress/config.mts | 8 + docs/feature/serialization.md | 5 + docs/feature/serialization/codecs.md | 241 ++++++++++++++++++ docs/feature/serialization/network-buffers.md | 236 +++++++++++++++++ 4 files changed, 490 insertions(+) create mode 100644 docs/feature/serialization.md create mode 100644 docs/feature/serialization/codecs.md create mode 100644 docs/feature/serialization/network-buffers.md diff --git a/.vitepress/config.mts b/.vitepress/config.mts index c32d9d1..a80ae1d 100644 --- a/.vitepress/config.mts +++ b/.vitepress/config.mts @@ -96,6 +96,14 @@ export default defineConfig({ text: "Feature", items: [ { text: "Adventure", link: "/docs/feature/adventure" }, + { + text: "Serialization", + link: "/docs/feature/serialization", + items: [ + { text: "Codecs", link: "/docs/feature/serialization/codecs" }, + { text: "Network Buffers", link: "/docs/feature/serialization/network-buffers" }, + ], + }, { text: "Items", link: "/docs/feature/items" }, { text: "Events", diff --git a/docs/feature/serialization.md b/docs/feature/serialization.md new file mode 100644 index 0000000..3bfdf9e --- /dev/null +++ b/docs/feature/serialization.md @@ -0,0 +1,5 @@ +# Serialization +Minestom provides two systems for serializing data: + +- **[Codecs](/docs/feature/serialization/codecs)**: Map-like serialization (JSON, NBT, etc.). Use it for configuration files or data persistence. +- **[Network Buffers](/docs/feature/serialization/network-buffers)**: Binary serialization designed for the Minecraft protocol. Use it when your storage needs to be compact, like block data or packets sent over the network. \ No newline at end of file diff --git a/docs/feature/serialization/codecs.md b/docs/feature/serialization/codecs.md new file mode 100644 index 0000000..af0eabd --- /dev/null +++ b/docs/feature/serialization/codecs.md @@ -0,0 +1,241 @@ +# Codecs +Codecs encode and decode data to multiple formats (JSON, NBT, etc.) using the same definition. This allows you to write your serialization logic once and use it with any supported format. + +```java +record PlayerData(String name, int level, @Nullable String nickname) { + static final StructCodec CODEC = StructCodec.struct( + "name", Codec.STRING, PlayerData::name, + "level", Codec.INT, PlayerData::level, + "nickname", Codec.STRING.optional(), PlayerData::nickname, + PlayerData::new + ); +} + +PlayerData data = new PlayerData("Steve", 67, null); +JsonElement json = PlayerData.CODEC.encode(Transcoder.JSON, data).orElseThrow(); +BinaryTag nbt = PlayerData.CODEC.encode(Transcoder.NBT, data).orElseThrow(); +PlayerData decodedData = PlayerData.CODEC.decode(Transcoder.JSON, json).orElseThrow(); +``` + +## Primitive Codecs +| Codec | Java Type | Description | +| ---------------------- | ------------------- | ---------------------------------------- | +| `Codec.BOOLEAN` | `Boolean` | Boolean value | +| `Codec.BYTE` | `Byte` | 8-bit integer | +| `Codec.SHORT` | `Short` | 16-bit integer | +| `Codec.INT` | `Integer` | 32-bit integer | +| `Codec.LONG` | `Long` | 64-bit integer | +| `Codec.FLOAT` | `Float` | 32-bit floating point | +| `Codec.DOUBLE` | `Double` | 64-bit floating point | +| `Codec.STRING` | `String` | UTF-8 string | +| `Codec.KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | +| `Codec.UUID` | `UUID` | UUID stored as int array | +| `Codec.UUID_STRING` | `UUID` | UUID stored as string | +| `Codec.COMPONENT` | `Component` | Adventure text component | +| `Codec.NBT` | `BinaryTag` | Any NBT tag | +| `Codec.NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | +| `Codec.BYTE_ARRAY` | `byte[]` | Byte array | +| `Codec.INT_ARRAY` | `int[]` | Integer array | +| `Codec.LONG_ARRAY` | `long[]` | Long array | +| `Codec.BLOCK_POSITION` | `Point` | Block coordinates | +| `Codec.VECTOR3D` | `Point` | Double precision coordinates | + +[ TODO: Item stack and other common codecs ] + +## Transforming Types +The `.transform()` method converts between types during encoding and decoding. This is useful for custom types that can be represented as a simpler type. + +```java +record GameMode(String mode) {} +Codec MODE_CODEC = Codec.STRING.transform(GameMode::new, GameMode::mode); +``` + +`Codec.Enum()` is a shorthand that serializes an enum as its lowercase name (e.g., `NORTH` → `"north"`): + +```java +Codec DIRECTION = Codec.Enum(Direction.class); +``` + +## Optional Fields +Fields marked with `.optional()` can be missing from the encoded data and will decode to `null`. You can also provide a default value. + +```java +record ItemData(String name, @Nullable String description) { + static final StructCodec CODEC = StructCodec.struct( + "name", Codec.STRING, ItemData::name, + "description", Codec.STRING.optional(), ItemData::description, + ItemData::new + ); +} + +ItemData itemData = ItemData.CODEC.decode(Transcoder.JSON, JsonParser.parseString("{\"name\": \"test\"}")).orElseThrow(); +``` + +Default values are used when the field is missing from the data: + +```java +StructCodec.struct( + "max_players", Codec.INT.optional(20), Config::maxPlayers, + // ... +) +``` + +## Lists and Collections +Use `.list()` for lists, `.set()` for sets, and `.listOrSingle()` for flexible decoding that accepts either a single value or an array. + +```java +Codec> tags = Codec.STRING.list(100); +Codec> players = Codec.UUID.set(); +Codec> flexible = Codec.STRING.listOrSingle(); +``` + +Example with a quest that has multiple objectives: + +```java +record Quest(String name, List objectives) { + static final StructCodec CODEC = StructCodec.struct( + "name", Codec.STRING, Quest::name, + "objectives", QuestObjective.CODEC.list(), Quest::objectives, + Quest::new + ); +} +``` + +## Maps +Use `.mapValue()` to create a codec for maps with string keys. + +```java +record Leaderboard(Map scores) { + static final StructCodec CODEC = StructCodec.struct( + "scores", Codec.STRING.mapValue(Codec.INT), Leaderboard::scores, + Leaderboard::new + ); +} +``` + +## Nested Structures +StructCodecs can be nested to create complex hierarchies. + +```java +record Position(double x, double y, double z) { + static final StructCodec CODEC = StructCodec.struct( + "x", Codec.DOUBLE, Position::x, + "y", Codec.DOUBLE, Position::y, + "z", Codec.DOUBLE, Position::z, + Position::new + ); +} + +record BedwarsMap(String name, Position spawnPosition) { + static final StructCodec CODEC = StructCodec.struct( + "name", Codec.STRING, BedwarsMap::name, + "spawn_position", Position.CODEC, BedwarsMap::spawnPosition, + BedwarsMap::new + ); +} +``` + +## Inlined Structures +Use `StructCodec.INLINE` to flatten nested fields into the parent object instead of creating a nested map. + +```java +record Inner(String innerValue) { + static final StructCodec CODEC = StructCodec.struct( + "inner_value", Codec.STRING, Inner::innerValue, + Inner::new + ); +} + +record Outer(String outerValue, Inner inner) { + static final StructCodec CODEC = StructCodec.struct( + "outer_value", Codec.STRING, Outer::outerValue, + StructCodec.INLINE, Inner.CODEC, Outer::inner, + Outer::new + ); +} +``` + +This produces `{"outer_value": "test", "inner_value": "innerValue"}` instead of `{"outer_value": "test", "inner": {"inner_value": "innerValue"}}`. + +## Error Handling +Codec operations return a `Result` type that represents either success or failure. Use pattern matching to handle both cases, or helper methods like `orElseThrow()` and `orElse()`. + +```java +Result result = PlayerData.CODEC.decode(Transcoder.JSON, json); + +if (result instanceof Result.Ok ok) { + PlayerData data = ok.value(); +} else if (result instanceof Result.Error error) { + player.sendMessage("Failed to decode your player data: " + error.message()); +} + +// If the data cannot be decoded successfully, throw a runtime exception +PlayerData data = result.orElseThrow(); + +// If the data cannot be decoded successfully, fallback to the default value +PlayerData data = result.orElse(defaultData); +``` + +## Transcoders +A transcoder bridges a codec to a specific file format. The two built-in ones are: +- `Transcoder.NBT`: Serializing to Minecraft NBT using the [Adventure](https://github.com/PaperMC/adventure) library +- `Transcoder.JSON`: Serializing to JSON files using the [GSON](https://github.com/google/gson) library + +> [!NOTE] +> Both of these libraries are built-in, so you don't have to worry about adding any dependencies to start using them. + +```java +PlayerData playerData = new PlayerData("Steve", 67, null); +JsonElement json = PlayerData.CODEC.encode(Transcoder.JSON, playerData).orElseThrow(); +BinaryTag nbt = PlayerData.CODEC.encode(Transcoder.NBT, playerData).orElseThrow(); +``` + +You can create your own transcoder, for example, one for reading YAML config files. +[ TODO: SnakeYAML example, probably link to a Gist ] + +## Saving to Files +### JSON +```java +void saveJson(PlayerData data, Path path) throws IOException { + JsonElement json = PlayerData.CODEC.encode(Transcoder.JSON, data).orElseThrow(); + String jsonString = new GsonBuilder().setPrettyPrinting().create().toJson(json); + Files.writeString(path, jsonString); +} + +PlayerData loadJson(Path path) throws IOException { + String jsonString = Files.readString(path); + JsonElement json = JsonParser.parseString(jsonString); + return PlayerData.CODEC.decode(Transcoder.JSON, json).orElseThrow(); +} +``` + +### NBT +```java +void saveNbt(PlayerData data, Path path) throws IOException { + CompoundBinaryTag nbt = (CompoundBinaryTag) PlayerData.CODEC + .encode(Transcoder.NBT, data) + .orElseThrow(); + + try (var output = Files.newOutputStream(path)) { + BinaryTagIO.writer().write(nbt, output); + } +} + +PlayerData loadNbt(Path path) throws IOException { + try (var input = Files.newInputStream(path)) { + CompoundBinaryTag nbt = BinaryTagIO.reader().read(input); + return PlayerData.CODEC.decode(Transcoder.NBT, nbt).orElseThrow(); + } +} +``` + +## Converting Between Formats +You can convert between formats using `RawValue` for direct conversion, or by decoding then encoding. + +```java +Codec.RawValue rawValue = Codec.RawValue.of(Transcoder.JSON, jsonElement); +BinaryTag nbt = rawValue.convertTo(Transcoder.NBT).orElseThrow(); + +PlayerData data = PlayerData.CODEC.decode(Transcoder.JSON, json).orElseThrow(); +BinaryTag nbt = PlayerData.CODEC.encode(Transcoder.NBT, data).orElseThrow(); +``` \ No newline at end of file diff --git a/docs/feature/serialization/network-buffers.md b/docs/feature/serialization/network-buffers.md new file mode 100644 index 0000000..1f84250 --- /dev/null +++ b/docs/feature/serialization/network-buffers.md @@ -0,0 +1,236 @@ +# Network Buffers +Network buffers read and write binary data sequentially. They maintain separate read and write positions and provide type-safe serialization. + +```java +NetworkBuffer buffer = NetworkBuffer.resizableBuffer(); + +// Writing +buffer.write(NetworkBuffer.STRING, "Hello"); +buffer.write(NetworkBuffer.VAR_INT, 42); +buffer.write(NetworkBuffer.UUID, playerId); + +// Reading +String message = buffer.read(NetworkBuffer.STRING); +int value = buffer.read(NetworkBuffer.VAR_INT); +UUID id = buffer.read(NetworkBuffer.UUID); +``` + +## Creating Buffers +```java +// Resizable buffer that grows automatically (default 256 bytes initial) +NetworkBuffer buffer = NetworkBuffer.resizableBuffer(); + +// Resizable with custom initial size +NetworkBuffer buffer = NetworkBuffer.resizableBuffer(1024); + +// Fixed-size buffer +NetworkBuffer buffer = NetworkBuffer.staticBuffer(512); + +// Wrap an existing byte array +byte[] data = new byte[]{1, 2, 3, 4}; +NetworkBuffer buffer = NetworkBuffer.wrap(data, 0, data.length); +``` + +## Built-in Types +### Primitives +| Type | Java Type | Size | Description | +| ---------------- | --------- | ---------- | --------------------------------- | +| `BOOLEAN` | `Boolean` | 1 byte | Boolean value | +| `BYTE` | `Byte` | 1 byte | Signed 8-bit integer | +| `UNSIGNED_BYTE` | `Short` | 1 byte | Unsigned 8-bit integer (0-255) | +| `SHORT` | `Short` | 2 bytes | Signed 16-bit integer | +| `UNSIGNED_SHORT` | `Integer` | 2 bytes | Unsigned 16-bit integer (0-65535) | +| `INT` | `Integer` | 4 bytes | Signed 32-bit integer | +| `UNSIGNED_INT` | `Long` | 4 bytes | Unsigned 32-bit integer | +| `LONG` | `Long` | 8 bytes | Signed 64-bit integer | +| `FLOAT` | `Float` | 4 bytes | 32-bit floating point | +| `DOUBLE` | `Double` | 8 bytes | 64-bit floating point | +| `VAR_INT` | `Integer` | 1-5 bytes | Variable-length integer | +| `VAR_LONG` | `Long` | 1-10 bytes | Variable-length long | + +VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte, larger values use up to 5 bytes (VAR_INT) or 10 bytes (VAR_LONG). + +### Strings and Text +| Type | Java Type | Description | +| -------------- | ------------------- | ---------------------------------------- | +| `STRING` | `String` | UTF-8 string with VAR_INT length prefix | +| `KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | +| `COMPONENT` | `Component` | Adventure text component | +| `NBT` | `BinaryTag` | NBT tag | +| `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | + +### Positions and Vectors +| Type | Java Type | Description | +| ---------------- | --------- | ---------------------------------------------------------- | +| `BLOCK_POSITION` | `Point` | Block coordinates packed into a long | +| `POS` | `Pos` | Position (double x, y, z) with rotation (float yaw, pitch) | +| `VECTOR3` | `Point` | Three floats (x, y, z) | +| `VECTOR3D` | `Point` | Three doubles (x, y, z) | + +### Arrays and Collections +| Type | Java Type | Description | +| ---------------- | --------- | -------------------------------------- | +| `BYTE_ARRAY` | `byte[]` | VAR_INT length, then bytes | +| `LONG_ARRAY` | `long[]` | VAR_INT length, then longs | +| `VAR_INT_ARRAY` | `int[]` | VAR_INT length, then VAR_INT elements | +| `VAR_LONG_ARRAY` | `long[]` | VAR_INT length, then VAR_LONG elements | +| `RAW_BYTES` | `byte[]` | All remaining readable bytes | + +### Other Types +| Type | Java Type | Description | +| ------------ | --------- | ----------------------------------- | +| `UUID` | `UUID` | UUID stored as two longs | +| `BITSET` | `BitSet` | Java BitSet | +| `INSTANT_MS` | `Instant` | Instant as milliseconds since epoch | + +## Transforming Types +`.transform()` converts between a network type and your custom type. + +```java +NetworkBuffer.Type POTION_TYPE = + NetworkBuffer.VAR_INT.transform( + PotionType::fromId, + PotionType::id + ); + +buffer.write(POTION_TYPE, potionType); +PotionType type = buffer.read(POTION_TYPE); +``` + +## Enums +Enums can be serialized by ordinal using `NetworkBuffer.Enum()`. Unlike `Codec.Enum()`, which encodes by name, this uses the enum's numeric ordinal. + +```java +NetworkBuffer.Type DIRECTION = NetworkBuffer.Enum(Direction.class); + +buffer.write(DIRECTION, Direction.NORTH); +Direction dir = buffer.read(DIRECTION); +``` + +For EnumSets, use `NetworkBuffer.EnumSet()`: + +```java +NetworkBuffer.Type> FEATURES = NetworkBuffer.EnumSet(Feature.class); +``` + +## Optional Types +`.optional()` wraps a type to allow null values. Writes a boolean (present/absent) followed by the value if present. + +```java +NetworkBuffer.Type<@Nullable Component> OPT_COMPONENT = + NetworkBuffer.COMPONENT.optional(); + +buffer.write(OPT_COMPONENT, null); +buffer.write(OPT_COMPONENT, someComponent); + +Component component = buffer.read(OPT_COMPONENT); +``` + +## Collections +`.list()` creates a list type. `.mapValue()` creates a map type with string keys. Both accept a maximum size to prevent malicious payloads. + +```java +NetworkBuffer.Type> STRING_LIST = + NetworkBuffer.STRING.list(Short.MAX_VALUE); + +buffer.write(STRING_LIST, List.of("a", "b", "c")); +List strings = buffer.read(STRING_LIST); + +NetworkBuffer.Type> STRING_INT_MAP = + NetworkBuffer.STRING.mapValue(NetworkBuffer.INT, Short.MAX_VALUE); + +buffer.write(STRING_INT_MAP, Map.of("health", 20, "armor", 5)); +``` + +## Templates +Templates serialize records. Alternate between type and getter, ending with the constructor. Supports up to 20 fields. + +```java +record Particle(Point position, int id, float scale) { + static final NetworkBuffer.Type NETWORK_TYPE = + NetworkBufferTemplate.template( + NetworkBuffer.BLOCK_POSITION, Particle::position, + NetworkBuffer.VAR_INT, Particle::id, + NetworkBuffer.FLOAT, Particle::scale, + Particle::new + ); +} + +buffer.write(Particle.NETWORK_TYPE, particle); +Particle particle = buffer.read(Particle.NETWORK_TYPE); +``` + +Templates support optional and nested types: + +```java +record PlayerInfo( + UUID uuid, + String name, + List properties, + @Nullable Component displayName, + int ping +) { + static final NetworkBuffer.Type NETWORK_TYPE = + NetworkBufferTemplate.template( + NetworkBuffer.UUID, PlayerInfo::uuid, + NetworkBuffer.STRING, PlayerInfo::name, + Property.NETWORK_TYPE.list(16), PlayerInfo::properties, + NetworkBuffer.COMPONENT.optional(), PlayerInfo::displayName, + NetworkBuffer.VAR_INT, PlayerInfo::ping, + PlayerInfo::new + ); +} +``` + +## Buffer Management +Query and modify read/write positions: + +```java +long writePos = buffer.writeIndex(); +long readPos = buffer.readIndex(); +buffer.writeIndex(100); +buffer.readIndex(50); + +buffer.advanceWrite(10); +buffer.advanceRead(5); + +long readable = buffer.readableBytes(); +long writable = buffer.writableBytes(); + +buffer.clear(); +``` + +## Read/Write at Position +Read or write at a specific index without moving the current read/write positions: + +```java +buffer.writeAt(100, NetworkBuffer.INT, 42); +int value = buffer.readAt(100, NetworkBuffer.INT); +``` + +## Extract Bytes +Serialize writes from a callback into a byte array: + +```java +byte[] bytes = buffer.extractBytes(buf -> { + buf.write(NetworkBuffer.VAR_INT, 42); + buf.write(NetworkBuffer.STRING, "test"); +}); +``` + +## Fixed-Size Types +Create types for fixed-size byte arrays and bitsets: + +```java +NetworkBuffer.Type BYTES_16 = NetworkBuffer.FixedRawBytes(16); +NetworkBuffer.Type BITSET_64 = NetworkBuffer.FixedBitSet(64); +``` + +## Either Types +Serialize one of two types. Uses a boolean tag to indicate which variant. + +```java +NetworkBuffer.Type> STRING_OR_INT = NetworkBuffer.Either(NetworkBuffer.STRING, NetworkBuffer.INT); +buffer.write(STRING_OR_INT, Either.left("hello")); +buffer.write(STRING_OR_INT, Either.right(67)); +``` \ No newline at end of file From 71f06344e216fe1879da645c6f83e6eb353ae26e Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Tue, 17 Mar 2026 03:25:00 -0600 Subject: [PATCH 2/7] more --- docs/feature/serialization/codecs.md | 55 +++++++------ docs/feature/serialization/network-buffers.md | 81 +++++++++++-------- 2 files changed, 79 insertions(+), 57 deletions(-) diff --git a/docs/feature/serialization/codecs.md b/docs/feature/serialization/codecs.md index af0eabd..a55eda0 100644 --- a/docs/feature/serialization/codecs.md +++ b/docs/feature/serialization/codecs.md @@ -18,29 +18,35 @@ PlayerData decodedData = PlayerData.CODEC.decode(Transcoder.JSON, json).orElseTh ``` ## Primitive Codecs -| Codec | Java Type | Description | -| ---------------------- | ------------------- | ---------------------------------------- | -| `Codec.BOOLEAN` | `Boolean` | Boolean value | -| `Codec.BYTE` | `Byte` | 8-bit integer | -| `Codec.SHORT` | `Short` | 16-bit integer | -| `Codec.INT` | `Integer` | 32-bit integer | -| `Codec.LONG` | `Long` | 64-bit integer | -| `Codec.FLOAT` | `Float` | 32-bit floating point | -| `Codec.DOUBLE` | `Double` | 64-bit floating point | -| `Codec.STRING` | `String` | UTF-8 string | -| `Codec.KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | -| `Codec.UUID` | `UUID` | UUID stored as int array | -| `Codec.UUID_STRING` | `UUID` | UUID stored as string | -| `Codec.COMPONENT` | `Component` | Adventure text component | -| `Codec.NBT` | `BinaryTag` | Any NBT tag | -| `Codec.NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | -| `Codec.BYTE_ARRAY` | `byte[]` | Byte array | -| `Codec.INT_ARRAY` | `int[]` | Integer array | -| `Codec.LONG_ARRAY` | `long[]` | Long array | -| `Codec.BLOCK_POSITION` | `Point` | Block coordinates | -| `Codec.VECTOR3D` | `Point` | Double precision coordinates | - -[ TODO: Item stack and other common codecs ] +| Codec | Java Type | Description | +| ----------------------- | ------------------- | ----------------------------------------------------------------------------------------- | +| `Codec.BOOLEAN` | `Boolean` | Boolean value | +| `Codec.BYTE` | `Byte` | 8-bit integer | +| `Codec.SHORT` | `Short` | 16-bit integer | +| `Codec.INT` | `Integer` | 32-bit integer | +| `Codec.LONG` | `Long` | 64-bit integer | +| `Codec.FLOAT` | `Float` | 32-bit floating point | +| `Codec.DOUBLE` | `Double` | 64-bit floating point | +| `Codec.STRING` | `String` | UTF-8 string | +| `Codec.KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | +| `Codec.UUID` | `UUID` | UUID stored as an integer array | +| `Codec.UUID_STRING` | `UUID` | UUID stored as string | +| `Codec.COMPONENT` | `Component` | Adventure text component | +| `Codec.NBT` | `BinaryTag` | Any NBT tag | +| `Codec.NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | +| `Codec.BYTE_ARRAY` | `byte[]` | Byte array | +| `Codec.INT_ARRAY` | `int[]` | Integer array | +| `Codec.LONG_ARRAY` | `long[]` | Long array | +| `Codec.BLOCK_POSITION` | `Point` | Block coordinates | +| `Codec.VECTOR3D` | `Point` | Double precision coordinates | +| `Codec.UNIT` | `Unit` | Represents the absence of a value (encodes to an empty object) | +| `Codec.TRI_STATE` | `TriState` | Three-state boolean: true, false, or absent | +| `Codec.UUID_COERCED` | `UUID` | UUID as integer array, falling back to string | +| `Codec.COMPONENT_STYLE` | `Style` | Adventure text style | +| `Codec.RAW_VALUE` | `RawValue` | Format-agnostic raw value (see [Converting Between Formats](#converting-between-formats)) | + +> [!NOTE] +> Codecs for game types are often defined on their respective classes rather than on `Codec` directly, such as `ItemStack.CODEC`. ## Transforming Types The `.transform()` method converts between types during encoding and decoding. This is useful for custom types that can be represented as a simpler type. @@ -191,7 +197,8 @@ BinaryTag nbt = PlayerData.CODEC.encode(Transcoder.NBT, playerData).orElseThrow( ``` You can create your own transcoder, for example, one for reading YAML config files. -[ TODO: SnakeYAML example, probably link to a Gist ] + + ## Saving to Files ### JSON diff --git a/docs/feature/serialization/network-buffers.md b/docs/feature/serialization/network-buffers.md index 1f84250..db51b7e 100644 --- a/docs/feature/serialization/network-buffers.md +++ b/docs/feature/serialization/network-buffers.md @@ -33,39 +33,50 @@ NetworkBuffer buffer = NetworkBuffer.wrap(data, 0, data.length); ## Built-in Types ### Primitives -| Type | Java Type | Size | Description | -| ---------------- | --------- | ---------- | --------------------------------- | -| `BOOLEAN` | `Boolean` | 1 byte | Boolean value | -| `BYTE` | `Byte` | 1 byte | Signed 8-bit integer | -| `UNSIGNED_BYTE` | `Short` | 1 byte | Unsigned 8-bit integer (0-255) | -| `SHORT` | `Short` | 2 bytes | Signed 16-bit integer | -| `UNSIGNED_SHORT` | `Integer` | 2 bytes | Unsigned 16-bit integer (0-65535) | -| `INT` | `Integer` | 4 bytes | Signed 32-bit integer | -| `UNSIGNED_INT` | `Long` | 4 bytes | Unsigned 32-bit integer | -| `LONG` | `Long` | 8 bytes | Signed 64-bit integer | -| `FLOAT` | `Float` | 4 bytes | 32-bit floating point | -| `DOUBLE` | `Double` | 8 bytes | 64-bit floating point | -| `VAR_INT` | `Integer` | 1-5 bytes | Variable-length integer | -| `VAR_LONG` | `Long` | 1-10 bytes | Variable-length long | +| Type | Java Type | Size | Description | +| ------------------ | ------------------- | ---------- | ------------------------------------------------- | +| `BOOLEAN` | `Boolean` | 1 byte | Boolean value | +| `BYTE` | `Byte` | 1 byte | Signed 8-bit integer | +| `UNSIGNED_BYTE` | `Short` | 1 byte | Unsigned 8-bit integer (0-255) | +| `SHORT` | `Short` | 2 bytes | Signed 16-bit integer | +| `UNSIGNED_SHORT` | `Integer` | 2 bytes | Unsigned 16-bit integer (0-65535) | +| `INT` | `Integer` | 4 bytes | Signed 32-bit integer | +| `UNSIGNED_INT` | `Long` | 4 bytes | Unsigned 32-bit integer | +| `LONG` | `Long` | 8 bytes | Signed 64-bit integer | +| `FLOAT` | `Float` | 4 bytes | 32-bit floating point | +| `DOUBLE` | `Double` | 8 bytes | 64-bit floating point | +| `VAR_INT` | `Integer` | 1-5 bytes | Variable-length integer | +| `VAR_LONG` | `Long` | 1-10 bytes | Variable-length long | +| `OPTIONAL_VAR_INT` | `@Nullable Integer` | 1-5 bytes | Nullable VAR_INT; encodes 0 for absent, n+1 for n | +| `VAR_INT_3` | `Integer` | 3 bytes | Fixed 3-byte VarInt, range 0–2²¹ | +| `UNIT` | `Unit` | 0 bytes | Represents the absence of a value | VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte, larger values use up to 5 bytes (VAR_INT) or 10 bytes (VAR_LONG). ### Strings and Text -| Type | Java Type | Description | -| -------------- | ------------------- | ---------------------------------------- | -| `STRING` | `String` | UTF-8 string with VAR_INT length prefix | -| `KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | -| `COMPONENT` | `Component` | Adventure text component | -| `NBT` | `BinaryTag` | NBT tag | -| `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | +| Type | Java Type | Description | +| ------------------- | ------------------- | ---------------------------------------------- | +| `STRING` | `String` | UTF-8 string with VAR_INT length prefix | +| `KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | +| `COMPONENT` | `Component` | Adventure text component | +| `NBT` | `BinaryTag` | NBT tag | +| `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | +| `JSON_COMPONENT` | `Component` | Adventure text component as JSON string | +| `STRING_TERMINATED` | `String` | Null-terminated string | +| `STRING_IO_UTF8` | `String` | UTF-8 string for stream I/O (no length prefix) | ### Positions and Vectors -| Type | Java Type | Description | -| ---------------- | --------- | ---------------------------------------------------------- | -| `BLOCK_POSITION` | `Point` | Block coordinates packed into a long | -| `POS` | `Pos` | Position (double x, y, z) with rotation (float yaw, pitch) | -| `VECTOR3` | `Point` | Three floats (x, y, z) | -| `VECTOR3D` | `Point` | Three doubles (x, y, z) | +| Type | Java Type | Description | +| -------------------- | ----------------- | ---------------------------------------------------------- | +| `BLOCK_POSITION` | `Point` | Block coordinates packed into a long | +| `OPT_BLOCK_POSITION` | `@Nullable Point` | Optional block coordinates | +| `POS` | `Pos` | Position (double x, y, z) with rotation (float yaw, pitch) | +| `VECTOR3` | `Point` | Three floats (x, y, z) | +| `VECTOR3D` | `Point` | Three doubles (x, y, z) | +| `VECTOR3I` | `Point` | Three VAR_INTs (x, y, z) | +| `VECTOR3B` | `Point` | Three signed bytes (x, y, z) | +| `LP_VECTOR3` | `Vec` | Lossy-precision quantized vector | +| `QUATERNION` | `float[]` | Rotation as four floats (x, y, z, w) | ### Arrays and Collections | Type | Java Type | Description | @@ -77,11 +88,15 @@ VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte | `RAW_BYTES` | `byte[]` | All remaining readable bytes | ### Other Types -| Type | Java Type | Description | -| ------------ | --------- | ----------------------------------- | -| `UUID` | `UUID` | UUID stored as two longs | -| `BITSET` | `BitSet` | Java BitSet | -| `INSTANT_MS` | `Instant` | Instant as milliseconds since epoch | +| Type | Java Type | Description | +| ------------ | --------------------- | ----------------------------------- | +| `UUID` | `UUID` | UUID stored as two longs | +| `BITSET` | `BitSet` | Java BitSet | +| `INSTANT_MS` | `Instant` | Instant as milliseconds since epoch | +| `OPT_CHAT` | `@Nullable Component` | Optional Adventure text component | +| `DIRECTION` | `Direction` | Cardinal direction (by ordinal) | +| `POSE` | `EntityPose` | Entity pose (by ordinal) | +| `PUBLIC_KEY` | `PublicKey` | RSA public key as byte array | ## Transforming Types `.transform()` converts between a network type and your custom type. @@ -227,7 +242,7 @@ NetworkBuffer.Type BITSET_64 = NetworkBuffer.FixedBitSet(64); ``` ## Either Types -Serialize one of two types. Uses a boolean tag to indicate which variant. +Serialize one of two types. Prefixed by a boolean/byte to indicate which variant. ```java NetworkBuffer.Type> STRING_OR_INT = NetworkBuffer.Either(NetworkBuffer.STRING, NetworkBuffer.INT); From 7797565a42fc04484c8056079b5aec12d4bab847 Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Sun, 22 Mar 2026 15:54:56 -0600 Subject: [PATCH 3/7] fix mistakes --- docs/feature/serialization/network-buffers.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/docs/feature/serialization/network-buffers.md b/docs/feature/serialization/network-buffers.md index db51b7e..06cf5a2 100644 --- a/docs/feature/serialization/network-buffers.md +++ b/docs/feature/serialization/network-buffers.md @@ -63,7 +63,7 @@ VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte | `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | | `JSON_COMPONENT` | `Component` | Adventure text component as JSON string | | `STRING_TERMINATED` | `String` | Null-terminated string | -| `STRING_IO_UTF8` | `String` | UTF-8 string for stream I/O (no length prefix) | +| `STRING_IO_UTF8` | `String` | Modified UTF-8 string for stream I/O with a 2-byte `SHORT` length prefix | ### Positions and Vectors | Type | Java Type | Description | @@ -224,10 +224,19 @@ int value = buffer.readAt(100, NetworkBuffer.INT); ``` ## Extract Bytes -Serialize writes from a callback into a byte array: +Extract the bytes consumed while a callback advances the read index: ```java byte[] bytes = buffer.extractBytes(buf -> { + buf.read(NetworkBuffer.VAR_INT); + buf.read(NetworkBuffer.STRING); +}); +``` + +To serialize writes into a new byte array, use `NetworkBuffer.makeArray(...)` instead: + +```java +byte[] bytes = NetworkBuffer.makeArray(buf -> { buf.write(NetworkBuffer.VAR_INT, 42); buf.write(NetworkBuffer.STRING, "test"); }); @@ -248,4 +257,4 @@ Serialize one of two types. Prefixed by a boolean/byte to indicate which variant NetworkBuffer.Type> STRING_OR_INT = NetworkBuffer.Either(NetworkBuffer.STRING, NetworkBuffer.INT); buffer.write(STRING_OR_INT, Either.left("hello")); buffer.write(STRING_OR_INT, Either.right(67)); -``` \ No newline at end of file +``` From 090558d9a7ae08643f90a82e25922b026f3592ef Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Sun, 22 Mar 2026 16:01:40 -0600 Subject: [PATCH 4/7] fix more mistakes --- docs/feature/serialization/network-buffers.md | 30 +++++++++++-------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/feature/serialization/network-buffers.md b/docs/feature/serialization/network-buffers.md index 06cf5a2..1650fca 100644 --- a/docs/feature/serialization/network-buffers.md +++ b/docs/feature/serialization/network-buffers.md @@ -26,11 +26,13 @@ NetworkBuffer buffer = NetworkBuffer.resizableBuffer(1024); // Fixed-size buffer NetworkBuffer buffer = NetworkBuffer.staticBuffer(512); -// Wrap an existing byte array +// Create a fixed-size buffer initialized from a byte array byte[] data = new byte[]{1, 2, 3, 4}; NetworkBuffer buffer = NetworkBuffer.wrap(data, 0, data.length); ``` +`NetworkBuffer.wrap(...)` copies the provided bytes into the buffer; it does not share the original `byte[]`. + ## Built-in Types ### Primitives | Type | Java Type | Size | Description | @@ -48,7 +50,7 @@ NetworkBuffer buffer = NetworkBuffer.wrap(data, 0, data.length); | `VAR_INT` | `Integer` | 1-5 bytes | Variable-length integer | | `VAR_LONG` | `Long` | 1-10 bytes | Variable-length long | | `OPTIONAL_VAR_INT` | `@Nullable Integer` | 1-5 bytes | Nullable VAR_INT; encodes 0 for absent, n+1 for n | -| `VAR_INT_3` | `Integer` | 3 bytes | Fixed 3-byte VarInt, range 0–2²¹ | +| `VAR_INT_3` | `Integer` | 3 bytes | Fixed 3-byte VarInt, range 0–2²¹-1 | | `UNIT` | `Unit` | 0 bytes | Represents the absence of a value | VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte, larger values use up to 5 bytes (VAR_INT) or 10 bytes (VAR_LONG). @@ -58,12 +60,12 @@ VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte | ------------------- | ------------------- | ---------------------------------------------- | | `STRING` | `String` | UTF-8 string with VAR_INT length prefix | | `KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | -| `COMPONENT` | `Component` | Adventure text component | +| `COMPONENT` | `Component` | Adventure text component in the standard network format | | `NBT` | `BinaryTag` | NBT tag | | `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | | `JSON_COMPONENT` | `Component` | Adventure text component as JSON string | -| `STRING_TERMINATED` | `String` | Null-terminated string | -| `STRING_IO_UTF8` | `String` | Modified UTF-8 string for stream I/O with a 2-byte `SHORT` length prefix | +| `STRING_TERMINATED` | `String` | Null-terminated UTF-8 string | +| `STRING_IO_UTF8` | `String` | Modified UTF-8 string for stream I/O with a 2-byte length prefix (`DataOutputStream.writeUTF` compatible) | ### Positions and Vectors | Type | Java Type | Description | @@ -94,7 +96,7 @@ VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte | `BITSET` | `BitSet` | Java BitSet | | `INSTANT_MS` | `Instant` | Instant as milliseconds since epoch | | `OPT_CHAT` | `@Nullable Component` | Optional Adventure text component | -| `DIRECTION` | `Direction` | Cardinal direction (by ordinal) | +| `DIRECTION` | `Direction` | Direction enum (including up/down, by ordinal) | | `POSE` | `EntityPose` | Entity pose (by ordinal) | | `PUBLIC_KEY` | `PublicKey` | RSA public key as byte array | @@ -113,7 +115,7 @@ PotionType type = buffer.read(POTION_TYPE); ``` ## Enums -Enums can be serialized by ordinal using `NetworkBuffer.Enum()`. Unlike `Codec.Enum()`, which encodes by name, this uses the enum's numeric ordinal. +Enums can be serialized by ordinal using `NetworkBuffer.Enum()`. Unlike `Codec.Enum()`, which encodes by name, this uses the enum's numeric ordinal as a `VAR_INT`. ```java NetworkBuffer.Type DIRECTION = NetworkBuffer.Enum(Direction.class); @@ -138,11 +140,11 @@ NetworkBuffer.Type<@Nullable Component> OPT_COMPONENT = buffer.write(OPT_COMPONENT, null); buffer.write(OPT_COMPONENT, someComponent); -Component component = buffer.read(OPT_COMPONENT); +@Nullable Component component = buffer.read(OPT_COMPONENT); ``` ## Collections -`.list()` creates a list type. `.mapValue()` creates a map type with string keys. Both accept a maximum size to prevent malicious payloads. +`.list()` creates a list type. `.mapValue()` creates a map type whose keys use the receiver type and whose values use the provided type. Both accept a maximum size used during reads to reject oversized payloads. ```java NetworkBuffer.Type> STRING_LIST = @@ -158,7 +160,7 @@ buffer.write(STRING_INT_MAP, Map.of("health", 20, "armor", 5)); ``` ## Templates -Templates serialize records. Alternate between type and getter, ending with the constructor. Supports up to 20 fields. +Templates serialize structured objects such as records. Alternate between type and getter pairs, ending with a constructor or factory function. Overloads are provided for up to 20 fields. ```java record Particle(Point position, int id, float scale) { @@ -242,16 +244,18 @@ byte[] bytes = NetworkBuffer.makeArray(buf -> { }); ``` -## Fixed-Size Types -Create types for fixed-size byte arrays and bitsets: +## Fixed-Length Bytes and Bounded BitSets +Create fixed-length byte-array types and bitset types with a maximum logical size: ```java NetworkBuffer.Type BYTES_16 = NetworkBuffer.FixedRawBytes(16); NetworkBuffer.Type BITSET_64 = NetworkBuffer.FixedBitSet(64); ``` +`FixedBitSet(length)` limits the highest set bit and reads up to `(length + 7) / 8` bytes; it does not pad writes to a fixed-width byte array. + ## Either Types -Serialize one of two types. Prefixed by a boolean/byte to indicate which variant. +Serialize one of two types. Prefixed by a boolean: `true` for the left variant, `false` for the right variant. ```java NetworkBuffer.Type> STRING_OR_INT = NetworkBuffer.Either(NetworkBuffer.STRING, NetworkBuffer.INT); From 27747496125c16c2021a0e88d381aa1bfd8c4fbb Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Sun, 22 Mar 2026 16:02:58 -0600 Subject: [PATCH 5/7] fix formatting --- docs/feature/serialization/network-buffers.md | 34 +++++++++---------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/feature/serialization/network-buffers.md b/docs/feature/serialization/network-buffers.md index 1650fca..cc7479f 100644 --- a/docs/feature/serialization/network-buffers.md +++ b/docs/feature/serialization/network-buffers.md @@ -56,15 +56,15 @@ NetworkBuffer buffer = NetworkBuffer.wrap(data, 0, data.length); VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte, larger values use up to 5 bytes (VAR_INT) or 10 bytes (VAR_LONG). ### Strings and Text -| Type | Java Type | Description | -| ------------------- | ------------------- | ---------------------------------------------- | -| `STRING` | `String` | UTF-8 string with VAR_INT length prefix | -| `KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | -| `COMPONENT` | `Component` | Adventure text component in the standard network format | -| `NBT` | `BinaryTag` | NBT tag | -| `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | -| `JSON_COMPONENT` | `Component` | Adventure text component as JSON string | -| `STRING_TERMINATED` | `String` | Null-terminated UTF-8 string | +| Type | Java Type | Description | +| ------------------- | ------------------- | --------------------------------------------------------------------------------------------------------- | +| `STRING` | `String` | UTF-8 string with VAR_INT length prefix | +| `KEY` | `Key` | Namespaced key (e.g., `minecraft:stone`) | +| `COMPONENT` | `Component` | Adventure text component in the standard network format | +| `NBT` | `BinaryTag` | NBT tag | +| `NBT_COMPOUND` | `CompoundBinaryTag` | NBT compound tag | +| `JSON_COMPONENT` | `Component` | Adventure text component as JSON string | +| `STRING_TERMINATED` | `String` | Null-terminated UTF-8 string | | `STRING_IO_UTF8` | `String` | Modified UTF-8 string for stream I/O with a 2-byte length prefix (`DataOutputStream.writeUTF` compatible) | ### Positions and Vectors @@ -90,15 +90,15 @@ VAR_INT and VAR_LONG encode small values in fewer bytes. Values 0-127 use 1 byte | `RAW_BYTES` | `byte[]` | All remaining readable bytes | ### Other Types -| Type | Java Type | Description | -| ------------ | --------------------- | ----------------------------------- | -| `UUID` | `UUID` | UUID stored as two longs | -| `BITSET` | `BitSet` | Java BitSet | -| `INSTANT_MS` | `Instant` | Instant as milliseconds since epoch | -| `OPT_CHAT` | `@Nullable Component` | Optional Adventure text component | +| Type | Java Type | Description | +| ------------ | --------------------- | ---------------------------------------------- | +| `UUID` | `UUID` | UUID stored as two longs | +| `BITSET` | `BitSet` | Java BitSet | +| `INSTANT_MS` | `Instant` | Instant as milliseconds since epoch | +| `OPT_CHAT` | `@Nullable Component` | Optional Adventure text component | | `DIRECTION` | `Direction` | Direction enum (including up/down, by ordinal) | -| `POSE` | `EntityPose` | Entity pose (by ordinal) | -| `PUBLIC_KEY` | `PublicKey` | RSA public key as byte array | +| `POSE` | `EntityPose` | Entity pose (by ordinal) | +| `PUBLIC_KEY` | `PublicKey` | RSA public key as byte array | ## Transforming Types `.transform()` converts between a network type and your custom type. From c45886d5d56867aa87aa4bc0da71ba708cef639b Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Tue, 24 Mar 2026 20:02:02 -0600 Subject: [PATCH 6/7] use new callouts --- docs/feature/serialization/codecs.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/feature/serialization/codecs.md b/docs/feature/serialization/codecs.md index a55eda0..63fc5e6 100644 --- a/docs/feature/serialization/codecs.md +++ b/docs/feature/serialization/codecs.md @@ -45,8 +45,9 @@ PlayerData decodedData = PlayerData.CODEC.decode(Transcoder.JSON, json).orElseTh | `Codec.COMPONENT_STYLE` | `Style` | Adventure text style | | `Codec.RAW_VALUE` | `RawValue` | Format-agnostic raw value (see [Converting Between Formats](#converting-between-formats)) | -> [!NOTE] -> Codecs for game types are often defined on their respective classes rather than on `Codec` directly, such as `ItemStack.CODEC`. +:::note +Codecs for game types are often defined on their respective classes rather than on `Codec` directly, such as `ItemStack.CODEC`. +::: ## Transforming Types The `.transform()` method converts between types during encoding and decoding. This is useful for custom types that can be represented as a simpler type. @@ -187,8 +188,9 @@ A transcoder bridges a codec to a specific file format. The two built-in ones ar - `Transcoder.NBT`: Serializing to Minecraft NBT using the [Adventure](https://github.com/PaperMC/adventure) library - `Transcoder.JSON`: Serializing to JSON files using the [GSON](https://github.com/google/gson) library -> [!NOTE] -> Both of these libraries are built-in, so you don't have to worry about adding any dependencies to start using them. +:::note +Both of these libraries are built-in, so you don't have to worry about adding any dependencies to start using them. +::: ```java PlayerData playerData = new PlayerData("Steve", 67, null); @@ -196,7 +198,9 @@ JsonElement json = PlayerData.CODEC.encode(Transcoder.JSON, playerData).orElseTh BinaryTag nbt = PlayerData.CODEC.encode(Transcoder.NBT, playerData).orElseThrow(); ``` +:::tip You can create your own transcoder, for example, one for reading YAML config files. +::: From d8f2650bc9166b73dd8959e5ac2cbbc91953c574 Mon Sep 17 00:00:00 2001 From: mudkipdev Date: Tue, 24 Mar 2026 20:07:47 -0600 Subject: [PATCH 7/7] upd --- docs/feature/serialization/codecs.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/feature/serialization/codecs.md b/docs/feature/serialization/codecs.md index 63fc5e6..e8e9739 100644 --- a/docs/feature/serialization/codecs.md +++ b/docs/feature/serialization/codecs.md @@ -188,9 +188,7 @@ A transcoder bridges a codec to a specific file format. The two built-in ones ar - `Transcoder.NBT`: Serializing to Minecraft NBT using the [Adventure](https://github.com/PaperMC/adventure) library - `Transcoder.JSON`: Serializing to JSON files using the [GSON](https://github.com/google/gson) library -:::note Both of these libraries are built-in, so you don't have to worry about adding any dependencies to start using them. -::: ```java PlayerData playerData = new PlayerData("Steve", 67, null); @@ -199,7 +197,7 @@ BinaryTag nbt = PlayerData.CODEC.encode(Transcoder.NBT, playerData).orElseThrow( ``` :::tip -You can create your own transcoder, for example, one for reading YAML config files. +You can create your own transcoder, for example, one for reading YAML configuration files. :::