Skip to content

Rework protocol IO, add Buffer abstraction & fix socket write issues#1127

Merged
ItsDrike merged 10 commits intomainfrom
update-io-logic
Apr 18, 2026
Merged

Rework protocol IO, add Buffer abstraction & fix socket write issues#1127
ItsDrike merged 10 commits intomainfrom
update-io-logic

Conversation

@ItsDrike
Copy link
Copy Markdown
Member

@ItsDrike ItsDrike commented Mar 5, 2026

The previous protocol implementation mixed multiple responsibilities in Connection. It acted both as a network transport and as an in-memory buffer for packet construction/parsing. This made the code confusing, hard to reason about, and resulted in duplicated serialization logic.

This change restructures the protocol IO layer and cleans up the design:

  • Extract binary read/write primitives into _protocol.io.base_io (BaseSync/AsyncReader and BaseSync/AsyncWriter).
  • Introduce a dedicated Buffer type for in-memory packet construction and decoding instead of abusing connection objects as buffers.
  • Move socket implementations into _protocol.io.connection.
  • Update protocol clients to use the new IO abstractions and the StructFormat helpers for typed serialization.

Besides the structural cleanup, this also fixes several important issues in the previous implementation:

  • The synchronous TCP connection used socket.send(data), which does not guarantee that all bytes are sent. The new implementation correctly uses socket.sendall().

  • The asynchronous TCP write method was synchronous. It was just calling writer.write(data) but never awaited drain(), meaning the actual network write could happen at an arbitrary later time. In practice this rarely broke because a read always followed the write, which implicitly allowed the event loop to flush the buffer, but the behavior was still incorrect and fragile. The new async connection now uses async write method and awaits writer.drain() to ensure proper write semantics.

Inspiration

A lot of the new design takes direct inspiration from how mcproto handles protocol interactions.

@ItsDrike ItsDrike added a: protocol Related to underlying networking protocol t: revision Complete or partial rewrite of something (code cleanup, performance improvements, etc.) labels Mar 5, 2026
Comment thread mcstatus/_protocol/io/base_io.py
Comment thread mcstatus/_protocol/io/connection.py
Comment thread mcstatus/_protocol/io/buffer.py Outdated
@ItsDrike ItsDrike force-pushed the update-io-logic branch 3 times, most recently from cc85e88 to 33da8fc Compare March 26, 2026 21:47
@ItsDrike ItsDrike marked this pull request as ready for review March 26, 2026 21:57
Comment thread tests/protocol/test_async_support.py Outdated
@ItsDrike ItsDrike force-pushed the update-io-logic branch 2 times, most recently from b3be0f4 to 542c1d0 Compare March 26, 2026 22:32
Comment thread tests/test_server.py
Comment thread tests/test_server.py
Comment thread tests/test_server.py
ItsDrike added 10 commits April 18, 2026 14:13
The previous protocol implementation mixed multiple responsibilities in
`Connection`. It acted both as a network transport and as an in-memory
buffer for packet construction/parsing. This made the code confusing,
hard to reason about, and resulted in duplicated serialization logic.

This change restructures the protocol IO layer and cleans up the design:

- Extract binary read/write primitives into `_protocol.io.base_io`
  (BaseSync/AsyncReader and BaseSync/AsyncWriter).
- Introduce a dedicated `Buffer` type for in-memory packet construction
  and decoding instead of abusing connection objects as buffers.
- Move socket implementations into `_protocol.io.connection`.
- Update protocol clients to use the new IO abstractions and the
  `StructFormat` helpers for typed serialization.

Besides the structural cleanup, this also fixes several important issues
in the previous implementation:

* The synchronous TCP connection used `socket.send(data)`, which does not
  guarantee that all bytes are sent. The new implementation correctly uses
  `socket.sendall()`.

* The asynchronous TCP write method was synchronous. It was just calling
  `writer.write(data)` but never awaited `drain()`, meaning the actual
  network write could happen at an arbitrary later time. In practice this
  rarely broke because a read always followed the write, which implicitly
  allowed the event loop to flush the buffer, but the behavior was still
  incorrect and fragile. The new async connection now uses async write
  method and awaits `writer.drain()` to ensure proper write semantics.
This was not used anywhere anyways and it's better to force being
explicit than to "magically" encode into utf-8.
Copy link
Copy Markdown
Member

@PerchunPak PerchunPak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread tests/protocol/test_java_client.py
@ItsDrike ItsDrike merged commit d5bb9b4 into main Apr 18, 2026
10 checks passed
@ItsDrike ItsDrike deleted the update-io-logic branch April 18, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: protocol Related to underlying networking protocol t: revision Complete or partial rewrite of something (code cleanup, performance improvements, etc.)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants