Base58 encoding loses leading zero bytes

[sumanjeet0012]

BaseStringConverter.encode() converts bytes to an integer before encoding, which silently drops leading \x00 bytes. For base58btc and base58flickr, this means leading 1 characters (which represent \x00 bytes in base58) are lost, producing incorrect output that cannot round-trip.

Problem

In multibase/converters.py, BaseStringConverter.encode() works by converting the entire byte string to a single integer:

class BaseStringConverter(BaseConverter):
    def encode(self, bytes):
        number = int.from_bytes(bytes, byteorder="big", signed=False)
        return ensure_bytes(super().encode(number))

This loses leading zero bytes because int.from_bytes(b'\x00\x00hello') == int.from_bytes(b'hello'):

from multibase import encode, decode

# Data with leading zero bytes
data = b'\x00\x00hello'

# Encode then decode — leading zeros are lost!
encoded = encode("base58btc", data)
decoded = decode(encoded)
assert decoded == data  # FAILS: decoded is b'hello', not b'\x00\x00hello'

In base58btc, each leading \x00 byte should be represented as a leading 1 character. Go's implementation (using mr-tron/base58) handles this correctly:

// Go: leading zeros are preserved
data := []byte{0x00, 0x00, 'h', 'e', 'l', 'l', 'o'}
encoded := b58.EncodeAlphabet(data, b58.BTCAlphabet)
// encoded starts with "11..."

This affects base58btc, base58flickr, base36, base36upper, base32z, base2, base8, and base10 — all encodings that use BaseStringConverter.

Proposed Solution

Override encode() and decode() in BaseStringConverter to count and preserve leading zero bytes:

class BaseStringConverter(BaseConverter):
    def encode(self, bytes):
        # Count leading zero bytes
        leading_zeros = 0
        for b in bytes:
            if b == 0:
                leading_zeros += 1
            else:
                break

        number = int.from_bytes(bytes, byteorder="big", signed=False)
        encoded = ensure_bytes(super().encode(number))

        # Prepend the zero-character for each leading zero byte
        zero_char = ensure_bytes(self.digits[0])
        return zero_char * leading_zeros + encoded

    def decode(self, bytes):
        # Count leading zero characters
        zero_char = self.digits[0]
        leading_zeros = 0
        for ch in bytes.decode("utf-8") if isinstance(bytes, (bytes, bytearray)) else bytes:
            if ch == zero_char:
                leading_zeros += 1
            else:
                break

        decoded_int = self.bytes_to_int(bytes)
        decoded_data = decoded_int.to_bytes(
            (decoded_int.bit_length() + 7) // 8, byteorder="big"
        )

        return b'\x00' * leading_zeros + decoded_data

Add round-trip tests with leading zero bytes for all affected encodings.

Related

• Go implementation: go-multibase multibase.go uses mr-tron/base58 which preserves leading zeros
• Affected encodings: base2, base8, base10, base32z, base36, base36upper, base58btc, base58flickr
• Reference: Bitcoin base58 spec