Skip to content

use liblzma(xz) to replace lzma sdk#3065

Merged
mercury233 merged 12 commits into
Fluorohydride:masterfrom
mercury233:patch-lzma-xz
May 25, 2026
Merged

use liblzma(xz) to replace lzma sdk#3065
mercury233 merged 12 commits into
Fluorohydride:masterfrom
mercury233:patch-lzma-xz

Conversation

@mercury233

@mercury233 mercury233 commented Apr 23, 2026

Copy link
Copy Markdown
Collaborator

close #2992

The LZMA SDK we are currently using is a very old version and has the known issues mentioned above.

Upgrading to the newer LZMA SDK appears to be less suitable than using liblzma (xz).

The current compression algorithm does not add an EOPM (end marker), but it can read data both with and without EOPM. Therefore, LZMA_FILTER_LZMA1EXT is required to maintain the same behavior.

The liblzma-dev package provided via apt on Ubuntu 22 is too outdated and does not support LZMA_FILTER_LZMA1EXT, so it needs to be built from source. This has not yet been implemented separately.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates replay compression/decompression from the bundled legacy LZMA SDK to liblzma (xz), aiming to avoid known issues in the old SDK and preserve replay compatibility (EOPM/no-EOPM) via the LZMA1EXT filter.

Changes:

  • Switch replay compression/decompression implementation to liblzma raw encode/decode.
  • Remove the in-tree LZMA SDK wrapper/library and update Premake linkage to lzma.
  • Update CI workflow to download/extract xz source for static builds and install liblzma-dev for dynamic builds.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
premake5.lua Adds BUILD_LZMA options, default include path, and conditionally includes a lzma premake project.
gframe/replay.cpp Replaces LzmaLib calls with liblzma raw encode/decode using LZMA1EXT.
gframe/premake5.lua Updates include dirs and link target from clzma to lzma, and adds LZMA_LIB_DIR handling.
gframe/lzma/* Removes the legacy bundled LZMA SDK implementation and build files.
.github/workflows/build.yml Downloads/extracts xz sources for static builds and installs liblzma-dev for dynamic Linux builds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread gframe/replay.cpp
Comment thread .github/workflows/build.yml
Comment thread gframe/premake5.lua Outdated
Comment thread premake5.lua
Comment thread premake5.lua
Comment thread gframe/replay.cpp
#include "deck_manager.h"
#include "lzma/LzmaLib.h"

#define LZMA_API_STATIC

Copilot AI Apr 23, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#define LZMA_API_STATIC is hard-coded in the source file. This will be wrong if someone builds against a shared/dynamic liblzma (e.g., --no-build-lzma on Windows), and it also makes the linkage decision impossible to control from the build system. Prefer passing LZMA_API_STATIC as a build define only for static-link configurations, and remove the source-level #define.

Suggested change
#define LZMA_API_STATIC

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will fix it together with _IRR_STATIC_LIB_, as they only affect MSVC

Comment thread gframe/replay.cpp
@purerosefallen

Copy link
Copy Markdown
Collaborator

The liblzma-dev package provided via apt on Ubuntu 22 is too outdated and does not support LZMA_FILTER_LZMA1EXT, so it needs to be built from source. This has not yet been implemented separately.

How about to make it build from source as default? If it requires a very new version, then it's not suitable to be from apt

@edo9300

edo9300 commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Old liblzma versions can be used as well with the right tinkering edo9300/edopro#241

@mercury233 mercury233 marked this pull request as ready for review April 25, 2026 09:27
@mercury233

Copy link
Copy Markdown
Collaborator Author

If it requires a very new version

5.4.0 (2022-12-13)

@purerosefallen

purerosefallen commented May 6, 2026

Copy link
Copy Markdown
Collaborator

side note:
server deployments should be considered as most servers are around Debian 11

Dropping support to srvpro and ask them to migrate to srvpro2

@purerosefallen

Copy link
Copy Markdown
Collaborator

Does it change yrp format?

@mercury233

Copy link
Copy Markdown
Collaborator Author

Does it change yrp format?

It shouldn't

@purerosefallen

Copy link
Copy Markdown
Collaborator

note:
the server should work with old lzma, with marker
at least to make the replay writing work

@mercury233 mercury233 merged commit a4c09fb into Fluorohydride:master May 25, 2026
15 checks passed
@salix5

salix5 commented May 25, 2026

Copy link
Copy Markdown
Collaborator

https://github.com/tukaani-project/xz/releases/tag/v5.8.3
xz-5.8.3.tar.gz
sha256:3d3a1b973af218114f4f889bbaa2f4c037deaae0c8e815eec381c3d546b974a0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants