libs : rename libcommon -> libllama-common

[ggerganov]

Overview

• Rename libcommon -> libllama-common
• Rename libbuild_info -> libllama-common-base
• Allow libllama-common to be shared (based on BUILD_SHARED_LIBS)
• Install alongside libllama

Additional information

This change reduces the output binaries size by making all binaries dynamically link to libllama-common.

Ideally, libllama-common should not be installed in the public lib path as it is not meant for 3rd-party usage. However, I'm not sure what is the canonical way to do that. For now, installing it the exact way as libllama using the same versioning scheme.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[ckastner]

For now, installing it the exact way as libllama using the same versioning scheme.

I think it would be OK to get the first version out like this, if that keeps things simpler for you, and to improve upon this later.

Improvement: the canonical way to do this would be to

1. Place this shared library in a non-public search path, typically /usr/lib/<package> or /usr/lib/<gnu-triplet>/<package>
2. Set RPATH/RUNPATH on relevant binaries so they can find it

You can see this with systemd, for example.

$ ldd /usr/bin/journalctl | grep libsystemd-shared
libsystemd-shared-255.so => /usr/lib/x86_64-linux-gnu/systemd/libsystemd-shared-255.so (0x00007e3a8167f000)

$ readelf -d /usr/bin/journalctl | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [/usr/lib/x86_64-linux-gnu/systemd]

Since Debian requires libraries in public paths to have stable ABI, we maintain a downstream-only patch that currently makes all llama.cpp libraries private following the example above (see our libllama0 file list). And at build time, all we do is pass -DCMAKE_INSTALL_RPATH=<private-dir>.

If you intend to implement this improvement, perhaps we can find a general solution that makes our patch obsolete.

[We never intended to upstream our patch as it goes too far for you, by hiding libllama0 that users actually want. In Debian, users can easily un-hide it through the pkg-config and cmake configurations that we provide in (non-default) search paths.]

[arthw]

I tested for SYCL backend.
It's OK for SYCL backend.

Thank you!

[ngxson]

A bit related to this, I've been thinking about moving server-context to a new lib named libllama-engine, the idea will be similar to llamax that you brought up before. I can create a dedicated issue for discussion if it's necessary.

The current PR starts to introduce the naming scheme libllama-something which is aligned with my point above.

Re. your concern about not allowing 3rd-party to use the libllama-common, a simple hack could be to have a #ifndef USE_INTERNAL_LIBCOMMON that the cpp code need to define before including the header, so if user doesn't include it, it will show an error. It's similar to mtmd-helper:

llama.cpp/tools/mtmd/mtmd-helper.cpp

Lines 35 to 37 in 8dc530b

	#ifdef MTMD_INTERNAL_HEADER
	#error "mtmd-helper is a public library outside of mtmd. it must not include internal headers"
	#endif

[ggerganov]

Improvement: the canonical way to do this would be to

1. Place this shared library in a non-public search path, typically /usr/lib/<package> or /usr/lib/<gnu-triplet>/<package>
2. Set RPATH/RUNPATH on relevant binaries so they can find it

Got it, thanks. We can do that in the next iteration.

If you intend to implement this improvement, perhaps we can find a general solution that makes our patch obsolete.

I guess to do that we need to automate the ABI stability validation on our end. We basically need to add a workflow to llama.cpp repo that would compare the current ABI with the latest released ABI and raise an error if there is backwards incompatibility. Probably something that we should address soon.

A bit related to this, I've been thinking about moving server-context to a new lib named libllama-engine, the idea will be similar to llamax that you brought up before. I can create a dedicated issue for discussion if it's necessary.

Yes, we need to reorganize the libllama-common and the server code in such a way that for example llama-cli should not require to link to httplib and still be able to use the "server engine" underneath.

Re. your concern about not allowing 3rd-party to use the libllama-common

It's not a problem. The only annoyance is that users will see a libllama-common.so in /usr/lib without having a header file to interface with it, but will fix it as @ckastner explained above.

[ckastner]

I guess to do that we need to automate the ABI stability validation on our end. We basically need to add a workflow to llama.cpp repo that would compare the current ABI with the latest released ABI and raise an error if there is backwards incompatibility. Probably something that we should address soon.

I had this on my TODO list, actually in both directions

1. Find ABI break with abi-compliance-checker or libabigail
2. Find new ABI features (backwards-compatible)

Point 2. would be a common occurence. It's when llama.cpp needs a newer ggml for new symbols/features but doesn't break existing ones. Which means a new ggml minor release is needed.

[ngxson]

Yes, we need to reorganize the libllama-common and the server code in such a way that for example llama-cli should not require to link to httplib and still be able to use the "server engine" underneath.

Forgot to mention, we're having a discussion in #21674 (especially the comment #21674 (comment)) about the direction of moving CLI to using HTTP instead native "server engine" version. Kindly asking if you can share you POV the discussion when you have time. Thanks in advance!

[CMay]

This change reduces the output binaries size by making all binaries dynamically link to libllama-common.

Smaller binaries are appealing. Some of the larger backends may not even notice, but Vulkan might.

I guess to do that we need to automate the ABI stability validation on our end. We basically need to add a workflow to llama.cpp repo that would compare the current ABI with the latest released ABI and raise an error if there is backwards incompatibility. Probably something that we should address soon.

This is one of the most exciting things I've heard out of llama.cpp development in a while. Llama.cpp interests me as a library more than a client front end or inference web API host, but I've got old code that doesn't work with the newer versions of the library and haven't gotten around to fixing it.

It does make me wonder about the balance between the flexibility of exposing new things that newer models use versus abstracting those so that old code can use drop-in replacements of llama.cpp and benefit from newer models without code changes.

Like a basic default usage path and a more manual usage path that allows more tinkering. Is an approach like that even compatible with aiming for stability without leaving a graveyard of old public methods? Would you have to create a kind of feature router that people can use to indirectly call a subset of private methods by request until they're stabilized and made public?

[ggerganov]

@ckastner If you have any suggestions how to implement the workflow let us know. I tried to prototype something that extracts the current ABI, but it boils down to parsing the header file which seems a bit fragile. Maybe the tools that you mentioned would be better suited for that.

@ngxson Yes, generally decoupling the communication layer from the logic (both server and client) is a good idea in terms of the implementation. The server-side though is more important IMO atm.

@CMay I'm not sure I follow everything, but the bottom line is that when we transition to semantic versioning it should generally be easier for 3rd parties using llama.cpp. We just need an automation for that because it is difficult to keep track of all the changes.

Would you have to create a kind of feature router that people can use to indirectly call a subset of private methods by request until they're stabilized and made public?

No, this sounds too complicated. We want to expose only functionality that will likely remain stable long term. Experiments can be done in tools/examples in the repo or in forks.