Llama LSP Server

A Language Server Protocol server providing inline code completions using a llama infill endpoint. Requires Deno, a running llama.cpp server with a model supporting the /infill endpoint, and an LSP client that supports textDocument/inlineCompletion.

Configuration

All settings must be nested under the llamaLsp section (snake_case).

Settings

• llama_endpoint (default: http://127.0.0.1:8012/infill)
• n_predict (default: 128)
• temperature (default: 0.0)
• top_k (default: 40)
• top_p (default: 0.9)
• debounce_ms (default: 150)
• t_max_prompt_ms (default: 500)
• t_max_predict_ms (default: 1000)

Helix

Note: Requires a fork of Helix with PR #14876 (not yet merged). See: helix-editor/helix#14876

Define the language server in ~/.config/helix/languages.toml:

[language-server.llama-lsp]
command = "deno"
args = ["run", "-A", "jsr:@ewired/llama-lsp", "--stdio"]

Add llama-lsp to the language-servers array for each language in languages.toml:

[[language]]
name = "typescript"
language-servers = [ "typescript-language-server", "deno-lsp", "llama-lsp" ]

Configure settings globally (applies to all languages using this server):

[language-server.llama-lsp.config.llamaLsp]
llama_endpoint = "http://127.0.0.1:8012/infill"
n_predict = 128
temperature = 0.0