swift-llama v0.1.0 — Swift package for local LLM inference

[thegdsks]

We just shipped swift-llama — a Swift package for running LLMs locally on Apple devices via llama.cpp.

What it does

• Actor-isolated inference with AsyncThrowingStream for token streaming
• Chat templates for Gemma, Llama 3, Mistral, and ChatML
• Streaming tool call parser that detects function calls as tokens arrive
• Model downloader (HuggingFace + SHA256 verification)
• Catalog of popular models with recommended configs
• Metal GPU acceleration, Swift 6 strict concurrency safe

Install

.package(url: "https://github.com/profclaw/swift-llama", from: "0.1.0")

Quick example

import SwiftLlama

let model = try LlamaModel(path: "model.gguf", config: .init(gpuLayers: .all))
let llama = try LlamaActor(model: model)

for try await chunk in llama.chat(messages: [.user("Hello!")], template: .gemma) {
    switch chunk {
    case .text(let token): print(token, terminator: "")
    case .toolCall(let call): print("Tool: \(call.name)")
    }
}

This is the same inference layer that powers ProfClaw Studio. We extracted it into a standalone package so anyone building Swift apps with local LLMs can use it.

MIT licensed. Feedback and contributions welcome.

GitHub · Release