scion-help-recipe

A reference implementation of the SCION-Help retrieval-augmented question- answering recipe, written so a practitioner can port it to another tightly- scoped technical domain. The thesis the recipe operationalises: for a tightly-scoped domain, the bulk of answer quality comes from an offline foundation (a fine-tuned domain bi- encoder, an LLM-written description per chunk, and a type-annotated hybrid index) rather than from online reasoning. Build that foundation once and each query is cheap. The repository ships the offline pipeline, both online architectures, the prompts, and the serving stack. It does not ship a corpus -- you supply your own.

The recipe at a glance

#	Step	Script	Effort
1	Partition the corpus by document type and chunk it	recipe/01_chunk_corpus.py	1-2 days, expert
2	Flagship-LLM pass: a one-line description per chunk	recipe/02_describe_chunks.py	~$5-50, one-time
3	Flagship-LLM pass: 5-8 synthetic questions per chunk	recipe/03_generate_questions.py	(same pass)
4	Fine-tune the bi-encoder on (question, chunk) pairs	recipe/04_finetune_embedder.py	< 1 h, one GPU
-	Measure the lift (Recall@k, MRR, gold rank)	recipe/eval_retrieval.py	minutes
5	(optional) fine-tune the cross-encoder reranker	recipe/05_finetune_reranker.py	< 1 h, one GPU
6	Index into Milvus (dense + BM25, RRF)	recipe/06_build_index.py	minutes
7	Serve v1 -- the deployed two-call pipeline	recipe/07_serve_pipeline.py	1-2 days, engineer
8	(optional) serve v2 -- the agentic loop	recipe/08_serve_agentic.py	--

Steps 1-3 are the corpus prep, step 4 is the load-bearing component (the paper's biggest single effect), step 6 builds the index, and steps 7-8 are the two online architectures the paper compares. See docs/ARCHITECTURE.md for how the pieces connect and docs/PIPELINE_NOTES.md for exactly how faithful each script is to the deployed system.

Two online architectures

• v1 (07_serve_pipeline.py) -- the deployed pipeline. Two LLM calls per query: a small rewrite/classify call, then type-routed hybrid retrieval, then a grounded generation call. Cheap (well under a cent per query) and fast. No reranker at serving time (the paper ablates it and finds it within noise).
• v2 (08_serve_agentic.py) -- an agentic select-then-draft loop. The agent drives wide retrieval as a tool, commits a chunk selection (or refuses on thin evidence), then drafts/critiques/redrafts. ~6x the cost and ~1.8x the latency of v1 for a modest faithfulness gain; prefer v1 unless strict faithfulness or explicit refusal matters.

Both consume the same offline foundation and the same open-weights generator.

Quick start

git clone https://github.com/netsys-lab/scion-help-recipe
cd scion-help-recipe
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

cp .env.example .env        # then edit .env with your API key (never commit it)

Bring up Milvus (or point at any Milvus 2.5 instance):

cp config/docker-compose.example.yml docker-compose.yml
docker compose up -d etcd minio milvusdb

Lay out a small corpus under ./sources/ (book/ papers/ docs/ specs/ code/ libraries/), then run the pipeline:

set -a; source .env; set +a                      # export the keys

python recipe/01_chunk_corpus.py     --input ./sources --output chunks.jsonl
python recipe/02_describe_chunks.py  --input chunks.jsonl           --output chunks_described.jsonl
python recipe/03_generate_questions.py --input chunks_described.jsonl --output chunks_with_questions.jsonl

python recipe/04_finetune_embedder.py --input chunks_with_questions.jsonl --output ./bi-encoder-finetuned
python recipe/eval_retrieval.py       --input chunks_with_questions.jsonl --finetuned ./bi-encoder-finetuned

python recipe/06_build_index.py --input chunks_with_questions.jsonl \
    --embedding-model ./bi-encoder-finetuned --collection my_domain_v1 --drop-existing

python recipe/07_serve_pipeline.py --query "What is X in <your domain>?"
# or the agentic variant:
python recipe/08_serve_agentic.py  --query "What is X in <your domain>?" --collection my_domain_v1

A full worked walkthrough (porting to a hypothetical "WireGuard Q&A" domain) is in docs/GETTING_STARTED.md.

Repository layout

recipe/    runnable Python, one file per step (01..08 + eval_retrieval.py)
prompts/   the production prompts; replace the domain term before running
config/    valves.example.json (Open-WebUI) and docker-compose.example.yml
docs/      GETTING_STARTED.md, ARCHITECTURE.md, PIPELINE_NOTES.md
.env.example  copy to .env; the repo is key-free by design

Released models

The SCION-Help domain models are on the Hugging Face Hub. Steps 4-5 reproduce the method that produced them; point --embedding-model / RERANKER_MODEL at your own fine-tunes for a new domain.

• Bi-encoder: tjohn327/scion-snowflake-arctic-embed-s-v2 (384-dim, base Snowflake/snowflake-arctic-embed-s)
• Cross-encoder: tjohn327/scion-cross-encoder-v3 (base cross-encoder/ms-marco-MiniLM-L-6-v2)

Security

This repository is key-free. All scripts read credentials from the environment (OPENROUTER_API_KEY / API_KEY, optional HF_TOKEN); copy .env.example to .env and keep .env out of git (it is gitignored). Never paste a key into a script or a config file you commit.

License

MIT, see LICENSE.