youtube-qna: self-hostable q&a platform over youtube transcripts

index.mdx

@@ -3,3 +3,4 @@
 a collection of system design documents for random ideas that come to mind.
 
 - [tribal-knowledge-mcp](./tribal-knowledge-mcp) - making tribal knowledge queryable by ai agents through an mcp server
+- [youtube-qna](./youtube-qna) - self-hostable q&a platform over youtube video transcripts using rag and local or openrouter models

youtube-qna/problem.mdx

@@ -0,0 +1,23 @@
+# the problem
+
+youtube videos are long. like, really long.
+
+## the content length problem
+
+a single conference talk, deep dive tutorial, or interview can run 2-4 hours. there's genuinely useful information in there- but the cost of extracting it is sitting through the whole thing, scrubbing around hoping you land on the right part, or just giving up and not watching it at all.
+
+bookmarking a video is not the same as having learned from it. and most people never go back.
+
+## the search problem
+
+even if you watched the whole thing, you can't "ctrl+f" a video. you can't ask it a question. the information is locked in a linear format that's hostile to retrieval. you either remember it or you don't.
+
+youtube's auto-generated chapters and descriptions help a little, but they're shallow- they tell you what's in a section, not what's actually being said or why it matters.
+
+## the context problem
+
+a lot of valuable content lives on youtube specifically- technical talks, research explanations, long-form interviews, course lectures. this isn't content you'd find in a blog post or docs page. it's information that only exists in video form, and that makes it basically opaque to any kind of structured retrieval.
+
+## what this means
+
+hours of dense content become effectively inaccessible once you've watched (or skipped) them. you can't query your youtube history. you can't ask "what did that person say about x in that video i watched last month." the information exists, but you have no way to get back to it.

youtube-qna/raw.mdx

@@ -0,0 +1,9 @@
+# raw inputs
+
+these are the raw unedited inputs that were provided to put together the design document- just for transparency and an open look at how the ideas were communicated before being cleaned up.
+
+---
+
+## initial brain dump (2026-04-05)
+
+put up a basic pr for now, but what i want to make it a qna sort of a platform which is self hostable and runs either open router or local models, and you can use it to checkout a youtube video- that could take up hours for example but extract the most important stuff from it, we mainly do this by getting the transcription and making a RAG componenet to it- https://github.com/jdepoix/youtube-transcript-api

youtube-qna/references.mdx

@@ -0,0 +1,8 @@
+# references
+
+- [youtube-transcript-api](https://github.com/jdepoix/youtube-transcript-api) - python library for fetching youtube transcripts without an api key
+- [chroma](https://www.trychroma.com/) - local vector database used for storing and querying transcript embeddings
+- [sentence-transformers](https://www.sbert.net/) - library for generating local embeddings
+- [ollama](https://ollama.com/) - run local llms with a simple api
+- [openrouter](https://openrouter.ai/) - unified api for routing to multiple llm providers
+- [fastapi](https://fastapi.tiangolo.com/) - python api framework for the backend layer

youtube-qna/solution.mdx

@@ -0,0 +1,47 @@
+# solution & requirements
+
+a self-hostable q&a platform that turns any youtube video into something you can actually have a conversation with.
+
+## the concept
+
+paste a youtube url, the system pulls the transcript, chunks it, embeds it, and drops it in a local vector store. from there you can ask it anything- get a summary, drill into a specific topic, ask follow-up questions. the llm is either a local model or routed through openrouter, your choice.
+
+no cloud dependency. runs on your machine. your data stays local.
+
+## how it works end to end
+
+1. user pastes a youtube url into the ui
+2. system pulls the transcript via `youtube-transcript-api`
+3. transcript gets chunked into overlapping segments (with timestamps where possible)
+4. chunks get embedded and stored in a local vector store
+5. user asks a question
+6. relevant chunks are retrieved via semantic similarity
+7. retrieved chunks + question go to the llm
+8. llm answers based on the actual transcript content
+
+## model flexibility
+
+the llm layer is pluggable:
+
+- **openrouter**: routes to whatever model you want (gpt-4o, claude, mistral, etc.) through a single api. useful if you have an openrouter key but don't want to manage multiple api keys.
+- **local model**: runs fully offline via ollama or llama.cpp. nothing leaves your machine.
+
+the system should make switching between these straightforward- ideally just a config change.
+
+## requirements
+
+### must have
+- youtube url input → transcript extraction via `youtube-transcript-api`
+- transcript chunking with overlap for coherent context windows
+- vector store for semantic retrieval (local)
+- q&a interface over the retrieved chunks
+- support for openrouter as the llm backend
+- support for local models (ollama)
+- self-hostable via docker
+
+### nice to have
+- timestamp-aware retrieval (link back to the moment in the video)
+- multi-video support (ask across a collection)
+- persistent storage so you don't re-process videos you've already loaded
+- summary generation on ingest (get the tldr without asking)
+- simple web ui (doesn't need to be fancy)

youtube-qna/technical-design.mdx

@@ -0,0 +1,119 @@
+# technical design
+
+## tech stack
+
+| layer | technology |
+|---|---|
+| transcript extraction | `youtube-transcript-api` (python) |
+| chunking | sliding window with overlap |
+| embeddings | local model via `sentence-transformers` or api |
+| vector store | chroma (local, persistent) |
+| llm backend | openrouter api or ollama |
+| api layer | fastapi |
+| ui | simple web frontend (or cli) |
+| deployment | docker / docker-compose |
+
+## architecture overview
+
+```
+user
+  ↓ youtube url
+transcript service (youtube-transcript-api)
+  ↓ raw transcript text + timestamps
+chunker
+  ↓ overlapping chunks with metadata
+embedding model
+  ↓ vectors
+chroma vector store (persistent)
+  ↑ semantic search on query
+retriever
+  ↓ top-k relevant chunks
+llm (openrouter or ollama)
+  ↓ answer grounded in transcript
+user
+```
+
+## transcript extraction
+
+`youtube-transcript-api` handles the heavy lifting here- it fetches auto-generated or manually uploaded captions directly from youtube without needing any api key. it returns a list of text segments with start timestamps and durations.
+
+```python
+from youtube_transcript_api import YouTubeTranscriptApi
+
+transcript = YouTubeTranscriptApi.get_transcript(video_id)
+# [{ "text": "...", "start": 12.34, "duration": 3.5 }, ...]
+```
+
+the raw transcript gets reassembled into a single text (or kept segment-aware for timestamp retrieval) before chunking.
+
+## chunking
+
+the transcript gets split into overlapping chunks. the overlap is important- concepts don't respect arbitrary split points, and without overlap you'll lose context at chunk boundaries.
+
+rough approach:
+- target chunk size: ~400-500 tokens
+- overlap: ~50-100 tokens
+- metadata stored per chunk: video id, start timestamp, end timestamp
+
+the timestamp metadata is useful for linking back- "this answer came from around 1:23:45 in the video."
+
+## vector store
+
+chroma is a good fit here- it's local, persistent, and easy to set up without running a separate service. it handles both the vector storage and the similarity search.
+
+each collection maps to a video. this makes it easy to isolate or combine videos later.
+
+## embedding
+
+two options:
+
+- **local**: `sentence-transformers` (e.g. `all-MiniLM-L6-v2`). runs inside the container, no api dependency, reasonable quality.
+- **api**: openai embeddings or similar. better quality, but adds a network call and key requirement.
+
+for a fully self-hosted setup, local embeddings are the default.
+
+## llm backend
+
+the llm call is made with the retrieved chunks as context:
+
+```
+[system prompt: answer based only on the provided transcript excerpts]
+[context: chunk 1, chunk 2, chunk 3...]
+[user: what did they say about X?]
+```
+
+**openrouter**: single api endpoint, model is configurable in `.env`. anything openrouter supports just works.
+
+**ollama**: runs a local model server. the fastapi backend calls ollama's api at `localhost:11434`. model is also configurable.
+
+the backend abstracts both behind a common interface so swapping is just a config change.
+
+## api
+
+fastapi handles the backend. key endpoints:
+
+- `POST /ingest` — takes a youtube url, runs extraction + chunking + embedding, returns video id
+- `POST /query` — takes a video id + question, returns answer + source chunks
+- `GET /videos` — lists ingested videos
+
+## persistence
+
+chroma persists its index to disk by default. this means you only process a video once- subsequent queries go straight to the vector store.
+
+## deployment
+
+```yaml
+# docker-compose.yml (rough)
+services:
+  api:
+    build: .
+    ports: ["8000:8000"]
+    volumes:
+      - ./data:/data  # chroma index lives here
+    environment:
+      - LLM_BACKEND=openrouter  # or ollama
+      - OPENROUTER_API_KEY=...
+      - OLLAMA_MODEL=llama3
+```
+
+for ollama, it either runs as a separate container or points to a host ollama instance.

youtube-qna/why.mdx

@@ -0,0 +1,13 @@
+# why i thought of this idea
+
+there are videos i want to "read" not watch.
+
+a 3-hour conference talk has maybe 20 minutes of content i actually care about. i don't want to watch the whole thing to find it- but i also can't just skip to the good parts because i don't know where they are.
+
+and then there's the retrieval problem after the fact. i'll half-watch something, close the tab, and a week later realize i actually needed that information. now i have to either re-watch it or just not have it.
+
+the obvious answer is "use the transcript"- and youtube does expose transcripts. but raw transcripts are pretty rough. no punctuation, dense walls of text, speaker cross-talk. they're not really readable, and they're definitely not queryable in any useful way.
+
+so the idea is- pull the transcript, chunk it up properly, throw it into a vector store, and then just ask it questions. point the whole thing at a local model or openrouter so it's fully self-contained. no subscriptions, no third-party data handling, just a URL in and a conversation out.
+
+the goal is to be able to drop any youtube link and immediately have a conversation with it- ask what the key points are, go deep on a specific claim, or just get a summary of the whole thing. same value as watching it, fraction of the time.