Skip to content

Add multiformat-hybrid-rag agent sample#1698

Open
lspataroG wants to merge 8 commits intogoogle:mainfrom
lspataroG:add-multiformat-hybrid-rag
Open

Add multiformat-hybrid-rag agent sample#1698
lspataroG wants to merge 8 commits intogoogle:mainfrom
lspataroG:add-multiformat-hybrid-rag

Conversation

@lspataroG
Copy link
Copy Markdown
Contributor

Summary

  • Adds a production RAG agent with multi-format document ingestion (PDF, Office, HTML, JSON), contextual chunk enrichment, and hybrid semantic + keyword search via Vector Search 2.0
  • Serves through an ADK agent, REST API, and MCP server from a single Cloud Run service
  • Includes full ingestion pipeline on Vertex AI Pipelines, Terraform IaC, and Agent Starter Pack integration

Key features

  • Multi-format parsing: PDF (Gemini multimodal), DOC/DOCX/PPT/PPTX (LibreOffice), HTML, JSON/JSONL, Markdown
  • Contextual chunking: Gemini-generated per-chunk context prepended before embedding
  • Hybrid search: semantic + BM25 via Reciprocal Rank Fusion on Vector Search 2.0
  • Three serving interfaces: ADK agent chat, REST search API, MCP server
  • Full ingestion pipeline: 3-step Vertex AI Pipeline (preprocess → chunk & index → cleanup) with daily scheduling
  • Infrastructure as code: Terraform for BQ, VS2, Cloud Run, IAM
  • Agent Starter Pack: agent-starter-pack create my_app -a adk@multiformat-hybrid-rag

Test plan

  • make install succeeds
  • make setup-infra provisions all resources
  • make data-ingestion-remote runs the pipeline end-to-end
  • make local-backend starts the agent + REST + MCP server
  • make playground launches ADK playground UI
  • make deploy deploys to Cloud Run
  • Search API returns grounded answers from indexed documents
  • MCP tool ask_knowledge_base works from external clients

Production RAG agent on GCP with multi-format document ingestion
(PDF, Office, HTML, JSON), contextual chunk enrichment, and hybrid
semantic + keyword search via Vector Search 2.0. Served through an
ADK agent, REST API, and MCP server from a single Cloud Run service.

Key features:
- Multi-format parsing: PDF (Gemini multimodal), DOC/DOCX/PPT/PPTX
  (LibreOffice), HTML, JSON/JSONL, Markdown
- Contextual chunking: Gemini-generated per-chunk context prepended
  before embedding
- Hybrid search: semantic + BM25 via Reciprocal Rank Fusion on VS 2.0
- Three serving interfaces: ADK agent, REST API, MCP server
- Full ingestion pipeline on Vertex AI Pipelines with daily scheduling
- Infrastructure as code via Terraform
- Agent Starter Pack integration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant