Feat/add agentic rag workshop - #29
Merged
Merged
Conversation
git-subtree-dir: agent git-subtree-split: fa34c25b829f14664d4b7f54e8a424cb3d1dc625
There was a problem hiding this comment.
Pull request overview
This PR adds a full “Agentic RAG workshop” documentation set (numbered specs + roadmap/impl plan) alongside a runnable Python demo package with deterministic fallbacks, fixture corpus, and a unittest-based contract suite to validate core workshop behaviors (retrieval, version scoping, terminology resolution, generation, and safety/trace invariants).
Changes:
- Introduces a dependency-ordered spec set under
agent/specs/(PRD → data model → ingestion/workflow/generation/UI → quality gates → roadmap/plan). - Adds a runnable demo under
agent/demo/(package, scripts, notebooks, sample data, eval fixtures) plus tests covering retrieval, embedding/generation adapters, entity resolution, and workflow policies. - Adds repository workshop guidelines (
agent/AGENTS.md) and agent skill metadata.
Reviewed changes
Copilot reviewed 95 out of 137 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| agent/specs/index.md | Adds canonical spec entry point with reading order and dependency graph. |
| agent/specs/archive/README.md | Documents legacy notes archive and traceability mapping. |
| agent/specs/archive/Optimize.md | Stores legacy optimization placeholders for traceability. |
| agent/specs/archive/notebook.md | Stores legacy notebook outline for traceability. |
| agent/specs/archive/Draft.md | Stores original workshop draft note for traceability. |
| agent/specs/93-improvements-review.md | Adds first-demo review backlog and closure criteria. |
| agent/specs/91-impl-plan.md | Adds dependency-ordered implementation phases and Phase 0 gates. |
| agent/specs/90-roadmap.md | Adds stakeholder roadmap (M0–M3) with exit criteria. |
| agent/specs/80-glossary.md | Defines project-specific terminology to reduce ambiguity. |
| agent/specs/70-quality-and-evaluation.md | Defines quality gates, eval metrics, and demo safety boundary. |
| agent/specs/20-ui-demo.md | Defines Streamlit UI contract, invariants, and error states. |
| agent/specs/13-llm-answer-generation.md | Defines grounded generation interface, citation guard, fallback, tests. |
| agent/specs/11-ingestion.md | Defines offline ingestion pipeline, identity/version rules, failure policy. |
| agent/specs/10b-conversation-memory.md | Defines session-scoped memory contract, TTL/isolation, UI surface. |
| agent/specs/10a-openai-text-embedding.md | Defines OpenAI embedding provider contract preserving 1024-dim seam. |
| agent/specs/10-data-model.md | Defines core collections/contracts, invariants, and fixture contracts. |
| agent/specs/00-prd.md | Defines scope/goals/non-goals and binding naming conventions. |
| agent/demo/tests/test_server_workflow.py | Tests Milvus workflow builder and collection readiness validation. |
| agent/demo/tests/test_retrieval.py | Tests local hybrid retrieval behavior, filters, version isolation. |
| agent/demo/tests/test_generation.py | Tests OpenAI/deterministic generation, citation validation, fallback reasons. |
| agent/demo/tests/test_entities.py | Tests entity catalog validation and ambiguity handling. |
| agent/demo/tests/test_embedding.py | Tests OpenAI embedding adapter contract, sanitization, config modes. |
| agent/demo/tests/test_config.py | Tests .env loader parsing and override behavior. |
| agent/demo/tests/test_agentic_tools.py | Tests tool routing, permission gating, version-scope behaviors, and UI constraints. |
| agent/demo/src/agent_workshop_demo/validation.py | Adds shared validation for questions, identifiers, and filter normalization. |
| agent/demo/src/agent_workshop_demo/schema/collections.py | Adds provisional Milvus schema/index definitions and defaults. |
| agent/demo/src/agent_workshop_demo/schema/init.py | Exposes schema objects via package public surface. |
| agent/demo/src/agent_workshop_demo/retrieval.py | Adds deterministic in-memory hybrid retriever mirroring Milvus contract. |
| agent/demo/src/agent_workshop_demo/reranker.py | Adds reranker interface and deterministic rule-based reranker. |
| agent/demo/src/agent_workshop_demo/models.py | Adds core dataclasses for chunks/results/state used across layers. |
| agent/demo/src/agent_workshop_demo/knowledge_tools.py | Adds bounded tool definitions and demo permission boundary. |
| agent/demo/src/agent_workshop_demo/generation.py | Adds grounded answer generation adapters, citation guard, fallback generator. |
| agent/demo/src/agent_workshop_demo/events.py | Adds allow-listed, presentation-safe trace event envelopes. |
| agent/demo/src/agent_workshop_demo/entities.py | Adds strict entity catalog loader and deterministic resolver. |
| agent/demo/src/agent_workshop_demo/embedding.py | Adds deterministic/OpenAI embedding providers + shared dense_vector seam. |
| agent/demo/src/agent_workshop_demo/dedup.py | Adds checksum + experimental MinHash signature helpers. |
| agent/demo/src/agent_workshop_demo/config.py | Adds constants, collection names, vector dims, and dotenv loader. |
| agent/demo/src/agent_workshop_demo/cli.py | Adds CLI entrypoint for local workflow execution and JSON output. |
| agent/demo/src/agent_workshop_demo/init.py | Initializes package exports and currently auto-loads .env. |
| agent/demo/scripts/run_eval.py | Adds golden-question evaluation runner script. |
| agent/demo/scripts/ingest_demo.py | Adds ingestion script to generate/insert/verify fixture chunks in Milvus. |
| agent/demo/scripts/create_indexes.py | Adds script to create/inspect Milvus index definitions. |
| agent/demo/scripts/create_collections.py | Adds script to create/inspect Milvus collection schemas. |
| agent/demo/scripts/cleanup_milvus.py | Adds script to safely drop only demo-owned Milvus collections. |
| agent/demo/sample_data/mock_s3/security/ttl_memory_policy.md | Adds fixture doc for TTL/memory policy examples. |
| agent/demo/sample_data/mock_s3/product/roadmap_notes.md | Adds fixture doc for roadmap coverage comparison questions. |
| agent/demo/sample_data/mock_s3/product/go_button_guide_v2.md | Adds v2 fixture doc for version-scoped terminology questions. |
| agent/demo/sample_data/mock_s3/product/go_button_guide_v1.md | Adds v1 fixture doc for exact-version questions. |
| agent/demo/sample_data/mock_s3/product/customer_meeting_notes.md | Adds fixture doc for multi-tool comparison workflow tests. |
| agent/demo/sample_data/mock_s3/hr/pto_policy.md | Adds HR policy fixture doc for routing/filter examples. |
| agent/demo/sample_data/mock_s3/engineering/s3_sync_design.md | Adds engineering fixture doc for the golden architecture question. |
| agent/demo/sample_data/local_docs/product/agent_ui_mvp.md | Adds local fixture doc describing UI contract in prose. |
| agent/demo/sample_data/local_docs/engineering/milvus_feature_map.md | Adds local fixture doc describing Milvus role/features for retrieval. |
| agent/demo/sample_data/local_docs/engineering/agentic_rag_workflow.md | Adds local fixture doc describing agentic workflow steps. |
| agent/demo/sample_data/document_versions.json | Adds version manifest fixture for stable doc_id/doc_version mapping. |
| agent/demo/sample_data/asset_manifest.json | Adds manifest for PDF/image pseudo-records for ingestion. |
| agent/demo/requirements.txt | Adds demo dependencies (Streamlit, pymilvus, langgraph, openai, pypdf). |
| agent/demo/pyproject.toml | Adds minimal packaging metadata for editable install. |
| agent/demo/notebooks/06_streamlit_ui_demo.ipynb | Adds notebook to launch Streamlit demo. |
| agent/demo/notebooks/05_langgraph_agentic_rag.ipynb | Adds notebook to run the LangGraph/local workflow. |
| agent/demo/notebooks/04_milvus_hybrid_search.ipynb | Adds notebook to exercise local hybrid retriever behavior. |
| agent/demo/notebooks/03_milvus_schema_and_insert.ipynb | Adds notebook to inspect schema definitions. |
| agent/demo/notebooks/02_text_image_embedding.ipynb | Adds notebook to demonstrate embeddings/image placeholder. |
| agent/demo/notebooks/01_ingestion_local_s3.ipynb | Adds notebook to run offline ingestion and inspect outputs. |
| agent/demo/eval/questions.json | Adds golden questions with expected sources/tools/entities/scope. |
| agent/demo/eval/golden_answers.yaml | Adds golden answer facts + required citations for eval gating. |
| agent/demo/config/predefined_entities.yaml | Adds predefined entity catalog fixture (JSON-compatible YAML). |
| agent/demo/.env.example | Adds example env configuration for embedding/generation/Milvus/memory. |
| agent/AGENTS.md | Adds repo contribution/testing guidance and workshop file layout. |
| agent/.gitignore | Ignores demo .env and Python artifacts. |
| agent/.agents/skills/spec/agents/openai.yaml | Adds spec skill metadata for the agent workspace. |
| agent/.agents/skills/research/SKILL.md | Adds research skill guidance and memo templates. |
| agent/.agents/skills/research/agents/openai.yaml | Adds research skill agent interface metadata. |
| agent/.agents/skills/impl/agents/openai.yaml | Adds impl skill agent interface metadata. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+142
to
+151
| try: | ||
| response = self.client.responses.create( | ||
| model=self.model, | ||
| instructions=OPENAI_INSTRUCTIONS, | ||
| input=_generation_input(request), | ||
| timeout=self.timeout_seconds, | ||
| ) | ||
| except Exception as exc: | ||
| raise AnswerGenerationError(_provider_reason(exc)) from exc | ||
| text = str(response.output_text).strip() |
Comment on lines
+3
to
+7
| from agent_workshop_demo.config import load_demo_env | ||
|
|
||
| load_demo_env() | ||
|
|
||
| from agent_workshop_demo.workflow import AgenticRAGWorkflow # noqa: E402 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.