Skip to content

Add /llm POST Endpoint#27

Open
CelestineAkpanoko wants to merge 6 commits into
mainfrom
llm-integration
Open

Add /llm POST Endpoint#27
CelestineAkpanoko wants to merge 6 commits into
mainfrom
llm-integration

Conversation

@CelestineAkpanoko

Copy link
Copy Markdown

Description:
Implements a new /llm POST endpoint for generic LLM inference using Groq Cloud (LLaMA 3.1). Separates request and response models into src/core/api/models/llm/request.py and src/core/api/models/llm/response.py. Tests are organized in tests/llm/unit/, tests/llm/integration/, and tests/llm/e2e/. Fixed test discovery and async mocking issues.

Changes:

  • Added LLMRequest in src/core/api/models/llm/request.py.
  • Added LLMResponse in src/core/api/models/llm/response.py.
  • Added src/core/api/routes/llm.py with new import paths.
  • Updated src/main.py to include the LLM router.
  • Added unit tests in tests/llm/unit/test_llm_models.py.
  • Added integration tests in tests/llm/integration/test_llm_routes.py.
  • Added E2E tests in tests/llm/e2e/test_llm_workflow.py.
  • Added tests/llm/conftest.py for fixtures.
  • Added pytest.ini to configure pytest-asyncio, set pythonpath = ., and add testpaths = tests/llm and e2e marker.
  • Added docs/llm.md
  • Fixed failing tests by correcting endpoint to /llm/ and using AsyncMock.

Testing:

  • Unit tests for LLMRequest and LLMResponse models.
  • Integration tests for /llm endpoint with mocked Groq responses.
  • E2E tests for full workflow (prompt + context).
  • All tests pass (pytest tests/llm/).

Checklist:

  • Tests written and passing
  • Documentation updated
  • Code follows project style guidelines
  • External services (Groq) mocked

@LokiMetaSmith

Copy link
Copy Markdown

This pull needs to be more generic.
It needs to support more than just groq and it shouldn't add in groq as a dependency.

For example, there is no reason why this couldn't run on ollama, llama.cpp, open.ai, or any of the vast number of other AI providers.

good start though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants