llm-vllm: unify environment variable LLM_MODEL_ID

[lianhao]

Description

Currently, llm-vllm services use the environment variable LLM_MODEL while all other services in llm category are using LLM_MODEL_ID. This PR unify all the services in llm category to use the same environment variable LLM_MODEL_ID to minimize the configuration management efforts such as docker-compose, helm, etc.

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

• [x ] Bug fix (non-breaking change which fixes an issue)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

[eero-t]

This PR replaces only first instance of LLM_MODEL in each file, but several of the files contain multiple instances of it:

$ git grep -c [^_]LLM_MODEL[:=]
comps/llms/text-generation/README.md:5
comps/llms/text-generation/vllm/langchain/README.md:1
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:2
comps/llms/text-generation/vllm/langchain/launch_microservice.sh:1
comps/llms/text-generation/vllm/llama_index/README.md:1
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:2
comps/llms/text-generation/vllm/llama_index/launch_microservice.sh:1
tests/llms/test_llms_text-generation_vllm_langchain_on_intel_hpu.sh:2
tests/llms/test_llms_text-generation_vllm_llamaindex_on_intel_hpu.sh:2

[eero-t]

PRs containing workarounds for the bug this PR will fix:

• Add vLLM support to DocSum Helm chart GenAIInfra#649
• docsum: reduce microservices in docsum GenAIInfra#659

[lianhao]

This PR replaces only first instance of LLM_MODEL in each file, but several of the files contain multiple instances of it:

$ git grep -c [^_]LLM_MODEL[:=]
comps/llms/text-generation/README.md:5
comps/llms/text-generation/vllm/langchain/README.md:1
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:2
comps/llms/text-generation/vllm/langchain/launch_microservice.sh:1
comps/llms/text-generation/vllm/llama_index/README.md:1
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:2
comps/llms/text-generation/vllm/llama_index/launch_microservice.sh:1
tests/llms/test_llms_text-generation_vllm_langchain_on_intel_hpu.sh:2
tests/llms/test_llms_text-generation_vllm_llamaindex_on_intel_hpu.sh:2

Already double checked, the remaining bash environment variable LLM_MODEL is used by the upstream tgi/vllm, not by the llm-vllm services this PR is trying to modify.

[eero-t]

These are not Bash exports:

$ git grep  [^_]LLM_MODEL[:=] | grep compose
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}

PR changes only first of those?

[lianhao]

These are not Bash exports:

$ git grep  [^_]LLM_MODEL[:=] | grep compose
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}

PR changes only first of those?

Yes, the remaining LLM_MODEL is not for llm-vllm wrapper container, but for the vllm itself(which is a opea rebuild of upstream vllm).

[xiguiw]

These are not Bash exports:

$ git grep  [^_]LLM_MODEL[:=] | grep compose
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/langchain/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}
comps/llms/text-generation/vllm/llama_index/docker_compose_llm.yaml:      LLM_MODEL: ${LLM_MODEL}

PR changes only first of those?

Yes, the remaining LLM_MODEL is not for llm-vllm wrapper container, but for the vllm itself(which is a opea rebuild of upstream vllm).

@lianhao
Greate! This is helpful. It makes things simple.
Do you know the status of TEI_MODEL and RERANKING MODLEL environment variables? Do you plan to unify them?
Thanks!

[lianhao]

Do you know the status of TEI_MODEL and RERANKING MODLEL environment variables? Do you plan to unify them?
Thanks!

Sorry, I'm not aware of that

[eero-t]

These are not Bash exports:
...
PR changes only first of those?

Yes, the remaining LLM_MODEL is not for llm-vllm wrapper container, but for the vllm itself(which is a opea rebuild of upstream vllm).

Sure, but it's given to vLLM with --model option, so it could be changed also. Usage of multiple env var names for the same purpose in same file is confusing. Why not change all of them?

[lianhao]

closed as new llm-textgen doesn't have this issue