Add Combo test cases#2873
Draft
charlesli640 wants to merge 8 commits into
Draft
Conversation
charlesli640
requested review from
Lumosis,
gxd3,
jrplatin,
pv97 and
wenxindongwork
as code owners
DescriptionStart with a short description of what the PR does and how this is a change from The rest of the description includes relevant details and context, examples:
If the change fixes a Github issue, please include a link, e.g.,: TestsPlease describe how you tested this change, and include any instructions and/or ChecklistBefore submitting this PR, please make sure:
|
charlesli640
force-pushed
the
combo_test/initial_cases
branch
from
3cf7a34 to
7383cc3
Compare
meiyeh123
requested review from
CienetStingLin,
QiliangCui and
yiw-wang
as code owners
meiyeh123
force-pushed
the
combo_test/initial_cases
branch
from
5ebb122 to
08daa6a
Compare
meiyeh123
force-pushed
the
combo_test/initial_cases
branch
from
9eb0599 to
a4578d9
Compare
charlesli640
force-pushed
the
combo_test/initial_cases
branch
from
939bcde to
4a5df75
Compare
For multiple features co-working, developed a few E2E test scripts. test_combo_chunked_offload_spec.py: Chuncked prefill + KV cache offload + Async Sheduling + Speculative decoding test_combo_hybrid_kv_cache_prefix.py: Prefix Caching + Hybrid KV Cache test_combo_offload_prefix_spec.py: Speculative Decoding + Prefix Caching + and KV Cache Cache Offloading test_combo_sampling_params_prefix.py: Prefix Caching + Sampling Params test_combo_structured_sampling_params.py: Structured Decoding + Sampling Params Signed-off-by: Charles Li <licharles@google.com>
Signed-off-by: dennis yeh <dennis.yeh@cienet.com>
Signed-off-by: dennis yeh <dennis.yeh@cienet.com>
Signed-off-by: Charles Li <licharles@google.com>
meiyeh123
force-pushed
the
combo_test/initial_cases
branch
from
4a5df75 to
2a4cc72
Compare
Signed-off-by: dennis yeh <dennis.yeh@cienet.com>
meiyeh123
force-pushed
the
combo_test/initial_cases
branch
from
60e8a28 to
7dfcace
Compare
circlepen
force-pushed
the
combo_test/initial_cases
branch
2 times, most recently
from
2af2773 to
41377aa
Compare
Signed-off-by: Lyle Lai <lyle.lai@cienet.com>
circlepen
force-pushed
the
combo_test/initial_cases
branch
from
41377aa to
e6134ff
Compare
Signed-off-by: Lyle Lai <lyle.lai@cienet.com>
This reverts commit 414de8c. Signed-off-by: Lyle Lai <lyle.lai@cienet.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
For multiple features co-working, developed a few E2E test scripts.
test_combo_chunked_offload_spec.py:Chuncked prefill + KV cache offload + Async Sheduling + Speculative decoding
test_combo_hybrid_kv_cache_prefix.py:Prefix Caching + Hybrid KV Cache
test_combo_offload_prefix_spec.py:Speculative Decoding + Prefix Caching + and KV Cache Cache Offloading
test_combo_sampling_params_prefix.py:Prefix Caching + Sampling Params
test_combo_structured_sampling_params.py:Structured Decoding + Sampling Params
Fix b/506848347
Tests
Please describe how you tested this change, and include any instructions and/or
commands to reproduce.
Checklist
Before submitting this PR, please make sure: