pywhispercpp has no vad support?

[aizimuji]

but whisper.cpp seems support vad .

how we use pywhipsercpp and vad , any example?

[femelo]

I won't create a PR here because I ended up passing my changes through ruff and I am not sure code style changes would be appreciated. You can find whisper.cpp VAD integrated here: https://github.com/femelo/pywhispercpp. You can pass: vad (bool), vad_model and vad_params (dict) to construct Model, or use the pwcpp script with vad_params flattened (e.g., --vad-threshold).

[aizimuji]

I won't create a PR here because I ended up passing my changes through ruff and I am not sure code style changes would be appreciated. You can find whisper.cpp VAD integrated here: https://github.com/femelo/pywhispercpp. You can pass: vad (bool), vad_model and vad_params (dict) to construct Model, or use the pwcpp script with vad_params flattened (e.g., --vad-threshold).

thanks for sharing , I will check it out.

could you summarize the changes you made in for fork , other than vad support ?

[femelo]

Very briefly (femelo#1):

• added VAD support (enabling and parameters)
• adapted example cli accordingly (pwcpp)
• changed download_model (in utils) to also download VAD models (in the same way as that for whisper models)
• code style; typing fixes (via ruff)
• bumped submodules (pybind11 and whisper.cpp) to the latest releases

Usage example:

$ pwcpp test.wav --model tiny-q5_1 --vad 1 --vad-model silero-v6.2.0 --vad-threshold 0.4 --language pt

Default values:

    "vad": {
        "type": bool,
        "description": "vad",
        "options": None,
        "default": False,
    },
    "vad_model_path": {
        "type": str,
        "description": "vad_model_path",
        "options": None,
        "default": "",
    },
    "vad_params": {
        "type": dict,
        "description": "vad_params",
        "options": None,
        "default": {  # per whisper.cpp/examples/cli/cli.cpp
            "threshold": 0.5,
            "min_speech_duration_ms": 250,
            "min_silence_duration_ms": 100,
            "max_speech_duration_s": (1.0 - 2**-24) * (2 ** 128),  # according to float.h
            "speech_pad_ms": 30,
            "samples_overlap": 0.1,
        }
    },

[aizimuji]

great, that's very helpful

actually , I thought of a feature

I need a way to stop processing long audio in middle(since I am writing a gui , user may cancel after start processing, from pyside6)

maybe add some singal processing and control, but i am not familar with c, could you consider add this feature? @femelo

[SebastianTusk]

I won't create a PR here because I ended up passing my changes through ruff and I am not sure code style changes would be appreciated. You can find whisper.cpp VAD integrated here: https://github.com/femelo/pywhispercpp. You can pass: vad (bool), vad_model and vad_params (dict) to construct Model, or use the pwcpp script with vad_params flattened (e.g., --vad-threshold).

Great work.
I tried your fork and VAD seems to work. I just noticed that the bindings to the VAD specific API are missing. Specifically all functions starting with whisper_vad_.

[femelo]

I won't create a PR here because I ended up passing my changes through ruff and I am not sure code style changes would be appreciated. You can find whisper.cpp VAD integrated here: https://github.com/femelo/pywhispercpp. You can pass: vad (bool), vad_model and vad_params (dict) to construct Model, or use the pwcpp script with vad_params flattened (e.g., --vad-threshold).

Great work. I tried your fork and VAD seems to work. I just noticed that the bindings to the VAD specific API are missing. Specifically all functions starting with whisper_vad_.

Indeed, none of those functions have been directly exposed, they are used internally by the library when you instruct the engine to use VAD (parameter vad=True). Those functions would be useful if you want to perform VAD only (or separately) from the ASR pipeline. It is not difficult to add those bindings, but I think only in very rare use cases one could justify them.