Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
efd610c
feat(skills): add demo-video skill for narrated video assembly
Jul 6, 2026
4341272
Merge branch 'main' into feat/demo-video-skill
rezatnoMsirhC Jul 9, 2026
c96b614
fix(skills): normalize demo-video copyright headers
Jul 9, 2026
e97818f
docs(skills): prioritize Azure AI Speech neural voices for demo-video…
Jul 9, 2026
3149051
test(skills): add demo-video eval coverage
Jul 9, 2026
c655057
fix(skills): use full words in demo-video eval patterns
Jul 9, 2026
1964f1d
fix(skills): address demo-video PR review feedback
Jul 9, 2026
c136b9c
fix(skills): update demo-video test call sites for dropped param
Jul 9, 2026
6fbe8ab
Merge branch 'main' into feat/demo-video-skill
WilliamBerryiii Jul 10, 2026
bf7e055
Merge branch 'main' into feat/demo-video-skill
jkim323 Jul 11, 2026
fea2e90
Merge remote-tracking branch 'origin/main' into feat/demo-video-skill
Jul 13, 2026
9c176b3
fix(skills): address demo-video review feedback on fps, type, and fuz…
Jul 13, 2026
a5b38c8
fix(skills): harden demo-video manifest validation and formatting
Jul 13, 2026
76e76fd
Merge branch 'main' into feat/demo-video-skill
auyidi1 Jul 14, 2026
a9d254a
fix(collections): regenerate hve-core-all markdown for demo-video and…
Jul 14, 2026
d51fa29
Merge branch 'main' into feat/demo-video-skill
auyidi1 Jul 15, 2026
c0b6597
Potential fix for pull request finding
auyidi1 Jul 15, 2026
f9af015
Potential fix for pull request finding
auyidi1 Jul 15, 2026
c30c7b8
Potential fix for pull request finding
auyidi1 Jul 15, 2026
cc499d4
Merge branch 'main' into feat/demo-video-skill
auyidi1 Jul 16, 2026
fcf59cd
Merge branch 'main' into feat/demo-video-skill
auyidi1 Jul 16, 2026
2f2d912
Merge branch 'main' into feat/demo-video-skill
auyidi1 Jul 16, 2026
dcf95b7
Merge branch 'main' into feat/demo-video-skill
WilliamBerryiii Jul 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions .github/skills/experimental/demo-video/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
name: demo-video
description: 'Assemble ordered frames or clips with narration into a narrated MP4 via FFmpeg'
license: MIT
compatibility: 'Requires FFmpeg on PATH'
metadata:
authors: "microsoft/hve-core"
spec_version: "1.0"
last_updated: "2026-07-09"
---

# Demo Video Assembly Skill

This skill assembles a narrated demo video from ordered visual segments and matching narration audio. It is designed for first-pass walkthrough videos that combine captured prototype frames or clips with per-segment voiceover WAV files.

## Overview

The workflow takes a manifest that describes each segment, resolves the visual source, and uses FFmpeg to render each segment into a normalized video clip before concatenating them into a final MP4. The narration track is muxed from WAV files so the output can be reviewed as a polished walkthrough without requiring a separate video-editing tool.

## Manifest Schema

Use a `segments.yml` manifest with an ordered list of segments. Each entry describes a visual source and the narration audio to combine for that portion of the video.

```yaml
segments:
- type: frame
visual: ./frames/intro.png
narration: ./audio/intro.wav
duration: 4.5
- type: clip
clip: ./clips/interaction.mp4
narration: ./audio/interaction.wav
```

### Segment fields

* `type` identifies whether the segment is a still image (`frame`) or a motion clip (`clip`)
* `visual` points to an image file for a frame segment
* `clip` points to a motion clip file for a clip segment
* `narration` points to the WAV file generated from narration text (the script also accepts `narration_wav` as an alias)
* `duration` is optional and overrides the inferred duration when you want a fixed segment length

## Quick Start

Comment thread
auyidi1 marked this conversation as resolved.
Use the bash or PowerShell wrappers to invoke the assembler from the skill directory.

```bash
scripts/assemble-video.sh --manifest examples/segments.yml --output ./output/demo.mp4
```

Comment on lines +28 to +50

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Manifest Schema section only shows the segments list. The three top-level keys that the assembler validates and uses (output, resolution, fps) are not documented, so a user reading SKILL.md alone would not know they exist or what their defaults are. The PR description's own schema example includes all three.

Consider expanding the schema example and field table to cover them:

output: ./output/demo.mp4   # optional; path for the assembled MP4
resolution: 1280x720        # optional; default 1280x720
fps: 24                     # optional; default 24
segments:
  - type: frame
    ...

And add field descriptions alongside the existing segment-field bullet list:

  • output sets the destination path for the assembled MP4 (resolved relative to the manifest; defaults to ./demo.mp4)
  • resolution controls output width and height in WIDTHxHEIGHT form (default 1280x720)
  • fps sets the frame rate applied when rendering each segment (default 24)

```powershell
scripts/Invoke-AssembleVideo.ps1 -ManifestPath examples/segments.yml -OutputPath ./output/demo.mp4
```

## Parameters Reference

The assembly step accepts the following high-level controls:

* `--manifest` or `-ManifestPath` selects the YAML manifest to process
* `--output` or `-OutputPath` sets the destination MP4 path
* `--fps` or `-Fps` controls the output frame rate for rendered segments
* `--resolution` or `-Resolution` controls the output width and height in the form `WIDTHxHEIGHT`
* `duration` per segment lets you override the inferred length when narration timing is known in advance

## Narration Quality

Narration quality is the single biggest driver of how polished the final video feels. Prioritize neural voices from **Azure AI Speech (part of Azure AI Foundry)** through the `tts-voiceover` skill for any video you intend to share.

* **Recommended:** Use the `tts-voiceover` skill backed by Azure AI Speech neural voices (for example `en-US-Andrew:DragonHDLatestNeural` or `en-US-Jenny:DragonHDLatestNeural`). These produce natural, presentation-grade narration and are the default for shareable output.
* **Fallback only:** Offline open-source engines such as `espeak-ng` require no credentials but sound noticeably robotic. Treat them as a no-network smoke-test fallback, not a delivery format. Regenerate narration with Azure AI Speech before publishing.

See the `tts-voiceover` skill for the neural voice catalog, `--voice` and `--rate` controls, and Azure authentication (Entra ID or key).

## Reuse Bridge

This skill is intentionally designed to fit into the existing media workflow:

* `tts-voiceover` provides the narration WAV files that this skill muxes into the final output; prefer its Azure AI Speech neural voices for production-quality narration
* `vscode-playwright` provides the frame-capture source for prototype walkthroughs and screen-based demos

## Prerequisites

FFmpeg and ffprobe must be available on your PATH.

### Linux

```bash
sudo apt update && sudo apt install ffmpeg
```

### macOS

```bash
brew install ffmpeg
```

### Windows

```powershell
winget install FFmpeg.FFmpeg
```
29 changes: 29 additions & 0 deletions .github/skills/experimental/demo-video/examples/README.md
Comment thread
auyidi1 marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
title: Demo Video Examples
description: Example manifest and fixture setup for the demo-video skill
author: Microsoft
ms.date: 2026-07-15
ms.topic: reference
keywords:
- video
- ffmpeg
- examples
estimated_reading_time: 2
---

## Demo Video Example Assets

This example manifest is designed to run with small fixture files that are kept out of source control. Create the following files locally before running the wrapper:

- examples/fixtures/intro.png
- examples/fixtures/intro.wav
- examples/fixtures/interaction.mp4
- examples/fixtures/interaction.wav

The fixtures can be generated from any short local assets, such as:

- a small PNG frame exported from a prototype screenshot
- a short WAV narration clip produced by the tts-voiceover skill (prefer Azure AI Speech neural voices for natural-sounding narration; offline open-source TTS is a no-credential fallback only)
- a short MP4 clip exported from a local browser recording or sample video

*🤖 Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.*
11 changes: 11 additions & 0 deletions .github/skills/experimental/demo-video/examples/segments.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
output: ./examples/output/demo.mp4
resolution: 1280x720
fps: 24
segments:
- type: frame
visual: ./examples/fixtures/intro.png
narration: ./examples/fixtures/intro.wav
duration: 3.0
- type: clip
clip: ./examples/fixtures/interaction.mp4
narration: ./examples/fixtures/interaction.wav
Comment on lines +1 to +11

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assembler resolves manifest-relative paths against manifest_path.parent (the examples/ directory). The paths in this file use the ./examples/... prefix, which causes them to resolve to <skill-root>/examples/examples/... — a location that does not exist. The example will fail on first run.

The examples/README.md documents fixtures at examples/fixtures/, which means the manifest (located at examples/segments.yml) should reference them as ./fixtures/....

Suggested fix:

output: ./output/demo.mp4
resolution: 1280x720
fps: 24
segments:
  - type: frame
    visual: ./fixtures/intro.png
    narration: ./fixtures/intro.wav
    duration: 3.0
  - type: clip
    clip: ./fixtures/interaction.mp4
    narration: ./fixtures/interaction.wav

30 changes: 30 additions & 0 deletions .github/skills/experimental/demo-video/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[project]
name = "demo-video-skill"
version = "0.0.0"
requires-python = ">=3.11"
dependencies = [
"pyyaml>=6.0",
]

[dependency-groups]
dev = [
"pytest>=9.0",
"pytest-cov>=5.0",
"pytest-mock>=3.14",
"ruff>=0.15",
]
fuzz = [
"atheris>=3.0",
]

[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["scripts"]
python_files = ["test_*.py", "fuzz_harness.py"]

[tool.ruff]
line-length = 88
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "I", "W"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#!/usr/bin/env pwsh
# Copyright (c) 2026 Microsoft Corporation. All rights reserved.
# SPDX-License-Identifier: MIT
#
# Invoke-AssembleVideo.ps1
# Wrapper for assemble_video.py that resolves the skill's Python environment
# and delegates FFmpeg video assembly.
#Requires -Version 7.4

<#
.SYNOPSIS
Assembles a narrated MP4 from image/clip segments and narration WAV files.

.DESCRIPTION
Resolves the demo-video skill's Python environment with uv and forwards
the supplied manifest and output arguments to assemble_video.py.

.PARAMETER ManifestPath
Path to the YAML manifest describing the visual segments.

.PARAMETER OutputPath
Destination MP4 path for the assembled video.

.PARAMETER Fps
Frame rate used when rendering each segment.

.PARAMETER Resolution
Output resolution in WIDTHxHEIGHT format.

.EXAMPLE
./Invoke-AssembleVideo.ps1 -ManifestPath examples/segments.yml -OutputPath ./output/demo.mp4
#>

[CmdletBinding()]
param(
[Parameter(Mandatory = $false)]
[string]$ManifestPath,

[Parameter(Mandatory = $false)]
[string]$OutputPath,

[Parameter(Mandatory = $false)]
[int]$Fps,

[Parameter(Mandatory = $false)]
[string]$Resolution
)

$ErrorActionPreference = 'Stop'

$ScriptDir = $PSScriptRoot
$SkillRoot = Split-Path $ScriptDir
$VenvDir = Join-Path $SkillRoot '.venv'

function Test-UvAvailability {
if (-not (Get-Command -Name 'uv' -ErrorAction SilentlyContinue)) {
throw "uv is required but was not found on PATH. See https://docs.astral.sh/uv/getting-started/installation/ (for example: 'winget install astral-sh.uv' on Windows, or 'curl -LsSf https://astral.sh/uv/install.sh | sh' on Linux/macOS)."
}
}

function Initialize-PythonEnvironment {
Write-Verbose 'Syncing Python environment via uv...'
& uv sync --directory $SkillRoot
if ($LASTEXITCODE -ne 0) {
throw 'uv sync failed'
}
}

function Get-VenvPythonPath {
if ($IsWindows) {
return Join-Path $VenvDir 'Scripts/python.exe'
}
return Join-Path $VenvDir 'bin/python'
}

if ($MyInvocation.InvocationName -ne '.') {
try {
Test-UvAvailability
Initialize-PythonEnvironment

$python = Get-VenvPythonPath
if (-not (Test-Path $python)) {
throw "Python not found at $python"
}

$script = Join-Path $ScriptDir 'assemble_video.py'
$PythonArgs = @()
if ($ManifestPath) { $PythonArgs += '--manifest', $ManifestPath }
if ($OutputPath) { $PythonArgs += '--output', $OutputPath }
if ($PSBoundParameters.ContainsKey('Fps')) { $PythonArgs += '--fps', $Fps }
if ($Resolution) { $PythonArgs += '--resolution', $Resolution }

& $python $script @PythonArgs
if ($LASTEXITCODE -ne 0) {
throw "assemble_video.py exited with code $LASTEXITCODE"
}
}
catch {
Write-Error -ErrorAction Continue "Invoke-AssembleVideo.ps1 failed: $($_.Exception.Message)"
exit 1
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Copyright (c) 2026 Microsoft Corporation. All rights reserved.
# SPDX-License-Identifier: MIT
#
# assemble-video.sh
# Wrapper for assemble_video.py that resolves the skill's Python environment
# and delegates FFmpeg video assembly.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_ROOT="$(dirname "${SCRIPT_DIR}")"

err() {
printf "ERROR: %s\n" "$1" >&2
exit 1
}

test_uv_availability() {
if ! command -v uv &>/dev/null; then
err "uv is required but was not found on PATH. Install with: curl -LsSf https://astral.sh/uv/install.sh | sh"
fi
}

initialize_python_environment() {
echo "Syncing Python environment via uv..."
uv sync --directory "${SKILL_ROOT}"
echo "Environment synchronized."
}

get_venv_python_path() {
echo "${SKILL_ROOT}/.venv/bin/python"
}

main() {
local python
local -a passthrough_args=("$@")

test_uv_availability
initialize_python_environment

python="$(get_venv_python_path)"
if [[ ! -x "${python}" ]]; then
err "Python not found at ${python}."
fi

"${python}" "${SCRIPT_DIR}/assemble_video.py" "${passthrough_args[@]}"
}

main "$@"
Loading
Loading