Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,12 @@ For agent-friendly repos, install the Vizzly skill and add a short project
vizzly init --agent-guidance
```

Use `vizzly init --agent-skill` to install only the local skill, or
This installs the portable Agent Skills bundle in `.agents/skills/vizzly` and
adds a small managed block to `AGENTS.md`. Run the same command again after a
CLI upgrade to refresh both without replacing the rest of `AGENTS.md` or an
existing Vizzly config.

Use `vizzly init --agent-skill` to install or refresh only the local skill, or
`vizzly init --skip-agent-skill` when you want config without the agent prompt.

### Start Local TDD
Expand Down Expand Up @@ -202,8 +207,8 @@ Generate a config file:
vizzly init
```

To teach project agents about Vizzly screenshot memory and the local visual TDD
loop, add the repo-local skill and AGENTS.md guidance:
To teach project agents the Vizzly evidence workflow, add the repo-local skill
and AGENTS.md guidance:

```bash
vizzly init --agent-guidance
Expand Down
5 changes: 5 additions & 0 deletions docs/json-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -1126,6 +1126,11 @@ vizzly init --agent-guidance --json
}
```

Run `vizzly init --agent-guidance --json` again after upgrading the CLI to
refresh the portable skill and Vizzly-owned `AGENTS.md` block. Existing output
then reports `"status": "refreshed"` for each updated artifact; unrelated
guidance and an existing Vizzly config remain unchanged.

### `vizzly project link`

```bash
Expand Down
94 changes: 42 additions & 52 deletions skills/vizzly/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,64 @@
---
name: vizzly
description: "Inspect, explain, and debug Vizzly visual regression evidence; use an existing Vizzly workflow before and after UI changes; add or adjust screenshot capture; troubleshoot local TDD or cloud builds; or configure Vizzly CI. Use when a task mentions Vizzly, screenshot baselines, comparisons, Honeydiff, visual review, dynamic regions, or a repository's Vizzly setup."
description: Inspect and explain Vizzly visual regression evidence, verify UI changes, troubleshoot local or cloud builds, adjust screenshot capture, or configure Vizzly CI. Use when a task mentions Vizzly, screenshot baselines, comparisons, Honeydiff, visual review, dynamic regions, or an existing Vizzly workflow.
---

# Vizzly

Use Vizzly as visual evidence for user-facing changes. Keep the repository's
existing test workflow in charge of how the UI is exercised.
Use Vizzly as evidence for user-facing changes. Keep the repository's existing
test workflow in charge of how the UI is exercised.

## Start From The Repository
## Prepare

1. Confirm that the repository uses Vizzly by checking its configuration,
dependencies, scripts, or existing `.vizzly` data.
2. Use the repository's established CLI invocation. The examples in this skill
use `vizzly`; substitute its package script, `pnpm exec vizzly`, or
`npx vizzly` when that is how the repository runs local binaries.
3. Do not install, initialize, log in, or change credentials unless the task
explicitly includes setup. Never print or persist a token.
1. Confirm the repository uses Vizzly by checking its config, dependencies,
scripts, or `.vizzly` data.
2. Use the repository's established CLI invocation exactly. Otherwise, replace
`vizzly` in these examples with `pnpm exec vizzly`; use `npx vizzly` when the
repository uses npm.
3. Do not install, initialize, log in, or change credentials unless setup is in
scope. Never print or persist a token.

## Follow The Evidence Loop
## Inspect And Verify

1. Choose the most relevant source:
- Use a supplied cloud build or comparison ID when the task names one.
- Use existing local context when `.vizzly` contains the run under review.
- Generate fresh evidence through the repository's existing visual test
workflow when stored evidence may be stale or absent.
2. Request the bounded, machine-readable build context:
1. Choose the supplied cloud build or comparison when one is named. Otherwise,
use current local evidence or find the relevant cloud build.
2. Request bounded JSON:

```bash
vizzly context build current --source local --agent --json
vizzly context build <build-id> --source cloud --agent --json
```

3. Check the build identity, source, branch, timestamps, baseline selection,
review state, and truncation fields before interpreting the evidence. Treat
omitted fields as unknown.
4. Inspect the current, baseline, and diff images together using whatever
image, browser, URL, or local-file capability is available. If an image is
inaccessible, say so and do not infer its visual contents from metadata.
5. Read viewport, browser, screenshot metadata, review state, and Honeydiff
facts alongside the images. Follow the returned `suggested_commands` for
exact comparison or screenshot drill-downs. Request raw diff regions only
when the compact summary is insufficient.
6. Separate observations from explanations. State what the evidence shows,
then label any proposed cause with appropriate confidence.
7. Make the smallest justified UI or test change, rerun the owning user
workflow, and inspect the resulting evidence again.
3. Confirm the build, source, branch, timestamps, baseline, review state, and
pagination before drawing conclusions. If `has_more` is true, run the
returned next-page command before concluding. Missing fields remain unknown.
4. Follow `suggested_commands` to inspect a comparison. View its baseline,
current, and diff images together. If an image cannot be opened, label the
result metadata-only; do not call it visual verification.
5. Read image dimensions, viewport, browser, diff regions, fingerprint, and
relevant history alongside the images. A prior approval is supporting
evidence, not permission to approve the current comparison.
6. State observations before possible causes. Make the smallest justified
change, rerun the owning workflow, and inspect the new evidence.

## Guardrails

- Treat an approved baseline as the accepted reference for that comparison,
not infallible truth. Check its identity and selection reason.
- Do not invent progress, counts, review state, visual causes, or missing API
- Do not invent progress, ranking, review state, visual causes, or missing API
values.
- Do not approve, reject, comment on, publish, or replace visual evidence
unless the user explicitly asks for that mutation.
- Preserve existing thresholds, cluster sizes, signature properties, and
dynamic-region behavior unless the task and evidence justify changing them.
- Prefer deterministic fixtures and existing end-to-end journeys over hiding a
diff with broader tolerances or a new screenshot-only test.
- Report the screenshot or comparison identity, the relevant build or link,
the observed evidence, any access limitations, and the command used.
- Do not approve, reject, comment on, publish, or replace evidence unless the
task explicitly asks for that mutation.
- Preserve thresholds, cluster sizes, signature properties, and dynamic-region
behavior unless repeated evidence justifies a change.
- Prefer deterministic fixtures and existing user journeys over hiding a diff
with broader tolerances or a new screenshot-only test.
- Report the comparison or screenshot identity, observed evidence, access
limitations, and the command or link used.

## Load References Only When Needed
## Load A Reference When Needed

- Read [references/cli-context.md](references/cli-context.md) for local and
cloud inspection commands, TDD lifecycle, and evidence drill-downs.
- Read [references/sdks.md](references/sdks.md) only when adding or changing
screenshot capture code.
- Read [references/dynamic-content.md](references/dynamic-content.md) when a
diff may involve unstable content, hotspots, or confirmed regions.
- Read [references/setup-ci.md](references/setup-ci.md) only when the task asks
to initialize Vizzly, change CI, or troubleshoot configuration.
- [CLI context](references/cli-context.md): local and cloud evidence, build
discovery, drill-downs, images, and TDD lifecycle.
- [SDK capture](references/sdks.md): add or change screenshot capture code.
- [Dynamic content](references/dynamic-content.md): investigate unstable
content, hotspots, or confirmed regions.
- [Setup and CI](references/setup-ci.md): initialize Vizzly or change CI.
4 changes: 0 additions & 4 deletions skills/vizzly/agents/openai.yaml

This file was deleted.

101 changes: 49 additions & 52 deletions skills/vizzly/references/cli-context.md
Original file line number Diff line number Diff line change
@@ -1,100 +1,97 @@
# CLI And Context
# CLI Context

Use the repository's existing CLI invocation. The examples use `vizzly` for
brevity; substitute the repository's package script or package-manager command
when needed.
Use the repository's established CLI invocation and existing authentication.
If cloud authentication is unavailable, report the blocker. Do not start an
interactive login unless setup is in scope.

Use existing authentication and project configuration. If cloud authentication
is missing, report the blocker. Do not start an interactive login or change
credentials unless the task includes setup.
## Choose The Evidence

## Inspect Existing Local Evidence
Use an ID supplied by the task. If no cloud build is supplied, list recent
builds and select the one matching the branch, commit, or pull request:

Request structured local evidence with both `--agent` and `--json`:
```bash
vizzly builds --branch <branch> --limit 5 --json
vizzly status <build-id> --json
vizzly context build <build-id> --source cloud --agent --json
```

Use status for lifecycle facts and build context for visual evidence. Do not
assume the first returned comparison is the most important; preserve API order
and inspect the records relevant to the task.

For saved local evidence:

```bash
vizzly context build current --source local --agent --json
vizzly context screenshot "<screenshot-name>" --source local --json
vizzly context review-queue --source local --json
```

Pin hand-written local drill-downs with `--source local`. Without it, automatic
source resolution may fall back to cloud data when a local item is unavailable.

Local context reads persisted `.vizzly` artifacts. Confirm that their build,
branch, timestamp, and baseline match the task before treating them as current.
Confirm the stored build, branch, timestamp, and baseline are current enough
for the task.

## Generate Fresh Local Evidence
## Generate Fresh Evidence

For a one-off run, let Vizzly own the complete local session:
For one run, let Vizzly own the local session:

```bash
vizzly tdd run "<existing visual test command>" --no-open
vizzly context build current --source local --agent --json
```

For repeated test runs, start the detached daemon once:
For repeated runs, start the detached daemon once:

```bash
vizzly tdd start --json
vizzly tdd status --json
<existing visual test command>
vizzly context build current --source local --agent --json
vizzly tdd stop --json
```

Treat `tdd run` and `tdd start` as alternatives. Do not shell-background
`tdd start`; it already launches a detached daemon. Stop only a server started
for the current task, and reuse the printed port for status or stop commands
when Vizzly selects a non-default port.
`tdd run` and `tdd start` are alternatives. Stop only a daemon started for the
current task.

## Inspect Cloud Evidence

When the task already has a build ID:
When a cloud build is in scope:

```bash
vizzly status <build-id> --json
vizzly run "<existing visual test command>" --wait --json
vizzly context build <build-id> --source cloud --agent --json
```

Use status for server-owned lifecycle, processing, comparison, and review
facts. Use build context for visual debugging.
## Inspect A Comparison

When creating a cloud build is in scope, wrap the repository's existing test
command:
Follow the build response's `suggested_commands`. The direct form is:

```bash
vizzly run "<existing visual test command>" --wait --json
vizzly context build <build-id> --source cloud --agent --json
vizzly context comparison <comparison-id> --source <local-or-cloud> --agent --json
vizzly context comparison <comparison-id> --source <local-or-cloud> --agent --include diffs --json
```

## Read And Drill Into Evidence
Open all three images together. Prefer `original_url` and fall back to `url`:

For each evidence record:
- Current: `comparison.screenshot.original_url` or
`comparison.screenshot.url`
- Baseline: `comparison.baseline.original_url` or `comparison.baseline.url`
- Diff: `comparison.analysis.diff_image_url`

1. Inspect the current, baseline, and diff images with an available harness
capability. State the limitation if an image cannot be accessed.
2. Read render metadata, review state, and compact Honeydiff diagnostics next
to the images.
3. Run the returned `suggested_commands` instead of reconstructing identifiers
or URLs.
4. Add `--include diffs` only when raw Honeydiff region geometry is needed.
Request comments only when human review context matters.
Then compare the visible change with diff regions, fingerprint, and the
separate `similar_by_fingerprint` and `recent_by_name` history streams.
Previous review decisions help explain recurring evidence but do not decide the
current review.

Useful manual drill-downs are:
Useful supporting commands:

```bash
vizzly context comparison <comparison-id> --source <local-or-cloud> --agent --json
vizzly context screenshot "<screenshot-name>" --source <local-or-cloud> --json
vizzly context similar <fingerprint-hash> --source cloud --json
vizzly context review-queue --source <local-or-cloud> --json
```

Use the source from the evidence you are inspecting in place of
`<local-or-cloud>`. `context similar` is cloud-only. Keep missing values
unknown, and do not turn metadata into a visual conclusion when the underlying
images are unavailable.
Use `--include diffs` only when compact diagnostics are insufficient. Request
comments only when human review context matters.

## Continue Without Guessing

When a build has more than 10 actionable records, run its returned next-page
command. The command uses `--offset` to preserve API order without pulling the
full build context into the agent handoff.
Run returned `suggested_commands` rather than reconstructing IDs, sources, or
pagination. When more evidence exists, the next-page command carries the API's
opaque `--cursor`; do not edit or interpret it. Keep follow-up commands pinned
to the source that produced the evidence.
47 changes: 18 additions & 29 deletions skills/vizzly/references/dynamic-content.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,28 @@
# Dynamic Content

Treat dates, timers, randomized data, API-backed content, generated images, and
responsive text as possible causes of change, not automatic explanations.
Dates, timers, random data, API content, generated images, and responsive text
can cause a diff. They are possible explanations, not conclusions.

## Diagnose Before Tuning

1. Inspect screenshot history and the actual image evidence:
1. Inspect the actual images and screenshot history:

```bash
vizzly context screenshot "<screenshot-name>" --source <local-or-cloud> --json
```

2. Replace `<local-or-cloud>` with the source of the evidence being diagnosed.
Record the observed region, recurrence, render metadata, and whether the
same change appears across builds or variants.
3. Check whether deterministic fixtures can remove irrelevant variation.
4. Treat hotspots and confirmed regions as server- or user-owned evidence.
They can affect comparison results, but this CLI exposes them for inspection
rather than authoring.
5. Preserve existing per-screenshot `threshold` and `minClusterSize` values
unless the task and repeated evidence justify a change. Do not invent new
tolerance values from one diff.

## Avoid

- Raising a global threshold for one unstable area.
- Masking a whole page when only a small region changes.
- Assuming a recurring change is harmless without inspecting it.
- Ignoring structural movement because a region contains dynamic content.
- Claiming a cause when only metadata, rather than images or history, is
available.

## Report Findings

State the observed change first. Then distinguish a likely explanation—such as
fixture drift, content disappearance, layout shift, capture timing, or baseline
mismatch—from unresolved alternatives. Include the screenshot identity and the
context command or link used.
2. Record the visible region, recurrence, render metadata, and whether the same
change appears across builds or variants.
3. Prefer deterministic fixtures when they can remove irrelevant variation.
4. Treat hotspots and confirmed regions as server- or user-authored evidence.
This CLI exposes them for inspection, not authoring.
5. Preserve `threshold` and `minClusterSize` unless repeated evidence
justifies a change.

Avoid broad masks, global threshold changes for one region, and claims based on
metadata alone. A recurring change can still be a bug, especially when content
movement affects nearby layout.

Report the observed change first. Label fixture drift, layout shift, capture
timing, or baseline mismatch as a possible cause until the evidence establishes
it.
Loading