diff --git a/.agents/skills/crabbox-quickstart/SKILL.md b/.agents/skills/crabbox-quickstart/SKILL.md new file mode 100644 index 000000000..da07cc608 --- /dev/null +++ b/.agents/skills/crabbox-quickstart/SKILL.md @@ -0,0 +1,153 @@ +--- +name: crabbox-quickstart +description: "First contact with Crabbox: run your repository's tests inside a disposable Docker or Podman container on your own machine, no account and no cloud spend, then stop the box. Use when someone asks what Crabbox is or how to try it, wants a throwaway sandbox for a repo with no crabbox.yaml yet, or is about to run crabbox init here; hand off to the crabbox skill for config that already exists, leased remote machines, jobs, secrets, or artifacts." +license: MIT +--- + +# Crabbox Quickstart + +Crabbox runs your repository's commands on a disposable box — a container on +your own machine, or a remote machine you lease. It syncs your working tree, +runs one command, streams the output back, and exits with that command's code. + +## The loop + +```text +lease -> sync -> run -> read output -> stop +``` + +- **Lease** a box from a provider. `--provider local-container` uses Docker or + Podman on your own machine: no account, no login, no spend. +- **Sync** your current checkout onto it. Crabbox builds the file list from + Git, so the directory must be a repository. +- **Run** one command there. Its exit code becomes Crabbox's exit code. +- **Read** the streamed output, and optionally timing and test-result files. +- **Stop** the box. A one-shot `run` does this for you; a warm box does not. + +## Install and check + +```sh +crabbox --version # already installed? skip the install +brew install openclaw/tap/crabbox # or github.com/openclaw/crabbox/releases +crabbox doctor --provider local-container +``` + +`doctor` with a provider is the readiness check that matters: it names the +container runtime, the leases you already hold, and the `image=` your commands +will run inside. + +## First run, no account needed + +With Docker or Podman running, this works in any Git repository with no config +file, no login, and no cloud spend: + +```sh +git init # only if not already a repository +crabbox run --provider local-container -- uname -a +``` + +Crabbox provisioned a container, synced the dirty checkout, ran the command +there, streamed its output, propagated its exit code, and deleted the lease. +Budget 30-45 seconds once the base image is local, nearly all of it container +startup; the very first run adds a one-time image pull. + +## The box is bare + +Read this before swapping `uname -a` for real work. The default image is plain +Ubuntu with `git`, `curl`, `tar`, `python3`, `rsync`, and passwordless `sudo`. +No node, npm, make, gcc, go, cargo, or java. A command needing a runtime fails +before your code does — usually exit 127 and `make: not found`, or for npm a +preflight that stops the run first. Install what you need once, on a warm box: + +```sh +crabbox warmup --provider local-container # prints the +crabbox run --provider local-container --id -- \ + sudo apt-get install -y make +``` + +Your argv runs through a shell on the box, so `&&`, pipes, and redirects work. +Making the setup permanent is a `.crabbox.yaml` job — that is the `crabbox` +skill, not this page. + +## Warm a box and reuse it + +One-shot runs pay for container startup every time, and throw away whatever +you installed. Keep one box and send several commands to it instead: + +```sh +crabbox warmup --provider local-container +crabbox run --provider local-container --id -- ./run-tests.sh +crabbox status --provider local-container --id +``` + +`warmup` prints both a `cbx_...` lease id and a friendly slug; either works as +`--id`. A cold run measured 33-45 seconds, the same run warm about 4 seconds. +Above and below, `./run-tests.sh` stands for your own test command. + +## What actually gets synced + +Your dirty working tree filtered by Git, not a committed ref, so +untracked-but-not-ignored files do get uploaded — minus a built-in exclude +list: `node_modules`, `dist`, `target`, `.venv`, `__pycache__`, and other +dependency and build output. That never travels; rebuild it in the box. +`crabbox sync-plan` prints the file count, total bytes, and the largest files +and directories without starting a container. Exclude anything else surprising +in `.crabboxignore`. + +## Errors you will meet first + +Exit 2 is a missing `--provider`, 6 a directory that is not a Git repository, +7 a runtime Crabbox cannot reach, 4 an `--id` naming no live lease. Your own +command's code passes through verbatim too, so the number alone never says +which failed — read the message, then rerun the `doctor` command above. + +When your command itself fails, Crabbox exits with its code, prints a +failure digest with `next:` commands, and drops a bundle in +`.crabbox/captures/`. Sync excludes it; add `.crabbox/` to `.gitignore`. + +## Environment and secrets + +Nothing from your shell crosses into the box automatically; forwarding is an +allowlist by name, set with `--allow-env` or `env.allow` in the repo config. +Never put a token on a command line; the `crabbox` skill covers the rest. + +## Evidence from a run + +Evidence flags attach to any `run` and cost no extra time: + +```sh +crabbox run --provider local-container --results-auto -- ./run-tests.sh +``` + +If your command writes a JUnit XML report, `--results-auto` finds it without +being told the path and summarizes it in one line: + +```text +test results files=1 tests=3 failures=1 errors=0 skipped=1 +``` + +If nothing writes a JUnit file the flag is a silent no-op. + +## Stop what you started + +Boxes from `warmup` outlive the command; so do runs given `--keep`. Stop them: + +```sh +crabbox stop +crabbox list --provider local-container +``` + +An empty `list` means no leases remain on that provider. Local containers kept +by `warmup` or `--keep` require explicit `stop`; they do not expire on their own. + +## When you outgrow this page + +Stop here and load the full `crabbox` skill as soon as the task involves any of: + +- a repository that already has `crabbox.yaml` or `.crabbox.yaml`, including + one `crabbox init --detect` has just written +- any provider other than `local-container`, or a broker login +- a toolchain the base image lacks that you want present on every run +- named jobs, pools, prewarming, a fresh PR checkout, or Windows targets +- environment or secret forwarding, artifacts, or desktop and UI proof +- a failure that `crabbox doctor` output does not explain diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 88face155..672ba1e9d 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -6,7 +6,7 @@ on: - main paths: - "docs/**" - - "skills/crabbox/**" + - "skills/**" - "scripts/build-docs-site.mjs" - "scripts/enhance-docs-site.mjs" - "scripts/normalize-provider-counts.mjs" diff --git a/README.md b/README.md index 62d08cfc8..737f20563 100644 --- a/README.md +++ b/README.md @@ -181,27 +181,37 @@ tracked in https://github.com/openclaw/crabbox/issues/1157. See the [integration catalog](docs/integrations/README.md) for current support and lifecycle boundaries. -Existing repositories that only need agent discovery can install the generic -Skill with GitHub CLI: +Repositories that only need agent discovery can install Crabbox's published +Agent Skills with GitHub CLI: ```sh gh skill install openclaw/crabbox skills/crabbox \ --pin refs/heads/main --agent codex --scope project +gh skill install openclaw/crabbox skills/crabbox-quickstart \ + --pin refs/heads/main --agent codex --scope project ``` -Or use the cross-client Skills CLI: +Or install the sandbox execution skill with the [skills.sh](https://skills.sh) CLI: ```sh -npx skills add https://github.com/openclaw/crabbox --skill crabbox +npx skills add openclaw/crabbox --skill crabbox +npx skills add openclaw/crabbox --skill crabbox-quickstart ``` +Choose `crabbox` for sandbox execution and remote testing, or +`crabbox-quickstart` for a first local Docker/Podman run. Skills teach your agent +how to use Crabbox; install the CLI separately using the instructions above. +See the [skill installation guide](docs/integrations/agents.md#install-through-ecosystem-skill-managers) +for discovery and supported clients. + Crabbox also publishes a digest-verified discovery index from its own domain: ```sh npx skills add https://crabbox.sh --skill crabbox +npx skills add https://crabbox.sh --skill crabbox-quickstart ``` -Cross-vendor discovery services can index the same Skill through Crabbox's +Cross-vendor discovery services can index both Skills through Crabbox's [draft-compatible AI Catalog](https://crabbox.sh/.well-known/ai-catalog.json). Herdr users can add Crabbox lease controls and repository workflows to the diff --git a/docs/getting-started.md b/docs/getting-started.md index 48d703c51..cc85c2023 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -9,7 +9,11 @@ Read this when: This is a cookbook, not a reference. It walks through one repo from install to `crabbox run -- pnpm test`. Each step links to deeper docs when you want more. If you are still deciding whether Crabbox fits your workflow, start with -[What Crabbox is](README.md#what-crabbox-is). +[What Crabbox is](README.md#what-crabbox-is). For a first run with no account +at all, `--provider local-container` executes against Docker or Podman on your +own machine; the +[`crabbox-quickstart` skill](integrations/agents.md#install-through-ecosystem-skill-managers) +walks that credential-free path end to end. ## Step 1. Install diff --git a/docs/integrations/README.md b/docs/integrations/README.md index 02dde2830..1426871e0 100644 --- a/docs/integrations/README.md +++ b/docs/integrations/README.md @@ -18,6 +18,7 @@ or marketplace, even when they consume Crabbox. | Goal | Surface | Status | | --- | --- | --- | | Teach a local coding agent when and how to use Crabbox | [`crabbox init` Agent Skill](agents.md#local-agent-clients) | Available | +| Give a coding agent the shortest path from install to a first successful run | [`crabbox-quickstart` skill](agents.md#install-through-ecosystem-skill-managers) | Available | | Run a repo-owned one-shot harness remotely | [`crabbox run` or a named job](agents.md#one-shot-harnesses) | Credential-free run-evidence pattern available | | Reuse repository setup on a warm lease | [GitHub Actions hydration](../features/actions-hydration.md) | Available | | Use Zed as a local Crabbox control surface | [Zed extension package](editors.md#zed-control-surface) | Package available; [registry submission not yet opened](https://github.com/openclaw/crabbox/issues/1157) | diff --git a/docs/integrations/agents.md b/docs/integrations/agents.md index 350d4ef0a..576ccefe7 100644 --- a/docs/integrations/agents.md +++ b/docs/integrations/agents.md @@ -97,50 +97,70 @@ required `name` and `description` frontmatter manually. ### Install through ecosystem skill managers -Crabbox also publishes its generic Skill at the non-hidden -`skills/crabbox/SKILL.md` ecosystem installer convention. This makes the -authoritative generic Skill visible to installers instead of requiring them to -search Crabbox's repo-local `.agents` projection. +Crabbox also publishes its generic Skills at the non-hidden +`skills//SKILL.md` ecosystem installer convention. This makes the +authoritative generic Skills visible to installers instead of requiring them to +search Crabbox's repo-local `.agents` projection. Two ship today: + +- **`crabbox`** (`skills/crabbox`): run repository commands in sandbox + environments, reuse remote machines, and collect execution evidence. +- **`crabbox-quickstart`** (`skills/crabbox-quickstart`): get from CLI + installation to a first disposable local-container run and explicit cleanup. + +Install `crabbox-quickstart` for first contact, or when the repository has no +`crabbox.yaml` yet; it hands off to `crabbox` as soon as a task needs a +provider login, a named job, secrets, or artifacts. Install both when +newcomers and regular users share a repository. GitHub CLI 2.90 or newer maps Agent Skills into many host-specific locations: ```sh gh skill install openclaw/crabbox skills/crabbox \ --pin refs/heads/main --agent codex --scope project +gh skill install openclaw/crabbox skills/crabbox-quickstart \ + --pin refs/heads/main --agent codex --scope project ``` Replace `codex` with the target reported by `gh skill install --help`. The -cross-client Skills CLI can install the same source and prompt for a target: +cross-client Skills CLI can install the same sources and prompt for a target: ```sh npx skills add https://github.com/openclaw/crabbox --skill crabbox +npx skills add https://github.com/openclaw/crabbox --skill crabbox-quickstart ``` -The generic source has an [official-repository skills.sh -listing](https://www.skills.sh/openclaw/crabbox/crabbox). -The checked-in `skills/crabbox` source and `.agents/skills/crabbox` projection -are byte-identical and CI rejects drift. Use `crabbox init` when the repository -also needs Crabbox configuration, Actions hydration, and detected project-job -instructions; use a skill manager when only agent discovery is missing. +These are the GitHub sources used by [skills.sh](https://skills.sh). Its +[leaderboard discovers skills through CLI installation telemetry](https://skills.sh/docs/faq); +publishing a domain discovery index alone does not submit a listing. +Use `npx skills add openclaw/crabbox --list` to check available skills. +Installing a skill adds agent instructions; install the Crabbox CLI separately +to create and run sandboxes. +Every checked-in `skills/` source and its `.agents/skills/` +projection are byte-identical and CI rejects drift. Use `crabbox init` when the +repository also needs Crabbox configuration, Actions hydration, and detected +project-job instructions; use a skill manager when only agent discovery is +missing. `--pin refs/heads/main` selects this unreleased branch explicitly; after 0.40.0 is tagged, omit it to follow GitHub CLI's latest-release resolution. ### Discover from crabbox.sh -The docs build publishes the same Skill with a content digest through +The docs build publishes the same Skills with content digests through Cloudflare's [draft Agent Skills discovery protocol](https://github.com/cloudflare/agent-skills-discovery-rfc): - `https://crabbox.sh/.well-known/agent-skills/index.json` - `https://crabbox.sh/.well-known/agent-skills/crabbox/SKILL.md` +- `https://crabbox.sh/.well-known/agent-skills/crabbox-quickstart/SKILL.md` -The index declares the draft 0.2.0 schema and a SHA-256 digest of the exact +The index declares the draft 0.2.0 schema and a SHA-256 digest of each exact published `SKILL.md`. Clients that implement domain discovery can therefore find and verify Crabbox without a GitHub-specific registry. After the next docs deployment, the Skills CLI can consume the same endpoint directly: ```sh npx skills add https://crabbox.sh --skill crabbox +npx skills add https://crabbox.sh --skill crabbox-quickstart ``` Domain discovery is an emerging transport, not part of the core Agent Skills @@ -158,9 +178,9 @@ https://crabbox.sh/.well-known/ai-catalog.json ``` The catalog follows the draft [Agentic Resource Discovery -specification](https://github.com/ards-project/ard-spec), identifies the +specification](https://github.com/ards-project/ard-spec), identifies each artifact as `application/agent-skills+md`, and points to the same published -`SKILL.md`. It includes representative queries for remote testing, +`SKILL.md` files. It includes representative queries for remote testing, cross-platform validation, and auditable evidence so ARD-compatible discovery services can match Crabbox at task time. The site also advertises the catalog through an HTML `ai-catalog` link and an `Agentmap` directive in `robots.txt`. diff --git a/docs/source-map.md b/docs/source-map.md index 4da4645a0..95dd92fc3 100644 --- a/docs/source-map.md +++ b/docs/source-map.md @@ -54,13 +54,14 @@ Crabbox has three implementation surfaces: - The catalog inventories Crabbox-hosted surfaces. Host-owned integrations are versioned and inventoried in their host repositories rather than duplicated in this source map. -- Publishable generic Agent Skill: `skills/crabbox/SKILL.md`, with the - byte-identical repo-discovery projection at - `.agents/skills/crabbox/SKILL.md` and drift validation in +- Publishable generic Agent Skills: `skills/crabbox/SKILL.md` for the remote + execution surface and `skills/crabbox-quickstart/SKILL.md` for the + getting-started path, each with a byte-identical repo-discovery projection + at `.agents/skills//SKILL.md` and per-skill drift validation in `scripts/check-agent-skills.mjs`. The docs builder publishes the same bytes - plus a SHA-256 digest at `/.well-known/agent-skills/` for domain discovery, - and advertises the artifact through `/.well-known/ai-catalog.json` for - Agentic Resource Discovery. + plus a SHA-256 digest per skill at `/.well-known/agent-skills/` for domain + discovery, and advertises the artifacts through `/.well-known/ai-catalog.json` + for Agentic Resource Discovery. - Generated repo-local Agent Skill: `internal/cli/init.go`, with onboarding behavior in `docs/commands/init.md`. - Versioned editor handoff and foreground lease activity: diff --git a/scripts/build-docs-site.mjs b/scripts/build-docs-site.mjs index e7650bc6e..8c864e26c 100644 --- a/scripts/build-docs-site.mjs +++ b/scripts/build-docs-site.mjs @@ -15,6 +15,45 @@ const providerMetadata = JSON.parse( const providerMetadataByDocs = new Map( Object.entries(providerMetadata).map(([name, metadata]) => [metadata.docs, { name, metadata }]), ); +const skillsDir = path.join(root, "skills"); +// AI Catalog editorial metadata, one entry per skills/. +const catalogMetadata = { + crabbox: { + displayName: "Crabbox Agent Skill", + tags: ["remote-testing", "remote-execution", "developer-tools", "agent-skill"], + capabilities: [ + "RemoteTestExecution", + "ReusableRemoteEnvironment", + "CrossPlatformValidation", + "AuditableExecutionEvidence", + ], + representativeQueries: [ + "run this repository's tests on a clean remote machine", + "validate this change on Linux, macOS, or Windows", + "use Crabbox to collect auditable remote test evidence", + ], + }, + "crabbox-quickstart": { + displayName: "Crabbox Quickstart Skill", + tags: ["getting-started", "onboarding", "local-container", "docker", "test-execution", "developer-tools"], + capabilities: [ + "GuidedFirstRun", + "LocalContainerExecution", + "CredentialFreeEvaluation", + "ZeroConfigEvaluation", + "LeaseLifecycleHygiene", + ], + representativeQueries: [ + "run my repository's tests in a throwaway Docker container without a cloud account", + "what is Crabbox and how do I try it without an account", + "set up Crabbox in this repo that has no crabbox.yaml yet", + "why does crabbox say no provider selected", + ], + }, +}; + +// Parsed at module load because llms.txt is written before the discovery files. +const agentSkills = readAgentSkills(); const legacyProviderFeatureNotes = new Set([ "aws.md", "azure.md", @@ -103,47 +142,79 @@ writeAgentSkillsDiscovery(); writeAgentMap(); console.log(`built docs site: ${path.relative(root, outDir)}`); +// Preserve the original first entry; sort any additional skills by name. +export function readAgentSkills(sourceDir = skillsDir, metadata = catalogMetadata) { + return fs + .readdirSync(sourceDir, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort((a, b) => (a === b ? 0 : a === "crabbox" ? -1 : b === "crabbox" ? 1 : a < b ? -1 : 1)) + .map((name) => { + const sourcePath = path.join(sourceDir, name, "SKILL.md"); + const skill = fs.readFileSync(sourcePath, "utf8"); + const frontmatter = skill.match(/^---\n([\s\S]*?)\n---\n/); + if (!frontmatter) throw new Error(`${path.relative(root, sourcePath)} has no YAML frontmatter`); + + const declared = frontmatter[1].match(/^name:\s*([a-z0-9-]+)$/m)?.[1]; + const encodedDescription = frontmatter[1].match(/^description:\s*("(?:\\.|[^"\\])*")$/m)?.[1]; + if (!declared || !encodedDescription) { + throw new Error(`${path.relative(root, sourcePath)} must declare a quoted description and name`); + } + if (declared !== name) { + throw new Error(`${path.relative(root, sourcePath)} declares name ${declared} but lives in skills/${name}`); + } + const catalog = Object.hasOwn(metadata, name) ? metadata[name] : undefined; + if (!catalog) { + throw new Error(`skills/${name} has no AI Catalog metadata in build-docs-site.mjs`); + } + if (typeof catalog.displayName !== "string" || !catalog.displayName.trim()) { + throw new Error(`skills/${name} AI Catalog displayName must be a non-empty string`); + } + for (const field of ["tags", "capabilities", "representativeQueries"]) { + if (!Array.isArray(catalog[field]) || !catalog[field].length || + catalog[field].some((value) => typeof value !== "string" || !value.trim())) { + throw new Error(`skills/${name} AI Catalog ${field} must be a non-empty array of non-empty strings`); + } + } + return { + name, + skill, + catalog, + description: JSON.parse(encodedDescription), + digest: crypto.createHash("sha256").update(skill).digest("hex"), + }; + }); +} + function writeAgentSkillsDiscovery() { - const sourcePath = path.join(root, "skills", "crabbox", "SKILL.md"); - const skill = fs.readFileSync(sourcePath, "utf8"); - const frontmatter = skill.match(/^---\n([\s\S]*?)\n---\n/); - if (!frontmatter) throw new Error(`${path.relative(root, sourcePath)} has no YAML frontmatter`); - - const name = frontmatter[1].match(/^name:\s*([a-z0-9-]+)$/m)?.[1]; - const encodedDescription = frontmatter[1].match(/^description:\s*("(?:\\.|[^"\\])*")$/m)?.[1]; - if (!name || !encodedDescription) { - throw new Error(`${path.relative(root, sourcePath)} must declare a quoted description and name`); - } - const description = JSON.parse(encodedDescription); - const digest = crypto.createHash("sha256").update(skill).digest("hex"); const discoveryDir = path.join(outDir, ".well-known", "agent-skills"); - const publishedSkillDir = path.join(discoveryDir, name); - fs.mkdirSync(publishedSkillDir, { recursive: true }); - fs.writeFileSync(path.join(publishedSkillDir, "SKILL.md"), skill, "utf8"); + for (const { name, skill } of agentSkills) { + const publishedSkillDir = path.join(discoveryDir, name); + fs.mkdirSync(publishedSkillDir, { recursive: true }); + fs.writeFileSync(path.join(publishedSkillDir, "SKILL.md"), skill, "utf8"); + } fs.writeFileSync( path.join(discoveryDir, "index.json"), `${JSON.stringify( { $schema: "https://schemas.agentskills.io/discovery/0.2.0/schema.json", - skills: [ - { - name, - type: "skill-md", - description, - url: `/.well-known/agent-skills/${name}/SKILL.md`, - digest: `sha256:${digest}`, - }, - ], + skills: agentSkills.map(({ name, description, digest }) => ({ + name, + type: "skill-md", + description, + url: `/.well-known/agent-skills/${name}/SKILL.md`, + digest: `sha256:${digest}`, + })), }, null, 2, )}\n`, "utf8", ); - writeAICatalog({ name, description }); + writeAICatalog(); } -function writeAICatalog({ name, description }) { +function writeAICatalog() { const origin = docsOrigin(); if (!origin) throw new Error("Agentic Resource Discovery requires a canonical docs origin"); const catalog = { @@ -152,29 +223,18 @@ function writeAICatalog({ name, description }) { displayName: "Crabbox", documentationUrl: `${origin}/integrations/agents.html`, }, - entries: [ - { - identifier: `urn:air:crabbox.sh:skill:${name}`, - displayName: "Crabbox Agent Skill", - // Current AI Catalog integrated-ecosystem type. ARD's draft examples - // and bundled conformance helper still disagree on older alternatives. - type: "application/agent-skills+md", - url: `${origin}/.well-known/agent-skills/${name}/SKILL.md`, - description, - tags: ["remote-testing", "remote-execution", "developer-tools", "agent-skill"], - capabilities: [ - "RemoteTestExecution", - "ReusableRemoteEnvironment", - "CrossPlatformValidation", - "AuditableExecutionEvidence", - ], - representativeQueries: [ - "run this repository's tests on a clean remote machine", - "validate this change on Linux, macOS, or Windows", - "use Crabbox to collect auditable remote test evidence", - ], - }, - ], + entries: agentSkills.map(({ name, description, catalog: meta }) => ({ + identifier: `urn:air:crabbox.sh:skill:${name}`, + displayName: meta.displayName, + // Current AI Catalog integrated-ecosystem type. ARD's draft examples + // and bundled conformance helper still disagree on older alternatives. + type: "application/agent-skills+md", + url: `${origin}/.well-known/agent-skills/${name}/SKILL.md`, + description, + tags: meta.tags, + capabilities: meta.capabilities, + representativeQueries: meta.representativeQueries, + })), }; fs.writeFileSync( path.join(outDir, ".well-known", "ai-catalog.json"), @@ -219,7 +279,7 @@ function llmsTxt() { "", "Agent Skill and resource discovery:", `- ${origin}/.well-known/agent-skills/index.json`, - `- ${origin}/.well-known/agent-skills/crabbox/SKILL.md`, + ...agentSkills.map(({ name }) => `- ${origin}/.well-known/agent-skills/${name}/SKILL.md`), `- ${origin}/.well-known/ai-catalog.json`, ); } diff --git a/scripts/build-docs-site.test.js b/scripts/build-docs-site.test.js index 16bf41b18..5d83244ba 100644 --- a/scripts/build-docs-site.test.js +++ b/scripts/build-docs-site.test.js @@ -2,17 +2,18 @@ import assert from "node:assert/strict"; import crypto from "node:crypto"; import fs from "node:fs"; import path from "node:path"; +import os from "node:os"; import test from "node:test"; -import { markdownToHtml } from "./build-docs-site.mjs"; +import { markdownToHtml, readAgentSkills } from "./build-docs-site.mjs"; const repoRoot = path.resolve(import.meta.dirname, ".."); const providersDir = path.join(repoRoot, "docs", "providers"); const integrationsDir = path.join(repoRoot, "docs", "integrations"); const useCasesFile = path.join(repoRoot, "docs", "use-cases.md"); const siteDir = path.join(repoRoot, "dist", "docs-site"); -const providerIndexFile = path.join(siteDir, "providers", "index.html"); -const generatedTest = fs.existsSync(providerIndexFile) ? test : test.skip; +// Importing the builder generates the site before these tests register. +const generatedTest = test; const providerMarkdown = fs .readdirSync(providersDir) @@ -66,41 +67,35 @@ generatedTest("generated site publishes Agent Skill and AI Catalog discovery", ( assert.equal(published, canonical); assert.equal(index.$schema, "https://schemas.agentskills.io/discovery/0.2.0/schema.json"); - assert.deepEqual(index.skills, [ - { - name: "crabbox", - type: "skill-md", - description, - url: "/.well-known/agent-skills/crabbox/SKILL.md", - digest: `sha256:${digest}`, - }, - ]); - assert.deepEqual(catalog, { - specVersion: "1.0", - host: { - displayName: "Crabbox", - documentationUrl: "https://crabbox.sh/integrations/agents.html", - }, - entries: [ - { - identifier: "urn:air:crabbox.sh:skill:crabbox", - displayName: "Crabbox Agent Skill", - type: "application/agent-skills+md", - url: "https://crabbox.sh/.well-known/agent-skills/crabbox/SKILL.md", - description, - tags: ["remote-testing", "remote-execution", "developer-tools", "agent-skill"], - capabilities: [ - "RemoteTestExecution", - "ReusableRemoteEnvironment", - "CrossPlatformValidation", - "AuditableExecutionEvidence", - ], - representativeQueries: [ - "run this repository's tests on a clean remote machine", - "validate this change on Linux, macOS, or Windows", - "use Crabbox to collect auditable remote test evidence", - ], - }, + assert.deepEqual(index.skills[0], { + name: "crabbox", + type: "skill-md", + description, + url: "/.well-known/agent-skills/crabbox/SKILL.md", + digest: `sha256:${digest}`, + }); + assert.equal(catalog.specVersion, "1.0"); + assert.deepEqual(catalog.host, { + displayName: "Crabbox", + documentationUrl: "https://crabbox.sh/integrations/agents.html", + }); + assert.deepEqual(catalog.entries[0], { + identifier: "urn:air:crabbox.sh:skill:crabbox", + displayName: "Crabbox Agent Skill", + type: "application/agent-skills+md", + url: "https://crabbox.sh/.well-known/agent-skills/crabbox/SKILL.md", + description, + tags: ["remote-testing", "remote-execution", "developer-tools", "agent-skill"], + capabilities: [ + "RemoteTestExecution", + "ReusableRemoteEnvironment", + "CrossPlatformValidation", + "AuditableExecutionEvidence", + ], + representativeQueries: [ + "run this repository's tests on a clean remote machine", + "validate this change on Linux, macOS, or Windows", + "use Crabbox to collect auditable remote test evidence", ], }); assert.match( @@ -116,6 +111,72 @@ generatedTest("generated site publishes Agent Skill and AI Catalog discovery", ( /actions\/upload-pages-artifact@[^\n]+\n\s+with:\n\s+path: dist\/docs-site\n\s+include-hidden-files: true/, "Pages artifact must include the generated .well-known directory", ); + assert.match( + fs.readFileSync(path.join(repoRoot, ".github", "workflows", "pages.yml"), "utf8"), + /^\s+- "skills\/\*\*"$/m, + "Pages must redeploy when any publishable Agent Skill changes", + ); +}); + +generatedTest("every publishable skill appears once in discovery and the AI catalog", () => { + const skillsDir = path.join(repoRoot, "skills"); + const names = fs + .readdirSync(skillsDir, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort((a, b) => (a === b ? 0 : a === "crabbox" ? -1 : b === "crabbox" ? 1 : a < b ? -1 : 1)); + const index = JSON.parse( + fs.readFileSync(path.join(siteDir, ".well-known", "agent-skills", "index.json"), "utf8"), + ); + const catalog = JSON.parse( + fs.readFileSync(path.join(siteDir, ".well-known", "ai-catalog.json"), "utf8"), + ); + const llms = fs.readFileSync(path.join(siteDir, "llms.txt"), "utf8"); + + assert.deepEqual( + index.skills.map((skill) => skill.name), + names, + ); + assert.deepEqual( + catalog.entries.map((entry) => entry.identifier), + names.map((name) => `urn:air:crabbox.sh:skill:${name}`), + ); + + for (const [position, name] of names.entries()) { + const canonical = fs.readFileSync(path.join(skillsDir, name, "SKILL.md"), "utf8"); + const published = fs.readFileSync( + path.join(siteDir, ".well-known", "agent-skills", name, "SKILL.md"), + "utf8", + ); + assert.equal(published, canonical, `${name} should publish canonical bytes`); + assert.equal( + index.skills[position].digest, + `sha256:${crypto.createHash("sha256").update(published).digest("hex")}`, + `${name} digest should cover the published bytes`, + ); + assert.equal( + index.skills[position].description, + catalog.entries[position].description, + `${name} description should match across discovery surfaces`, + ); + const entry = catalog.entries[position]; + assert.ok( + entry.displayName && entry.tags.length && entry.capabilities.length, + `${name} needs display name, tags, and capabilities in the AI catalog`, + ); + assert.ok( + entry.representativeQueries.length >= 3, + `${name} needs at least three representative queries in the AI catalog`, + ); + assert.match( + llms, + new RegExp( + `^- https://crabbox\\.sh/\\.well-known/agent-skills/${escapeRegExp(name)}/SKILL\\.md$`, + "m", + ), + `${name} should be listed in llms.txt`, + ); + } }); generatedTest("generated navigation includes every integration page exactly once", () => { @@ -495,3 +556,50 @@ const voidElements = new Set([ "track", "wbr", ]); + + +test("skill discovery preserves crabbox first even when another skill sorts earlier", (t) => { + const directory = fs.mkdtempSync(path.join(os.tmpdir(), "crabbox-skills-")); + t.after(() => fs.rmSync(directory, { recursive: true, force: true })); + const metadata = {}; + for (const name of ["zeta", "crabbox", "alpha"]) { + fs.mkdirSync(path.join(directory, name)); + fs.writeFileSync(path.join(directory, name, "SKILL.md"), + `---\nname: ${name}\ndescription: "Use when testing ${name}"\n---\n`); + metadata[name] = { + displayName: name, + tags: ["sandbox"], + capabilities: ["Execution"], + representativeQueries: ["run a test"], + }; + } + assert.deepEqual(readAgentSkills(directory, metadata).map(({ name }) => name), + ["crabbox", "alpha", "zeta"]); + fs.rmSync(path.join(directory, "crabbox"), { recursive: true }); + assert.deepEqual(readAgentSkills(directory, metadata).map(({ name }) => name), ["alpha", "zeta"]); +}); + +test("skill discovery rejects malformed catalog metadata before publishing", (t) => { + const directory = fs.mkdtempSync(path.join(os.tmpdir(), "crabbox-catalog-")); + t.after(() => fs.rmSync(directory, { recursive: true, force: true })); + fs.mkdirSync(path.join(directory, "crabbox")); + fs.writeFileSync(path.join(directory, "crabbox", "SKILL.md"), + '---\nname: crabbox\ndescription: "Use when testing"\n---\n'); + const valid = { + displayName: "Crabbox", + tags: ["sandbox"], + capabilities: ["Execution"], + representativeQueries: ["run a test"], + }; + assert.throws(() => readAgentSkills(directory, {}), /no AI Catalog metadata/); + for (const value of [undefined, null, "", " ", 42, []]) { + assert.throws(() => readAgentSkills(directory, { crabbox: { ...valid, displayName: value } }), + /displayName must be a non-empty string/); + } + for (const field of ["tags", "capabilities", "representativeQueries"]) { + for (const value of [undefined, null, "sandbox", [], [""], [" "], [42], ["valid", null]]) { + assert.throws(() => readAgentSkills(directory, { crabbox: { ...valid, [field]: value } }), + new RegExp(`${field} must be a non-empty array of non-empty strings`)); + } + } +}); diff --git a/scripts/check-agent-skills.mjs b/scripts/check-agent-skills.mjs index e98d8d915..85a99c487 100644 --- a/scripts/check-agent-skills.mjs +++ b/scripts/check-agent-skills.mjs @@ -4,24 +4,50 @@ import fs from "node:fs"; import path from "node:path"; const root = process.cwd(); -const canonicalPath = path.join(root, "skills", "crabbox", "SKILL.md"); -const projectionPath = path.join(root, ".agents", "skills", "crabbox", "SKILL.md"); -const canonical = fs.readFileSync(canonicalPath, "utf8"); -const projection = fs.readFileSync(projectionPath, "utf8"); +const skillsDir = path.join(root, "skills"); +const names = fs + .readdirSync(skillsDir, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => entry.name) + .sort(); +assert.ok(names.length, "skills/ must contain at least one publishable Agent Skill"); -assert.equal( - projection, - canonical, - ".agents/skills/crabbox/SKILL.md must remain byte-identical to skills/crabbox/SKILL.md", -); -assert.match(canonical, /^---\nname: crabbox\ndescription: "[^"]*Use when[^"]*"\nlicense: MIT\n---\n/); +// A quickstart that grows into a second full reference stops being a quickstart, +// so its budget is a CI contract rather than an editorial intention. +const defaultLineCap = 500; +const lineCaps = { "crabbox-quickstart": 155 }; + +for (const name of names) { + const canonical = fs.readFileSync(path.join(skillsDir, name, "SKILL.md"), "utf8"); + const projection = fs.readFileSync(path.join(root, ".agents", "skills", name, "SKILL.md"), "utf8"); + + assert.equal( + projection, + canonical, + `.agents/skills/${name}/SKILL.md must remain byte-identical to skills/${name}/SKILL.md`, + ); + assert.match( + canonical, + new RegExp(`^---\\nname: ${name}\\ndescription: "[^"]*Use when[^"]*"\\nlicense: MIT\\n---\\n`), + ); + + const lineCount = canonical.split("\n").length - (canonical.endsWith("\n") ? 1 : 0); + const cap = lineCaps[name] ?? defaultLineCap; + assert.ok(lineCount <= cap, `skills/${name}/SKILL.md should stay at or below ${cap} lines; found ${lineCount}`); + console.log(`validated publishable Agent Skill ${name}: ${lineCount} lines, canonical and projection identical`); +} + +// Crabbox-specific: pins the exact detection trigger and heading emitted by +// skillTemplate() in internal/cli/init.go, and asserts init.go still emits it, +// so the CLI and the published Skill cannot drift apart. +const detectionTrigger = /Use when crabbox\.yaml or \.crabbox\.yaml exists, the crabbox CLI is available/; +const crabbox = fs.readFileSync(path.join(skillsDir, "crabbox", "SKILL.md"), "utf8"); +assert.match(crabbox, detectionTrigger); +assert.match(crabbox, /\n---\n\n# Crabbox\n/); assert.match( - canonical, - /Use when crabbox\.yaml or \.crabbox\.yaml exists, the crabbox CLI is available/, + fs.readFileSync(path.join(root, "internal", "cli", "init.go"), "utf8"), + detectionTrigger, + "skillTemplate() in internal/cli/init.go must emit the same detection trigger as skills/crabbox", ); -assert.match(canonical, /\n---\n\n# Crabbox\n/); - -const lineCount = canonical.split("\n").length - (canonical.endsWith("\n") ? 1 : 0); -assert.ok(lineCount <= 500, `Crabbox Skill should stay at or below 500 lines; found ${lineCount}`); -console.log(`validated publishable Crabbox Agent Skill: ${lineCount} lines, canonical and projection identical`); +console.log(`validated ${names.length} publishable Agent Skill${names.length === 1 ? "" : "s"}: ${names.join(", ")}`); diff --git a/scripts/docs-ui-proof.mjs b/scripts/docs-ui-proof.mjs index c12413e7a..89f429804 100644 --- a/scripts/docs-ui-proof.mjs +++ b/scripts/docs-ui-proof.mjs @@ -35,6 +35,8 @@ for (const artifact of [ "home-desktop-light.png", "home-desktop-dark.png", "home-mobile.png", + "skills-desktop.png", + "skills-mobile.png", "interaction-proof.json", "SHA256SUMS", ]) { @@ -115,6 +117,7 @@ try { await openHome(desktopLight.page); await assertHomeShell(desktopLight.page, "desktop light restored"); await screenshot(desktopLight.page, "home-desktop-light.png"); + await proveSkillGuide(desktopLight.page, "desktop"); await assertNoPageErrors(desktopLight); await desktopLight.context.close(); activeProofPage = undefined; @@ -150,6 +153,7 @@ try { await assertTheme(mobile.page, "light"); await assertHomeShell(mobile.page, "mobile light"); await screenshot(mobile.page, "home-mobile.png"); + await proveSkillGuide(mobile.page, "mobile"); await assertNoPageErrors(mobile); await mobile.context.close(); activeProofPage = undefined; @@ -847,9 +851,9 @@ async function featureState(page) { }); } -async function screenshot(page, file) { +async function screenshot(page, file, fullPage = true) { const target = path.join(outDir, file); - await page.screenshot({ path: target, fullPage: true, animations: "disabled", caret: "hide", scale: "css" }); + await page.screenshot({ path: target, fullPage, animations: "disabled", caret: "hide", scale: "css" }); const buffer = fs.readFileSync(target); const dimensions = pngDimensions(buffer); const artifact = { @@ -922,3 +926,39 @@ function pngDimensions(buffer) { height: buffer.readUInt32BE(20), }; } + + +async function proveSkillGuide(page, viewport) { + await page.goto(`${baseURL}/integrations/agents.html`, { waitUntil: "domcontentloaded" }); + const guide = page.locator("#install-through-ecosystem-skill-managers"); + await guide.waitFor({ state: "visible" }); + const layout = await guide.evaluate((heading) => { + const article = heading.closest("article"); + const bounds = article.getBoundingClientRect(); + const items = []; + for (let node = heading.nextElementSibling; node && !/^H[1-3]$/.test(node.tagName); node = node.nextElementSibling) { + if (node.tagName === "UL") { + for (const item of node.querySelectorAll("li")) { + const rect = item.getBoundingClientRect(); + items.push({ text: item.textContent, left: rect.left, right: rect.right, + width: item.clientWidth, scrollWidth: item.scrollWidth }); + } + } + } + return { left: bounds.left, right: bounds.right, items }; + }); + record(`${viewport}: both skill choices fit the article without horizontal scrolling`, + layout.items.length === 2 && layout.items.every((item) => + item.left >= layout.left && item.right <= layout.right + 1 && item.scrollWidth <= item.width + 1), layout); + await guide.evaluate((heading) => heading.scrollIntoView({ block: "start" })); + await screenshot(page, `skills-${viewport}.png`, false); + const response = await page.request.get(`${baseURL}/.well-known/agent-skills/index.json`); + const index = await response.json(); + record(`${viewport}: both installable skills are discoverable`, + response.ok() && ["crabbox", "crabbox-quickstart"].every((name) => index.skills.some((skill) => skill.name === name))); + for (const skill of index.skills) { + const published = await page.request.get(`${baseURL}${skill.url}`); + record(`${viewport}: ${skill.name} download matches its discovery digest`, + published.ok() && `sha256:${sha256(await published.body())}` === skill.digest); + } +} diff --git a/skills/crabbox-quickstart/SKILL.md b/skills/crabbox-quickstart/SKILL.md new file mode 100644 index 000000000..da07cc608 --- /dev/null +++ b/skills/crabbox-quickstart/SKILL.md @@ -0,0 +1,153 @@ +--- +name: crabbox-quickstart +description: "First contact with Crabbox: run your repository's tests inside a disposable Docker or Podman container on your own machine, no account and no cloud spend, then stop the box. Use when someone asks what Crabbox is or how to try it, wants a throwaway sandbox for a repo with no crabbox.yaml yet, or is about to run crabbox init here; hand off to the crabbox skill for config that already exists, leased remote machines, jobs, secrets, or artifacts." +license: MIT +--- + +# Crabbox Quickstart + +Crabbox runs your repository's commands on a disposable box — a container on +your own machine, or a remote machine you lease. It syncs your working tree, +runs one command, streams the output back, and exits with that command's code. + +## The loop + +```text +lease -> sync -> run -> read output -> stop +``` + +- **Lease** a box from a provider. `--provider local-container` uses Docker or + Podman on your own machine: no account, no login, no spend. +- **Sync** your current checkout onto it. Crabbox builds the file list from + Git, so the directory must be a repository. +- **Run** one command there. Its exit code becomes Crabbox's exit code. +- **Read** the streamed output, and optionally timing and test-result files. +- **Stop** the box. A one-shot `run` does this for you; a warm box does not. + +## Install and check + +```sh +crabbox --version # already installed? skip the install +brew install openclaw/tap/crabbox # or github.com/openclaw/crabbox/releases +crabbox doctor --provider local-container +``` + +`doctor` with a provider is the readiness check that matters: it names the +container runtime, the leases you already hold, and the `image=` your commands +will run inside. + +## First run, no account needed + +With Docker or Podman running, this works in any Git repository with no config +file, no login, and no cloud spend: + +```sh +git init # only if not already a repository +crabbox run --provider local-container -- uname -a +``` + +Crabbox provisioned a container, synced the dirty checkout, ran the command +there, streamed its output, propagated its exit code, and deleted the lease. +Budget 30-45 seconds once the base image is local, nearly all of it container +startup; the very first run adds a one-time image pull. + +## The box is bare + +Read this before swapping `uname -a` for real work. The default image is plain +Ubuntu with `git`, `curl`, `tar`, `python3`, `rsync`, and passwordless `sudo`. +No node, npm, make, gcc, go, cargo, or java. A command needing a runtime fails +before your code does — usually exit 127 and `make: not found`, or for npm a +preflight that stops the run first. Install what you need once, on a warm box: + +```sh +crabbox warmup --provider local-container # prints the +crabbox run --provider local-container --id -- \ + sudo apt-get install -y make +``` + +Your argv runs through a shell on the box, so `&&`, pipes, and redirects work. +Making the setup permanent is a `.crabbox.yaml` job — that is the `crabbox` +skill, not this page. + +## Warm a box and reuse it + +One-shot runs pay for container startup every time, and throw away whatever +you installed. Keep one box and send several commands to it instead: + +```sh +crabbox warmup --provider local-container +crabbox run --provider local-container --id -- ./run-tests.sh +crabbox status --provider local-container --id +``` + +`warmup` prints both a `cbx_...` lease id and a friendly slug; either works as +`--id`. A cold run measured 33-45 seconds, the same run warm about 4 seconds. +Above and below, `./run-tests.sh` stands for your own test command. + +## What actually gets synced + +Your dirty working tree filtered by Git, not a committed ref, so +untracked-but-not-ignored files do get uploaded — minus a built-in exclude +list: `node_modules`, `dist`, `target`, `.venv`, `__pycache__`, and other +dependency and build output. That never travels; rebuild it in the box. +`crabbox sync-plan` prints the file count, total bytes, and the largest files +and directories without starting a container. Exclude anything else surprising +in `.crabboxignore`. + +## Errors you will meet first + +Exit 2 is a missing `--provider`, 6 a directory that is not a Git repository, +7 a runtime Crabbox cannot reach, 4 an `--id` naming no live lease. Your own +command's code passes through verbatim too, so the number alone never says +which failed — read the message, then rerun the `doctor` command above. + +When your command itself fails, Crabbox exits with its code, prints a +failure digest with `next:` commands, and drops a bundle in +`.crabbox/captures/`. Sync excludes it; add `.crabbox/` to `.gitignore`. + +## Environment and secrets + +Nothing from your shell crosses into the box automatically; forwarding is an +allowlist by name, set with `--allow-env` or `env.allow` in the repo config. +Never put a token on a command line; the `crabbox` skill covers the rest. + +## Evidence from a run + +Evidence flags attach to any `run` and cost no extra time: + +```sh +crabbox run --provider local-container --results-auto -- ./run-tests.sh +``` + +If your command writes a JUnit XML report, `--results-auto` finds it without +being told the path and summarizes it in one line: + +```text +test results files=1 tests=3 failures=1 errors=0 skipped=1 +``` + +If nothing writes a JUnit file the flag is a silent no-op. + +## Stop what you started + +Boxes from `warmup` outlive the command; so do runs given `--keep`. Stop them: + +```sh +crabbox stop +crabbox list --provider local-container +``` + +An empty `list` means no leases remain on that provider. Local containers kept +by `warmup` or `--keep` require explicit `stop`; they do not expire on their own. + +## When you outgrow this page + +Stop here and load the full `crabbox` skill as soon as the task involves any of: + +- a repository that already has `crabbox.yaml` or `.crabbox.yaml`, including + one `crabbox init --detect` has just written +- any provider other than `local-container`, or a broker login +- a toolchain the base image lacks that you want present on every run +- named jobs, pools, prewarming, a fresh PR checkout, or Windows targets +- environment or secret forwarding, artifacts, or desktop and UI proof +- a failure that `crabbox doctor` output does not explain