Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
d3ea038
merge main into refactor/124-download-source (resolve litellm/postgre…
0xdkay Apr 2, 2026
cfade97
test(133): add edge case and cross-layer contract tests for download-…
0xdkay Apr 3, 2026
b33d9b3
feat(infra): add builder-sidecar and runner-sidecar services
azchin Mar 24, 2026
e10a2c5
feat(libcrs): add sidecar API methods, rename run-test to apply-patch…
azchin Mar 24, 2026
ae3c338
refactor(framework): inject sidecars based on has_snapshot, not type=…
azchin Mar 24, 2026
cdec8c8
test(sidecar): add unit/integration tests and sandbox e2e test CRS
azchin Mar 24, 2026
a44e46e
feat(04-01): remove snapshot fields from Pydantic models and CRSConfi…
azchin Mar 24, 2026
4fd2652
refactor: remove snapshot/run_snapshot fields and all snapshot logic
azchin Mar 24, 2026
6286cbc
feat: always-on builder/runner sidecar with per-CRS DNS aliases
azchin Mar 24, 2026
831cf0d
feat: add --incremental-build flag with per-builder BASE_IMAGE env vars
azchin Mar 25, 2026
0a29afa
fix: use test-{build_id} snapshot tag format (no builder_name)
azchin Mar 25, 2026
55ec90e
feat: snapshot orchestration and pre-run validation in CRSCompose
azchin Mar 25, 2026
7823230
feat: test.sh runs in project image via PROJECT_BASE_IMAGE
azchin Mar 25, 2026
625e29c
fix: mount oss_crs_handler.sh into ephemeral containers, gate snapsho…
azchin Mar 26, 2026
7c0933a
refactor: rebuild_id-based artifact storage, parallel jobs, CRS resou…
azchin Mar 30, 2026
044bd22
feat: builder optional, builder_name param, phase-aware submit, stand…
azchin Mar 30, 2026
9304efd
feat: upload-build-output endpoint, runner rebuild_id, shared rebuild…
azchin Mar 30, 2026
b3effb2
fix: install libCRS as importable package via uv pip install --system
azchin Mar 30, 2026
2adbdb4
fix: forward OSS-Fuzz env vars into incremental snapshot containers
azchin Mar 30, 2026
47dafe8
fix: make test.sh snapshot optional, skip if test.sh missing or fails
azchin Mar 30, 2026
34027ab
feat: unified run_tests.sh with precedence (run_tests.sh > test.sh > …
azchin Mar 30, 2026
0ddf5ce
fix: auto-resolve builder_name from BASE_IMAGE_* env vars when not sp…
azchin Mar 31, 2026
6580b7e
fix: use docker cp for rebuild artifacts, runner resolves harness in …
azchin Mar 31, 2026
b21959e
fix: don't apply CRS mem_limit to ephemeral build containers (OOM on …
azchin Mar 31, 2026
d651568
Revert "fix: don't apply CRS mem_limit to ephemeral build containers …
azchin Mar 31, 2026
0eaee38
fix: snapshot lookup uses CLI build_id (from env), not rebuild_id
azchin Mar 31, 2026
d388925
fix: bake builder env vars into snapshots, merge image env in ephemer…
azchin Mar 31, 2026
14ece4d
fix: rework ephemeral rebuild to use shared volume instead of docker cp
azchin Mar 31, 2026
fa542ec
fix: update snapshot tags, preserve builder images, and fix tests
azchin Mar 31, 2026
9a19fe0
test: add builder-sidecar-full and builder-sidecar-lite integration t…
azchin Mar 31, 2026
7f10e5f
fix: resolve comments
azchin Apr 2, 2026
4e370a5
fix: magic string
azchin Apr 2, 2026
f201984
fix: deduplicate ephemeral container logic, fix test tmpdir bug, add …
azchin Apr 2, 2026
1a566b6
feat: content-hash snapshot dedup with file lock
azchin Apr 2, 2026
b01e3e2
fix: ephemeral builder needs OSS_CRS_PROJ_PATH
azchin Apr 2, 2026
a6e7b41
fix: ephemeral containers should not commit snapshots
azchin Apr 3, 2026
e1cd067
fix: content-hash snapshot dedup with file lock
azchin Apr 3, 2026
360252f
refactor: extract CI verify steps into scripts/verify.py
azchin Apr 3, 2026
3f27e87
fix: resolve ruff, ruff format, and pyright violations
azchin Apr 3, 2026
b824d9b
fix: persist ephemeral build logs to shared volume, read from filesystem
azchin Apr 3, 2026
3a0c06b
fix: update unit tests to match current sidecar and snapshot APIs
azchin Apr 3, 2026
848bf9b
fix: integration test_sidecar_lifecycle
azchin Apr 7, 2026
8c2727e
fix: sidecar integration test image
azchin Apr 7, 2026
b9c0c87
refactor: crs-codex registry
azchin Apr 7, 2026
9e9b68c
fix: sidecar integration test resources
azchin Apr 7, 2026
562e249
fix: ci patch smoke test hard exit and sidecar health check
azchin Apr 7, 2026
b87cfdf
fix: builder default field and silly things
azchin Apr 7, 2026
810055b
docs: changelog and builder sidecar
azchin Apr 7, 2026
5bb6a33
chore: ci ignore md
azchin Apr 7, 2026
cc6df5e
Merge pull request #166 from fab1ano/atlantis-java-url
fab1ano Apr 8, 2026
3657f7a
feat(compose): forward extra target env keys to builder-sidecar conta…
acorn421 Apr 9, 2026
1b297a9
feat(sidecar): support SIDECAR_PASSTHROUGH_KEYS for env forwarding
acorn421 Apr 9, 2026
340494d
feat(compose): merge CRS additional_env into sidecar passthrough keys
acorn421 Apr 9, 2026
68658b8
feat(compose): merge CRS additional_env into test snapshot and includ…
acorn421 Apr 9, 2026
919067d
refactor(sidecar-lite): use crs.run_pov instead of direct subprocess …
acorn421 Apr 9, 2026
f10d1ae
chore(registry): add builder-sidecar-full and builder-sidecar-lite re…
acorn421 Apr 9, 2026
558b098
Merge remote-tracking branch 'origin/main' into refactor/124-download…
azchin Apr 10, 2026
f60c77e
Merge pull request #133 from ossf/refactor/124-download-source
azchin Apr 10, 2026
018c730
Merge branch 'main' into feat/125-sidecar
azchin Apr 10, 2026
5dfd696
fix: import generate_random_name in target.py
azchin Apr 10, 2026
6772141
refactor: consolidate test_renderer_run_compose fixtures
azchin Apr 10, 2026
e938dea
style: apply ruff format to test_renderer_run_compose
azchin Apr 10, 2026
d775da4
fix(compose): increase docker client timeout to 300s for snapshot com…
acorn421 Apr 11, 2026
32306ff
fix(sidecar): handle non-utf8 output in run_pov subprocess
acorn421 Apr 11, 2026
c0b1e97
refactor(verify_patch): always write structured result via atexit hook
acorn421 Apr 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 7 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,9 @@ on:
- "oss_crs/**"
- "oss-crs-infra/**"
- "libCRS/**"

env:
RUFF_PATHS: oss_crs oss-crs-infra libCRS
PYRIGHT_PATHS: oss_crs/src libCRS/libCRS oss-crs-infra
- "uv.lock"
- "pyproject.toml"
- "!**/*.md"

jobs:
verify:
Expand All @@ -23,14 +22,8 @@ jobs:
with:
enable-cache: true
- run: uv sync --frozen --dev
- name: Ruff check
run: uv run ruff check $RUFF_PATHS
- name: Ruff format
run: uv run ruff format --check $RUFF_PATHS
- name: Pyright
run: uv run pyright $PYRIGHT_PATHS
- name: Unit Tests
run: uv run pytest -m "not integration" -v oss_crs
- name: Verify
run: uv run verify

integration-test:
name: Integration tests
Expand Down Expand Up @@ -253,7 +246,8 @@ jobs:
echo "Found patches:"
ls -la "$patch_dir"
else
echo "WARNING: No patches found (may be expected if LLM could not fix)"
echo "ERROR: No patches found"
exit 1
fi

- name: Create smoke-c-patch workdir tarball
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,5 @@ tags
.oss-crs-workdir/
third_party/
.planning/
docs/meeting/*/main.md
docs/meeting/*/*.svg
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ stricter subset of Keep a Changelog).
## [Unreleased]

### Added
- `--incremental-build` flag for `oss-crs build-target` and `oss-crs run` — creates Docker snapshots of compiled builder images for faster rebuilds across runs
- Framework-injected builder and runner sidecars during run phase — CRS developers no longer declare them in `crs.yaml`
- `libCRS apply-patch-test` command — applies a patch and runs the project's `test.sh` in a fresh ephemeral container
- `--early-exit` flag to `oss-crs run` to stop on the first discovered artifact (POV or patch)
- GitHub Actions CI pipeline with lint (ruff check), format check (ruff format), type check (pyright), unit tests, and parallel C/Java smoke tests
- atlantis-java-main to registry/ and example/
Expand All @@ -18,6 +21,10 @@ stricter subset of Keep a Changelog).
- `libCRS download-source target-source <dest>`: copies clean target source

### Changed
- Builder sidecar redesigned: framework-injected ephemeral containers replace CRS-declared long-running builders. Rebuilds launch a fresh container per patch from the preserved builder image.
- `libCRS apply-patch-build`: `--builder` no longer required (framework injects `BUILDER_MODULE`), `--builder-name` auto-detected. Response fields renamed: `retcode`, `rebuild_id`, `stdout.log`/`stderr.log`.
- `libCRS run-pov`: `--build-id` renamed to `--rebuild-id`, `--builder` no longer required.
- `libCRS apply-patch-test` replaces `run-test`: takes a patch file, applies it, and runs `test.sh` in a fresh container.
- Clarified that target env `repo_path` is the effective in-container source
path (Dockerfile final `WORKDIR`) used for `OSS_CRS_REPO_PATH`, not a host
path override.
Expand All @@ -44,6 +51,9 @@ stricter subset of Keep a Changelog).
future minor release.

### Removed
- `crs.yaml`: `snapshot` field from `target_build_phase`, `run_snapshot` field from `crs_run_phase` — snapshot behavior is now operator-controlled via `--incremental-build`
- `libCRS run-test` — replaced by `libCRS apply-patch-test`
- `OSS_CRS_SNAPSHOT_IMAGE` environment variable
- Removed legacy CLI alias `--target-repo-path`; use `--target-source-path`.
- Removed `libCRS download-source target` and `download-source repo` commands.
- Removed `SourceType.TARGET` and `SourceType.REPO` enum values from libCRS.
Expand Down
10 changes: 5 additions & 5 deletions PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,11 @@ Features that will significantly improve the platform's utility for real-world c

### Infrastructure Services

| Service | Status | Design Doc | Purpose |
|---|---|---|---|
| **Seed Deduplication** | 📝 Planned | [seed-dedup.md](docs/design/oss-crs-infra/seed-dedup.md) | Cross-CRS seed deduplication to reduce redundant fuzzing effort |
| **PoV Verification / Deduplication** | 📝 Planned | [pov-dedup.md](docs/design/oss-crs-infra/pov-dedup.md) | Verify crash inputs and deduplicate bugs found by multiple CRSs |
| **WebUI Dashboard** | 📝 Planned | [webui.md](docs/design/oss-crs-infra/webui.md) | Real-time monitoring: coverage metrics, bug candidates, PoV status, LLM usage |
| Service | Status | Purpose |
|---|---|---|
| **Seed Deduplication** | 📝 Planned | Cross-CRS seed deduplication to reduce redundant fuzzing effort |
| **PoV Verification / Deduplication** | 📝 Planned | Verify crash inputs and deduplicate bugs found by multiple CRSs |
| **WebUI Dashboard** | 📝 Planned | Real-time monitoring: coverage metrics, bug candidates, PoV status, LLM usage |

### Bug-Fixing Pipeline

Expand Down
137 changes: 0 additions & 137 deletions builder/README.md

This file was deleted.

55 changes: 0 additions & 55 deletions builder/oss-crs/crs.yaml

This file was deleted.

3 changes: 0 additions & 3 deletions builder/oss-crs/server.Dockerfile

This file was deleted.

9 changes: 3 additions & 6 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,7 @@ For a quick introduction and setup instructions, see the [project README](../REA
| [Architecture Overview](design/architecture.md) | System design, component diagram, and lifecycle walkthrough |
| [Parallel Builds and Runs](design/parallel.md) | Build/run isolation with `--build-id` and `--run-id` |
| [libCRS](design/libCRS.md) | CRS communication library — submit/fetch seeds, PoVs, and patches |
| [LiteLLM Integration](design/oss-crs-infra/litellm.md) | LLM proxy, per-CRS API keys, and budget enforcement |
| [Seed Deduplication](design/oss-crs-infra/seed-dedup.md) | Cross-CRS seed deduplication service (planned) |
| [PoV Deduplication](design/oss-crs-infra/pov-dedup.md) | Crash verification and deduplication service (planned) |
| [WebUI](design/oss-crs-infra/webui.md) | Monitoring dashboard for campaigns (planned) |
| [LLM Providers](llm-providers.md) | LiteLLM proxy setup for local and remote models |

## Key Concepts

Expand All @@ -43,8 +40,8 @@ For a quick introduction and setup instructions, see the [project README](../REA
Every CRS campaign follows three phases managed by `oss-crs`:

1. **Prepare** — Pull CRS source repositories and build Docker images (`oss-crs prepare`)
2. **Build Target** — Compile the target project and run each CRS's target build pipeline (`oss-crs build-target`)
3. **Run** — Launch all CRSs and shared infrastructure via Docker Compose (`oss-crs run`)
2. **Build Target** — Compile the target project and run each CRS's target build pipeline (`oss-crs build-target`). Pass `--incremental-build` to create Docker snapshots for faster rebuilds.
3. **Run** — Launch all CRSs and shared infrastructure via Docker Compose (`oss-crs run`). Pass `--incremental-build` to use snapshot images for ephemeral rebuild containers.

### CRS Isolation

Expand Down
42 changes: 3 additions & 39 deletions docs/config/crs.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,7 @@ The target build phase is a list of `BuildConfig` objects, each defining a named
|-------|------|----------|---------|-------------|
| `name` | `string` | Yes | - | The name of the build step |
| `dockerfile` | `string` | Yes | - | Path to the Dockerfile (must contain "Dockerfile" or end with `.Dockerfile`). Use an `oss-crs-infra:` prefix for framework-provided builds (e.g., `oss-crs-infra:default-builder`). |
| `outputs` | `list[string]` | No | `[]` | List of output paths. Can be empty for snapshot builds where no explicit outputs are needed. |
| `snapshot` | `bool` | No | `false` | When `true`, creates a snapshot Docker image during the build phase. The snapshot captures the fully compiled target and is used as the base image for incremental builds. |
| `outputs` | `list[string]` | No | `[]` | List of output paths to persist from the build. |
| `additional_env` | `dict[string, string]` | No | `{}` | Additional environment variables to pass during the build. Keys must match `[A-Za-z_][A-Za-z0-9_]*`. |

### Example
Expand All @@ -126,24 +125,6 @@ target_build_phase:
- cov-builder.tar.gz
```

### Example with Snapshot Builds

When using incremental builds, declare snapshot build steps that create reusable Docker images:

```yaml
target_build_phase:
- name: asan-snapshot
dockerfile: oss-crs-infra:default-builder
snapshot: true
additional_env:
SANITIZER: address
- name: coverage-snapshot
dockerfile: oss-crs-infra:default-builder
snapshot: true
additional_env:
SANITIZER: coverage
```

### Directed Build Inputs

For directed fuzzing workflows, `oss-crs build-target` can stage input artifacts into build containers:
Expand Down Expand Up @@ -185,8 +166,7 @@ The CRS run phase is a dictionary where each key is a module name and each value

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `dockerfile` | `string` | Conditional | - | Path to the Dockerfile (must contain "Dockerfile" or end with `.Dockerfile`). Can use `oss-crs-infra:` prefix for framework-provided services. **Required** when `run_snapshot` is `false`; **optional** when `run_snapshot` is `true`. |
| `run_snapshot` | `bool` | No | `false` | When `true`, uses the snapshot image from the build phase as this module's base image. Enables the module to use builder sidecar commands (`apply-patch-build`, `run-pov`, `run-test`). |
| `dockerfile` | `string` | Yes | - | Path to the Dockerfile (must contain "Dockerfile" or end with `.Dockerfile`). Can use `oss-crs-infra:` prefix for framework-provided services. |
| `additional_env` | `dict[string, string]` | No | `{}` | Additional environment variables to pass to the module. Keys must match `[A-Za-z_][A-Za-z0-9_]*`. |

### Example
Expand All @@ -203,23 +183,7 @@ crs_run_phase:
RUNNING_TIME_ENV: "XXX2"
```

### Example with Incremental Build Modules

Builder sidecar modules use `run_snapshot: true` to start from the snapshot image. The patcher module is a regular module that communicates with builder sidecars using the `--builder` flag on libCRS commands:

```yaml
crs_run_phase:
patcher:
dockerfile: oss-crs/docker-compose/patcher.Dockerfile
additional_env:
MAX_PATCHES: "100"
builder-asan:
dockerfile: oss-crs-infra:default-builder
run_snapshot: true
builder-coverage:
dockerfile: oss-crs-infra:default-builder
run_snapshot: true
```
**Note:** Builder and runner sidecars are injected automatically by the framework during the run phase. CRS developers do not need to declare them in `crs_run_phase`. The `BUILDER_MODULE` environment variable is set automatically.

### `additional_env` Key Rules

Expand Down
Loading