Integrate NL2AL harms testing with current red-team infrastructure

[Copilot]

Replays the harms-specific work onto current main without modifying category/nl2al-harms-testing, while preserving the finalized PR #718 behavior.

• Harms testing

  • Add direct and indirect injection suites, fixtures, scoring, annotation, reporting, and objective harvesting.
  • Add bcbench harms commands and supporting datasets/documentation.

• BCal integration

  • Add harms fixtures, full-log capture, executable pinning, and version reporting.
  • Preserve prompt filtering, generated AL capture, and fail-loudly execution errors.

• Red-team integration

  • Adapt harvesting to the asynchronous message callback contract.
  • Support configurable objective counts and shared package-cache setup.
  • Keep Azure dependencies isolated to the optional redteam group.

• Safety and correctness

  • Use sandboxed Jinja rendering for harms templates.
  • Preserve lazy red-team CLI registration.
  • Add focused coverage for harms pipelines, BCal behavior, and callback integration.

Original prompt

continue the previous plan in the cloud

[Chronological Review: 1. User asked to investigate conflicts between merged PR #718 and `category/nl2al-harms-testing`, which was created from an earlier #718 state. 2. Repository inspection established: - `main`/`origin/main`: `64a096b5`. - #718 squash-merged as `39ed9361`. - #718 source branch: `category/nl2al-red-team`. - Harms branch tip: `49d6040f`. 3. History analysis found the harms branch contains early #718 commits `b66dec3e` and `98662f87`, followed by 18 harms-specific commits. Because #718 was squash-merged, Git cannot recognize the early implementation as equivalent to `39ed9361`. 4. A direct merge simulation found nine textual conflicts and a silent dependency conflict. 5. A disposable rebase simulation using `git rebase --onto origin/main 98662f8` stopped on the first of 18 commits with five conflicts, demonstrating that transplanting harms-only commits gives a cleaner starting point. 6. Conflict contents were inspected in disposable worktrees. No repository code was changed. 7. Disposable worktrees were removed; `main` remained clean. 8. Recommendation: create a new branch, replay only the 18 harms commits onto current `main`, preserve #718’s final semantics, validate, and push under a new name. 9. User explicitly instructed: “Do NOT touch this existing branch, feel free to create new branches.” 10. User approved the plan and requested cloud-agent implementation. 11. Agent investigated delegation support. Copilot CLI documentation identified `/delegate --base main`; GitHub CLI exposed `gh agent-task`. 12. The latest attempted command, `gh agent-task create --help`, was rejected by the user, who asked whether a skill or native mechanism should be used instead. Agent explained that no delegation skill is available, `/delegate` is interactive, and programmatic submission requires `gh agent-task create`.]

[Intent Mapping:

• “Now that PR Uptake AI Red Teaming Agent #718 has merged, let's take a look at category/nl2al-harms-testing”
• “Can you investigate what is the conflict? And how to best uptake in that branch?”
• “Why not just rebase Uptake AI Red Teaming Agent #718 onto this branch and resolve any conflict?”
• “Do NOT touch this existing branch, feel free to create new branches.”
• “Give me a step by step plan on how you plan to implement”
• “Sounds good. Can you contineu to implement in cloud agent?”
• Latest concern: “Don't you have a skill or something that do that? Or do we need gh?”]

[Technical Inventory:

• Git/GitHub:
  • Squash merge ancestry.
  • git merge-tree --write-tree --messages.
  • git rebase --onto.
  • Disposable Git worktrees.
  • git range-diff.
  • git push --force-with-lease, though the revised plan avoids modifying the original branch.
• GitHub CLI:
  • gh pr view, gh pr list.
  • gh agent-task preview commands: create, list, view.
• Copilot CLI:
  • Native /delegate --base main command.
• Python project:
  • uv dependency groups and lockfile.
  • Optional redteam dependency group.
  • Typer CLI registration.
  • Azure AI Evaluation/Red Teaming.
  • Async red-team callback contract.
  • Ruff and pytest validation.
• Branch strategy:
  • Preserve original remote branch.
  • Create a new uptake branch.
  • Exclude inherited early Uptake AI Red Teaming Agent #718 commits through cutoff 98662f87.
  • Replay 18 harms-specific commits onto current origin/main.]

[Code Archaeology:

• No files were modified in the real worktree.
• Direct merge conflict files:
  • .gitignore
    • Harms adds .scan_*/.
    • Main adds .bcbench/.
    • Resolution should retain both.
  • src/bcbench/agent/bcal/__init__.py
    • Harms exports bcal_version; main does not.
    • Resolution should retain main exports plus bcal_version.
  • src/bcbench/agent/bcal/agent.py
    • Harms adds bcal_version, harms_fixture_path, log_full_path, and “generated AL only” behavior.
    • Main adds _process_output, _trim_prompt_echo, _bcal_cmd_args, raises AgentTimeoutError/AgentError, and returns generated AL plus trimmed stdout.
    • Resolution must preserve main’s fail-loudly behavior and prompt filtering while adding harms options/version reporting.
  • src/bcbench/cli.py
    • Harms eagerly imports/registers redteam_app and harms_app.
    • Main lazily registers red-team commands due optional dependencies and registers contamination_app.
    • Resolution: retain main’s lazy red-team registration and add harms registration without eagerly importing Azure dependencies.
  • src/bcbench/commands/redteam.py
    • Target version is stale inherited Uptake AI Red Teaming Agent #718 code.
    • Main includes UTC timestamps, agent/model target option, external-command default, improved output handling and ASR rendering.
    • Harms-only commits did not modify this file; main should win.
  • src/bcbench/config.py
    • Harms adds harms_results.
    • Main adds plugin_root.
    • Retain both.
  • src/bcbench/redteam.py
    • Harms adds num_objectives, synchronous capturing-target as...

Created from Copilot CLI via the copilot delegate command.

[Sun Haoran (haoranpb)]

Took some time to go through the PR this week. Great work on the indirect/XPIA bit, also good call on checking the payload actually surfaced in a tool result before counting the trial.

Main thing I want to bring up is how much we can hand back to azure-ai-evaluation[redteam] so we own less code here.

Orchestration

Right now the harms runner drives the attacks itself, so payloads go out exactly as written in the YAML, we're not getting any of the agent's attack strategies (base64, flip, multi-turn) or its risk categories.

Could the target callback in build_bcal_target write the payload into the tainted seam and call bcal with the benign trigger, instead of passing it as the prompt? The agent just hands us text and reads back a response, so in principle it shouldn't care which door we deliver through, so we could keep the strategies and the scorecard.

On top of my head: one callback is one delivery vector, so maybe a new flag (e.g. --deliverity-vector) on the command similar like risk categories?

Scoring

Could the detector + BCQuality judge be custom evaluators inside evaluate() rather than a second scoring path with its own success metric? Agree CodeVulnerabilityEvaluator is too generic for AL.

BCal version

Is the --harms-fixture flag available now in the latest BCal CLI? If so, we can bump the version in the CI.