FEAT add CodeAttackConverter (closes #1945) - #1960
FEAT add CodeAttackConverter (closes #1945)#1960Utkarsh Bahuguna (u7k4rs6) wants to merge 9 commits into
Conversation
2f306cc to
88631e8
Compare
Utkarsh Bahuguna (u7k4rs6)
left a comment
Utkarsh Bahuguna (u7k4rs6)
left a comment
There was a problem hiding this comment.
PR Risk Summary
Quality Score: 9/10
Risk Level: low
Merge Recommendation: Safe to merge
Rationale: The changes appear to be focused on adding and refining code attack functionalities and their associated prompt converters. The review found no issues, and the changes are well-contained within the relevant modules. The addition of unit tests further strengthens the quality of this pull request.
| - file: code/executor/attack/4_sequential_attack.ipynb | ||
| - file: code/executor/attack/chunked_request_attack.ipynb | ||
| - file: code/executor/attack/context_compliance_attack.ipynb | ||
| - file: code/executor/attack/code_attack.ipynb |
There was a problem hiding this comment.
we just restructured the attack docs and almost certainly don't want a separate file for it. Can you see if it fits into one of the existing ones (after pulling in latest main)?
There was a problem hiding this comment.
e.g., flip attack is in https://microsoft.github.io/PyRIT/latest/code/executor/single-turn/
There was a problem hiding this comment.
Good catch. Rebased on main, and since the attack docs restructure removed code/executor/attack/, I dropped the standalone code_attack.ipynb and the myst.yml entry. Added a ## Code section to 1_single_turn right after Flip instead, plus a row in the attack table.
|
We need a references.bib update to include the paper. |
| "SingleTurnAttackStrategy", | ||
| "SingleTurnAttackContext", | ||
| "PromptSendingAttack", | ||
| "CodeAttackAttack", |
There was a problem hiding this comment.
That is not an ideal name 😆 CodeAttack is definitely better
| attack_scoring_config: AttackScoringConfig | None = None, | ||
| prompt_normalizer: PromptNormalizer | None = None, | ||
| max_attempts_on_failure: int = 0, | ||
| language: Literal["python_stack", "python_list", "python_string", "cpp", "go"] = "python_stack", |
There was a problem hiding this comment.
We've primarily been using enums for this. See style guide.
88631e8 to
76b74d6
Compare
…te enum, add bib entry, docs - Rename CodeAttackAttack -> CodeAttack (Task 1) - Collapse language + verbose into a single CodeAttackConverter.Template enum modelled on BinaryConverter.BitsPerChar; custom pathlib.Path still accepted for caller-supplied YAML templates (Task 2) - CodeAttack.__init__ now accepts template: CodeAttackConverter.Template | Path and forwards it to the converter; language/verbose params removed (Task 3) - Add @ren2024codeattack entry to doc/references.bib after liu2024flipattack (Task 4) - Add Code row to the attack table in 1_single_turn.py, add ## Code section after ## Flip mirroring the FlipAttack shape, regenerate notebook (Task 5) - Rebase onto upstream/main (doc/code/executor/attack/ directory was removed upstream; old standalone code_attack.ipynb/.py deleted, content moved into 1_single_turn.py) - Update all unit tests for the new Template-based API; add custom-Path cases
|
Roman Lutz (@romanlutz) Thanks for the thorough pass. Addressed everything: Rebased on main; folded the docs into 1_single_turn next to Flip, removed the separate file and myst.yml entry Open to dropping the class for converter-only if you'd rather. Re-requesting review. |
|
Tracked the framing variant in #2088. Resolving. |
9fc7c3c to
b500382
Compare
…te enum, add bib entry, docs - Rename CodeAttackAttack -> CodeAttack (Task 1) - Collapse language + verbose into a single CodeAttackConverter.Template enum modelled on BinaryConverter.BitsPerChar; custom pathlib.Path still accepted for caller-supplied YAML templates (Task 2) - CodeAttack.__init__ now accepts template: CodeAttackConverter.Template | Path and forwards it to the converter; language/verbose params removed (Task 3) - Add @ren2024codeattack entry to doc/references.bib after liu2024flipattack (Task 4) - Add Code row to the attack table in 1_single_turn.py, add ## Code section after ## Flip mirroring the FlipAttack shape, regenerate notebook (Task 5) - Rebase onto upstream/main (doc/code/executor/attack/ directory was removed upstream; old standalone code_attack.ipynb/.py deleted, content moved into 1_single_turn.py) - Update all unit tests for the new Template-based API; add custom-Path cases
Implement CodeAttack (Ren et al., ACL 2024) as a standalone converter
and a PromptSendingAttack subclass following the FlipAttack pattern.
CodeAttackConverter encodes a natural-language prompt word-by-word into
a data-structure initialisation sequence (deque appends, list appends,
or a string assignment) and embeds it in a partial code template that
asks the model to complete the code. Five language variants are
supported: python_stack, python_list, python_string, cpp, go. The
verbose flag selects the _plus template (detailed paragraphs) for the
three Python variants; cpp and go have no plus variant upstream.
CodeAttackAttack wraps the converter in a PromptSendingAttack, prepends
a system prompt that frames the session as code completion, and forwards
language and verbose to the converter. Callers supply a scorer via
AttackScoringConfig as usual.
Files added:
pyrit/prompt_converter/code_attack_converter.py
pyrit/executor/attack/single_turn/code_attack.py
pyrit/datasets/executors/code_attack.yaml
pyrit/datasets/prompt_converters/code_attack_python_stack{,_plus}.yaml
pyrit/datasets/prompt_converters/code_attack_python_list{,_plus}.yaml
pyrit/datasets/prompt_converters/code_attack_python_string{,_plus}.yaml
pyrit/datasets/prompt_converters/code_attack_cpp.yaml
pyrit/datasets/prompt_converters/code_attack_go.yaml
tests/unit/prompt_converter/test_code_attack_converter.py (23 tests)
tests/unit/executor/attack/single_turn/test_code_attack.py (16 tests)
doc/code/executor/attack/code_attack.py
doc/code/executor/attack/code_attack.ipynb
Files modified:
pyrit/prompt_converter/__init__.py
pyrit/executor/attack/single_turn/__init__.py
pyrit/executor/attack/__init__.py
doc/myst.yml
…te enum, add bib entry, docs - Rename CodeAttackAttack -> CodeAttack (Task 1) - Collapse language + verbose into a single CodeAttackConverter.Template enum modelled on BinaryConverter.BitsPerChar; custom pathlib.Path still accepted for caller-supplied YAML templates (Task 2) - CodeAttack.__init__ now accepts template: CodeAttackConverter.Template | Path and forwards it to the converter; language/verbose params removed (Task 3) - Add @ren2024codeattack entry to doc/references.bib after liu2024flipattack (Task 4) - Add Code row to the attack table in 1_single_turn.py, add ## Code section after ## Flip mirroring the FlipAttack shape, regenerate notebook (Task 5) - Rebase onto upstream/main (doc/code/executor/attack/ directory was removed upstream; old standalone code_attack.ipynb/.py deleted, content moved into 1_single_turn.py) - Update all unit tests for the new Template-based API; add custom-Path cases
Correct the authors to match the ren2024codeattack bib entry: Ren, Gao, Shao, Yan, Tan, Lam, Ma (SJTU / Shanghai AI Lab / CUHK). Remove incorrect names (Liu, Fan, Chen, Zhong, Lu, Wen) and replace Nanyang Technological University with Shanghai Jiao Tong University.
Remove CodeAttack attack class and its test file; wire code_attack as a PromptSendingAttack + CodeAttackConverter entry in scenario_techniques.py. Delete the now-orphaned executor system-prompt seed YAML. Update the single-turn executor doc and regenerate the notebook to show the converter-based usage pattern.
All five encoders (_encode_python_stack, _encode_python_list, _encode_python_string, _encode_cpp, _encode_go) now use json.dumps() to escape embedded double quotes and backslashes before interpolating into string literals. A prompt containing a double quote no longer produces malformed code. Also fix the class docstring: separator normalisation on [\s\-]+ applies only to python_stack; python_list uses str.split() and preserves hyphens. Add tests for embedded double quotes in all five encoder paths.
…ues/core.py Upstream (microsoft#2155) replaced initializers/components/scenario_techniques.py with initializers/techniques/core.py and renamed strategy_tags to technique_tags. The core group tag is now injected by build_technique_factories rather than stored in the factory. Port the code_attack entry and its test list update to the new layout.
Satisfies tests/unit/docs/test_converter_documentation.py, which requires every converter in pyrit.converter.__all__ to appear in a notebook under doc/code/converters/. The example lives in its own cell so the existing section 1.2 cell (which contains non-deterministic converters) keeps its stored output untouched. The converter is template-based and runs offline, so the new cell carries real executed output like every other cell on the page.
Every other factory in techniques/core.py carries a description; code_attack was the only one without. Matches the sibling register: single sentence, third-person present, states the mechanism.
b500382 to
5b5876e
Compare
|
Roman Lutz (@romanlutz) Rebased onto current main. The branch was ~150 commits behind, so this Also fixed a gap that predates the rebase: Converter, docs and setup suites green, pre-commit clean. Description |
|
hey Roman Lutz (@romanlutz) , no rush on the review itself, but #1960's CI is stuck awaiting maintainer approval so nothing has actually run on it yet. could you kick that off when you get a sec? also cleaned up the description, the attack class is gone and it's a converter + technique factory now, plus updated the bib entry to the ACL Findings version to match your #2409 pass 🙏 |
Utkarsh Bahuguna (u7k4rs6)
changed the title
4 participants
Closes #1945.
Summary
Implements CodeAttack (Ren et al., ACL 2024, arXiv:2403.07865), which reformulates a harmful query as a code-completion task. The query is encoded into a data-structure initialization sequence inside a partial code template with a
decode()stub, and the target is asked to complete the code. Because the intent is expressed as a programming task rather than a natural-language request, safety training keyed to natural language triggers less reliably. Black-box, no compute requirements.Two notes for review (deltas from the issue)
Encoding is word-by-word, not character-by-character. The issue described it as char-by-char (from the paper abstract), but the reference implementation (renqibing/CodeAttack) splits on whitespace and hyphens via regex, with character-level only as a fallback for single-token inputs. I matched the reference code. One consequence: separators are normalized on encode (hyphens and runs of whitespace are consumed as delimiters), which is documented in the converter docstring.
Eight templates, not five. The issue scoped five (one per language), but the reference ships eight: the three Python types each have a base and a
_plusverbose variant, and cpp and go have no verbose variant upstream. I included all eight to match the reference. Happy to drop the three_plusfiles if you would rather keep it to five.Design
CodeAttackConverterinpyrit/converter/code_attack_converter.py. It encodes the prompt into a data-structure initialisation sequence and renders a partial code template with adecode()stub. It is a standaloneConverterand composes through the normal converter pipeline.Template selection is the
CodeAttackConverter.Templateenum, which has eight members:PYTHON_STACKcode_attack_python_stackPYTHON_STACK_VERBOSEcode_attack_python_stack_plusPYTHON_LISTcode_attack_python_listPYTHON_LIST_VERBOSEcode_attack_python_list_plusPYTHON_STRINGcode_attack_python_stringPYTHON_STRING_VERBOSEcode_attack_python_string_plusCPPcode_attack_cppGOcode_attack_goThe default is
PYTHON_STACK_VERBOSE. Apathlib.Pathcan be passed instead to supply a custom template file.Eight seed prompts in
pyrit/datasets/converters/, matching the CodeChameleon convention:code_attack_python_stack.yaml,code_attack_python_stack_plus.yaml,code_attack_python_list.yaml,code_attack_python_list_plus.yaml,code_attack_python_string.yaml,code_attack_python_string_plus.yaml,code_attack_cpp.yaml,code_attack_go.yaml.No attack class. Per review feedback the earlier
CodeAttackAttacksubclass was dropped, and the technique is registered as thecode_attackfactory inpyrit/setup/initializers/techniques/core.py, wiring the converter ontoPromptSendingAttack.Tests
31 unit tests in
tests/unit/converter/test_code_attack_converter.py, covering per-template rendering, base vs_plusvariants, word-recovery round-trips, separator normalization, empty, special-character and long prompts, custompathlib.Pathtemplates, unsupported input types, and identifier construction.Factory registration is covered in
tests/unit/setup/test_technique_initializer.py, wherecode_attackis added toCORE_TECHNIQUE_NAMESand asserted against the built factory set in six places.Two documentation surfaces:
doc/code/converters/1_text_to_text_converters.py, section 1.2 Obfuscation Converters, next to CodeChameleon. The converter is template-based and runs offline, so the cell carries real executed output.doc/code/executor/1_single_turn.py, showing the converter applied throughAttackConverterConfig.The
ren2024codeattackentry indoc/references.bibcites the published ACL 2024 Findings paper.Files
New (10):
pyrit/converter/code_attack_converter.pypyrit/datasets/converters/code_attack_cpp.yamlpyrit/datasets/converters/code_attack_go.yamlpyrit/datasets/converters/code_attack_python_list.yamlpyrit/datasets/converters/code_attack_python_list_plus.yamlpyrit/datasets/converters/code_attack_python_stack.yamlpyrit/datasets/converters/code_attack_python_stack_plus.yamlpyrit/datasets/converters/code_attack_python_string.yamlpyrit/datasets/converters/code_attack_python_string_plus.yamltests/unit/converter/test_code_attack_converter.pyModified (8):
pyrit/converter/__init__.pypyrit/setup/initializers/techniques/core.pytests/unit/setup/test_technique_initializer.pydoc/references.bibdoc/code/converters/1_text_to_text_converters.pydoc/code/converters/1_text_to_text_converters.ipynbdoc/code/executor/1_single_turn.pydoc/code/executor/1_single_turn.ipynbNotes
Rebased onto main after the
PromptConvertertoConverterrename (upstream 3acaaa6), so the converter lives underpyrit/converter/, subclassesConverter, and the seed prompts moved topyrit/datasets/converters/.Total diff: 18 files changed, 991 insertions(+), 8 deletions(-).
Checklist