fix(tailor): expand diff scope + preserve casing (#805); e2e-monitor JD-keyword tolerance

[srbhr]

Fixes #805 (AI tailor only ever rewrote the same ~5 descriptions + summary, and lowercased text). A clean re-implementation of the useful parts of #809, plus a maintainer-requested e2e-monitor tuning.

Why not just merge #809

I reviewed #809 closely (security + correctness). It's not malicious (no exec/eval/network/deps/CI touched), but pulling it as-is has three problems, so this PR reworks it:

1. Undisclosed regression: fix(tailor): expand diff scope and preserve casing (fixes #805) #809 overwrites diff-prompt rules Bump django from 3.2.11 to 3.2.12 #10/Bump pillow from 9.0.0 to 9.0.1 #11 (verified skill targets) and removes the add_skill path while improver.py still computes/accepts them → prompt↔code desync that silently disables JD-sourced skill additions. Here we keep Bump django from 3.2.11 to 3.2.12 #10/Bump pillow from 9.0.0 to 9.0.1 #11 and add casing as rule Bump notebook from 6.4.1 to 6.4.10 #12.
2. Latent crash: fix(tailor): expand diff scope and preserve casing (fixes #805) #809 emits field_type="language"/"award" but never updates the ResumeFieldDiff Literal in models.py → Pydantic ValidationError when those diffs are built. Fixed (backend Literal + frontend union).
3. Correctness nit: fix(tailor): expand diff scope and preserve casing (fixes #805) #809 allows education[i].description[j] (list/bullet), but Education.description is a scalar str | None. Here it's scalar-only, with a test asserting the [j] form is rejected.
4. Scope creep: fix(tailor): expand diff scope and preserve casing (fixes #805) #809's branch drags in unrelated [Bug]: Enter key doesn't create new lines in Additional Info textareas #763/fix(builder): allow Enter key to create new lines in Additional Info textareas (#763) #788 frontend changes (empty-line filtering in templates/forms). Excluded — those belong in their own PR.

Changes

#805 fix (app):

• Allow-list: education[i].description (scalar), additional.languages, additional.certificationsTraining, additional.awards. Education degree/institution/years stay blocked (leaf-name check + scoped regex).
• Diff prompt: add casing-preservation rule (Bump notebook from 6.4.1 to 6.4.10 #12); keep verified-skill-target rules + add_skill; document new targetable paths.
• calculate_resume_diff: surface education-description, language, award changes in the preview modal; add language/award to ResumeFieldDiff.field_type.
• Frontend: render Language/Award change sections; i18n keys in all 5 locales (parity verified).
• Tests: new allowed paths apply; blocked education fields still rejected; scalar-only education description.

e2e-monitor tuning (maintainer-requested):

• Add documented, tunable JD_KEYWORD_TOLERANCE = 0.20 to the judge rubric: incorporating JD keywords/skills the master lacked is treated as legitimate ATS tailoring (not fabrication) up to ~20% of content. Hard structural guards unchanged (no_fabricated_employers, personal_info_unchanged); invented employers/titles/dates and wholesale profession swaps still penalized.
• ⚠️ The committed monitor baseline may warrant a refresh (judge scores can rise; regressions only fire on drops) — left to a deliberate maintainer commit.

Verification

• uv run pytest → 365 passed, 1 deselected (gated eval).
• Locale JSON parses + key parity across en/es/ja/pt-BR/zh verified.
• Frontend lint/build NOT run locally (nvm constraint) — relies on reviewer/CI.

🤖 Generated with Claude Code

---

Summary by cubic

Expands the tailor’s diff scope to education descriptions, languages, certifications, and awards while preserving original capitalization. Updates the e2e monitor judge to allow ~20% JD keyword incorporation without truthfulness penalties.

• New Features

  • Allow targets: education[i].description (replace only), additional.languages, additional.certificationsTraining, additional.awards; degree/institution/years remain blocked.
  • Diff preview shows Language and Award changes; i18n keys added in en/es/ja/pt-BR/zh.
  • e2e monitor: adds JD_KEYWORD_TOLERANCE = 0.20 in judge.py and documents policy in AGENT_PLAYBOOK.md (scores may rise; baseline may need refresh).

• Bug Fixes

  • Avoid validation errors by adding language and award to ResumeFieldDiff.field_type (backend + frontend).
  • Enforce scalar-only education.description; reject education[i].description[j] and keep other education fields blocked.
  • Deduplicate education diffs by ignoring description in entry-level compare; editing only the description yields one change.
  • Diff logic surfaces education description, language, and award changes while ignoring list reorders.

^{Written for commit fba7dae. Summary will update on new commits.}

[Copilot]

Pull request overview

This PR addresses Tailor issue #805 by expanding the diff allow-list beyond summary and a small subset of bullets, while also adding an explicit “preserve original capitalization” rule to prevent unwanted lowercasing. It also updates the e2e-monitor judge rubric to tolerate a configurable amount of JD-keyword injection as legitimate ATS tailoring.

Changes:

• Expanded backend diff allow-list to include education[i].description (scalar only) plus additional.languages, additional.certificationsTraining, and additional.awards, while keeping key education identity fields blocked.
• Updated diff prompt rules/targetable paths and extended diff preview generation + UI types/rendering to support new field_types (language, award) with i18n parity across locales.
• Tuned e2e-monitor judge rubric via a documented JD_KEYWORD_TOLERANCE parameter and updated the monitor playbook accordingly.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
apps/frontend/messages/en.json	Adds i18n strings for new Language/Award diff sections in the tailor diff modal.
apps/frontend/messages/es.json	Same as above for Spanish locale.
apps/frontend/messages/ja.json	Same as above for Japanese locale.
apps/frontend/messages/pt-BR.json	Same as above for Portuguese (Brazil) locale.
apps/frontend/messages/zh.json	Same as above for Chinese locale.
apps/frontend/components/tailor/diff-preview-modal.tsx	Renders new grouped sections for language and award diffs in the preview modal.
apps/frontend/components/common/resume_previewer_context.tsx	Extends ResumeFieldDiff.field_type union to include language and award.
apps/backend/tests/unit/test_apply_diffs.py	Adds unit tests covering newly allowed paths and ensures blocked education fields + scalar-only education description.
apps/backend/e2e_monitor/judge.py	Introduces configurable JD keyword tolerance and updates rubric wording accordingly.
apps/backend/e2e_monitor/AGENT_PLAYBOOK.md	Documents the JD-keyword tolerance policy for monitor runs and ties it to judge.py.
apps/backend/app/services/improver.py	Expands allowed diff paths, refines education blocking logic, and surfaces education/language/award diffs in calculate_resume_diff.
apps/backend/app/schemas/models.py	Extends backend ResumeFieldDiff.field_type Literal to include language and award.
apps/backend/app/prompts/templates.py	Updates diff prompt rules/paths and adds casing-preservation rule.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kilo-code-bot]

WARNING: Education descriptions are compared by array index. If an education entry is added or removed at a different position (e.g., a new entry prepended at index 0), descriptions will be compared against the wrong entries — original[1] vs improved[0] — producing false "modified" diffs. The workExperience path avoids this by using SequenceMatcher in _append_list_changes for its description list comparison. Consider using a similar alignment approach or matching entries by a stable key (e.g., institution + degree) before comparing descriptions.

[kilo-code-bot]

Code Review Summary

Status: 1 WARNING, 1 SUGGESTION (1 FIXED) | Recommendation: Address WARNING before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	1

Incremental Changes (since previous review)

Two new commits added:

• improver.py line 1402: Added {"description"}, ignore_keys to education _append_entry_changes call — fixes the previous SUGGESTION about duplicate education description diffs.
• test_resume_diff.py: Added 4 regression tests for education dedup, language add/remove, language order ignored, and award add. Tests are clean and well-structured.

No NEW issues found in the incremental diff.

Issue Details (click to expand)

WARNING (Carried Forward)

File	Line	Issue
apps/backend/app/services/improver.py	1302	Education descriptions compared by array index — if entries are added/removed at different positions, descriptions mismatch against wrong entries (workExperience uses SequenceMatcher for alignment)

SUGGESTION (Carried Forward)

File	Line	Issue
apps/backend/app/schemas/models.py	475–484	ResumeDiffSummary model doesn't track languages/awards — summary stats will omit these new change types entirely

RESOLVED

File	Line	Issue
apps/backend/app/services/improver.py	1402	Education _append_entry_changes missing ignore_keys={"description"} — FIXED: Now passes {"description"} correctly, matching the workExperience pattern

Other Observations (not in diff)

Issues found in unchanged code that cannot receive inline comments:

File	Line	Issue
apps/backend/app/schemas/models.py	475–484	ResumeDiffSummary has fields for skills, descriptions, and certifications but no equivalent for languages or awards added/removed. The calculate_resume_diff function now generates diffs with field_type="language" and field_type="award", but the summary never counts them. Frontend stat cards only display the tracked fields. Consider adding languages_added, languages_removed, awards_added, awards_removed to ResumeDiffSummary and updating the summary builder at lines 1414–1426.

Files Reviewed (11 files total, 2 incremental)

• apps/backend/app/prompts/templates.py — No issues
• apps/backend/app/schemas/models.py — 1 SUGGESTION (ResumeDiffSummary gap)
• apps/backend/app/services/improver.py — 1 WARNING (carried forward), 1 SUGGESTION (resolved)
• apps/backend/e2e_monitor/AGENT_PLAYBOOK.md — No issues
• apps/backend/e2e_monitor/judge.py — No issues
• apps/backend/tests/unit/test_apply_diffs.py — No issues
• apps/backend/tests/unit/test_resume_diff.py — No issues (4 new tests added incrementally)
• apps/frontend/components/common/resume_previewer_context.tsx — No issues
• apps/frontend/components/tailor/diff-preview-modal.tsx — No issues
• apps/frontend/messages/{en,es,ja,pt-BR,zh}.json — No issues (i18n parity verified)

---

_{Reviewed by qwen3.6-plus · 607,962 tokens}

[srbhr]

Thanks @kilo-code-bot / @copilot-pull-request-reviewer — went through all three findings:

✅ SUGGESTION (improver.py ~1395) — duplicate education diff: FIXED (fba7dae). Confirmed real: with a dedicated education-description diff (step 4b), the entry-level _append_entry_changes for education needed ignore_keys={"description"} like workExperience, otherwise a description-only edit produced a second spurious education[i] modified entry (with identical degree/institution/years labels). Added a regression test (test_education_description_change_is_not_duplicated) + language/award diff tests.

↩️ WARNING (improver.py ~1302) — index-based education pairing: declined (not reachable). Education entries cannot be added, removed, or reordered through the tailor pipeline — the allow-list permits only education[i].description, and degree/institution/years stay blocked — so original/improved education lists are always the same length and order. The entry pairing here is the same index-based approach the existing workExperience entry loop uses; SequenceMatcher in _append_list_changes aligns bullets within an entry (a list), which doesn't apply to education's scalar description. Matching by a stable key for a list that can't reorder would add complexity without changing behavior.

🕒 SUGGESTION (models.py — ResumeDiffSummary lacks languages/awards counts): deferred (non-blocking, cosmetic). The summary is a coarse stats roll-up; the preview modal renders language/award changes from detailedChanges, not from the summary, so the feature is complete without it. Adding languages_added/removed etc. would also mean new frontend stat cards — happy to do it as a follow-up if you want the stat row to include them.

Full suite: 369 passed, 1 deselected.

[cubic-dev-ai]

2 issues found across 13 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/backend/app/services/improver.py">

<violation number="1" location="apps/backend/app/services/improver.py:88">
P2: These new list targets need path-specific action gating; as written, malformed diffs can append or replace values instead of only reordering them.</violation>

<violation number="2" location="apps/backend/app/services/improver.py:1323">
P2: Education description edits are now emitted twice: once in the new `education[i].description` diff and again by entry-level education diffing. Exclude `description` from education entry comparisons to avoid duplicate change records.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Education description edits are now emitted twice: once in the new education[i].description diff and again by entry-level education diffing. Exclude description from education entry comparisons to avoid duplicate change records.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At apps/backend/app/services/improver.py, line 1323:

<comment>Education description edits are now emitted twice: once in the new `education[i].description` diff and again by entry-level education diffing. Exclude `description` from education entry comparisons to avoid duplicate change records.</comment>

<file context>
@@ -1286,6 +1296,91 @@ def calculate_resume_diff(
+            change_type = "added"
+        else:
+            change_type = "modified"
+        changes.append(ResumeFieldDiff(
+            field_path=f"education[{idx}].description",
+            field_type="education",
</file context>

[cubic-dev-ai]

P2: These new list targets need path-specific action gating; as written, malformed diffs can append or replace values instead of only reordering them.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At apps/backend/app/services/improver.py, line 88:

<comment>These new list targets need path-specific action gating; as written, malformed diffs can append or replace values instead of only reordering them.</comment>

<file context>
@@ -81,7 +81,13 @@ def _check_for_truncation(data: dict[str, Any]) -> None:
+    # so only the scalar path is allowed — not a [j]-indexed bullet form.
+    re.compile(r"^education\[\d+\]\.description$"),
     re.compile(r"^additional\.technicalSkills$"),
+    re.compile(r"^additional\.languages$"),
+    re.compile(r"^additional\.certificationsTraining$"),
+    re.compile(r"^additional\.awards$"),
</file context>