Support multiple reviewer/area chair roles per submission#3009
Support multiple reviewer/area chair roles per submission#3009
Conversation
There was a problem hiding this comment.
Pull request overview
Adds first-class support for venues that have multiple per-submission reviewer and/or area-chair roles (e.g., Expert_Reviewers + Technical_Reviewers), ensuring per-role paper groups, assignment deployment, review invitations, and notifications target the correct role-specific groups while preserving single-role behavior.
Changes:
- Introduces
submission_reviewer_roles/submission_area_chair_rolesand updates venue/group/invitation/stage logic to generate per-role per-paper groups and role-specific review invitation wiring. - Updates matching + assignment deployment to resolve the per-paper committee group name from the deployed Assignment invitation (
submission_committee_name) instead of parsing group ids. - Refactors recruitment processes to derive Invited/Declined/Recruitment invitation ids from
committee_id(removing per-role domain content keys), and adds an end-to-end test covering a two-role workflow.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_two_submission_committee_roles.py | New end-to-end test covering two reviewer roles + two AC roles under per_role layout. |
| tests/test_tasks.py | Updates assertions to reflect derived recruitment group ids. |
| tests/test_reviewers_only.py | Removes assertions for domain content keys that are no longer stored. |
| tests/test_matching_v2.py | Sets submission_*_roles explicitly for matching tests. |
| tests/test_icml_conference_with_templates.py | Wires ReviewStage with submission_reviewer_roles for template workflow regression coverage. |
| tests/test_change_venue_email.py | Updates assertions to reflect derived recruitment group ids. |
| tests/test_acs_and_reviewers.py | Removes assertions for domain content keys that are no longer stored. |
| openreview/workflows/workflows.py | Adds request-form fields for multi-role reviewer/AC configuration and group layout selection. |
| openreview/workflows/workflow_process/conference_review_workflow_deployment.py | Sets default ReviewStage to target the primary submission reviewer role. |
| openreview/workflows/workflow_process/committee_invited_group_template_process.py | Stops persisting invited-message invitation ids on the domain. |
| openreview/workflows/workflow_process/committee_group_template_process.py | Stops persisting per-role committee id/invited/declined/recruitment ids on the domain. |
| openreview/workflows/process/deploy_assignments_process.py | Forces materialization of per-role paper groups from proposed assignment edges before status updates. |
| openreview/workflows/process/committee_recruitment_response_process.py | Derives declined + invited-message ids directly from committee_id. |
| openreview/workflows/process/committee_recruitment_request_reminder_process.py | Derives invited/declined/recruitment/message ids directly from committee_id. |
| openreview/workflows/process/committee_recruitment_request_process.py | Derives invited group + recruitment/message ids directly from committee_id. |
| openreview/workflows/process/committee_recruitment_request_edit_reminder_process.py | Derives invited/declined/recruitment/message ids directly from committee_id. |
| openreview/venue/venue.py | Adds submission role lists, name overrides for committee id helpers, and per-role creation of submission-group + assignment/matching setup. |
| openreview/venue/process/review_process.py | Tracks reviewer roles per invitation; updates Submitted subgroup creation + reviewer notification targeting. |
| openreview/venue/matching.py | Uses submission_committee_name from deployed assignment invitation to deploy/undeploy into correct per-paper role groups. |
| openreview/venue/invitation.py | Stores reviewer_roles on review invitations; role-specific per-paper invitees/readers; persists submission_committee_name on Assignment invitations. |
| openreview/venue/helpers.py | Loads submission_*_roles from the domain group content. |
| openreview/venue/group.py | Threads role overrides into per-paper group invitations; persists submission role lists; creates umbrella groups for multi-role venues. |
| openreview/stages/venue_stages.py | Adds submission_reviewer_roles to ReviewStage; iterates roles for readers/signatures; allows identity-reader overrides per role. |
| openreview/conference/helpers.py | Keeps legacy Conference path single-role by explicitly setting submission_reviewer_roles. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| readers.append(self.venue.get_senior_area_chairs_id(number)) | ||
| readers.append(self.venue.get_area_chairs_id(number)) | ||
| readers.append(self.venue.get_area_chairs_id(number, name=name)) | ||
| if openreview.stages.IdentityReaders.REVIEWERS_ASSIGNED in self.venue.area_chair_identity_readers: | ||
| readers.append(self.venue.get_reviewers_id(number)) | ||
| return readers |
There was a problem hiding this comment.
In per_role layouts, self.venue.get_reviewers_id(number) can point to a per-submission group that is never created (e.g. .../Submission1/Reviewers when submissions use .../Submission1/Expert_Reviewers + .../Submission1/Technical_Reviewers). This means area-chair paper-group readers may reference a non-existent group and won't actually grant access to the assigned reviewers. Consider deriving the correct per-submission reviewer group(s) from name (e.g., map AC role -> corresponding submission_reviewer_roles entry) or include all venue.submission_reviewer_roles for that paper when REVIEWERS_ASSIGNED is enabled.
Summary
Lets a venue define more than one reviewer role (and more than one area chair role), with a choice between a shared per-submission group or separate per-role per-submission groups. Single-role venues behave identically to before.
Motivation
Some venues need to run distinct reviewer committees for different kinds of submissions or different evaluation criteria — for example
Expert_ReviewersvsTechnical_Reviewers, or a main AC track alongside a technical AC track. The workflow already accepted alternate names inreviewer_roles/area_chair_rolesat the top-level group, but the per-submission plumbing collapsed everything back to a single primary role: oneSubmission1/Reviewersgroup, one Official_Review invitation, one assignment deployment, oneSubmittedsubgroup. That made it impossible to keep the committees separate once submissions came in — assignments, review forms, and "submitted" tracking all landed on the wrong group, and notifications fanned out to the umbrella instead of the reviewer's own role.This branch treats the multi-role case as a first-class configuration. Two new Venue Request fields —
reviewer_group_layoutandarea_chair_group_layout— let the PC pickshared(existing behavior) orper_role(each role gets its own per-submission group). The role name is then threaded through group creation, invitation setup, matching, assignment deployment, and the review process function, so per-role groups, per-role review forms, and per-role notifications all work end-to-end.Key changes
submission_reviewer_rolesandsubmission_area_chair_rolesdrive how per-submission groups and review invitations are fanned out.get_reviewers_id/get_area_chairs_id/get_anon_reviewer_idtake an optionalname=so callers can target a specific role.submission_committee_name; matching deployment reads it instead of inferring the target group name from the committee id.Submittedsubgroup and reviewer-email recipients to that role.reviewer_roles/reviewer_group_layout/area_chair_roles/area_chair_group_layoutfields on theConference_Review_Workflowrequest form.Test plan
pytest tests/test_two_submission_committee_roles.py(new end-to-end with two reviewer roles and two AC roles underper_role).pytest tests/test_reviewers_only.py(single-role regression).pytest tests/test_acs_and_reviewers.py(reviewer + AC regression).pytest tests/test_matching_v2.py tests/test_icml_conference_with_templates.py tests/test_tasks.py tests/test_change_venue_email.py.