ARR: Add additional submission query params by haroldrubio · Pull Request #2996 · openreview/openreview-py

haroldrubio · 2026-04-02T14:18:23Z

The following query params are being added in this PR for the ARR PC/SAC console:

Count of emergency declarations
Count of delay notifications
Number of completed reviews plus the number of delay notifications
Assigned reviewers minus emergency declarations

One request: "it would be better to be able to sort on the number of submissions with fewer than three reviews" - I think this should be supported by querying numOfficialReviewDone and sorting by reviews submitted, but you cannot jointly do this in the UI at the moment

Copilot

Pull request overview

Adds additional computed submission query parameters to ARR consoles (Program Chairs and Senior Area Chairs) to enable filtering/sorting by emergency declarations, delay notifications, and derived review/assignment counts, with Selenium coverage to exercise the new query fields in the UI.

Changes:

Added computed submission query params for emergency declarations and delay notifications (reviewer + area chair) in PC and SAC console webfields.
Added derived query params for “assigned reviewers minus emergency declarations” and “completed reviews + delay notifications” style metrics.
Extended Selenium integration tests to validate filtering via the new query params in the Submission Status table.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
`tests/test_arr_venue_v2.py`	Adds UI-level query assertions for the newly introduced submission query params.
`openreview/arr/webfield/programChairsWebfield.js`	Introduces new `propertiesAllowed` computed fields to support the additional submission query params in the PC console.
`openreview/arr/webfield/seniorAreaChairsWebfield.js`	Introduces analogous `propertiesAllowed` computed fields for the SAC console submission query params.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-24T13:14:39Z

+      assignedReviewersAfterEmergencyDeclarationsCount: `
+      const replies = row.note?.details?.replies ?? [];
+      const reviewers = row.reviewers ?? [];
+      const signatureToReviewerIndex = new Map();
+      const reviewersWithEmergencyDeclaration = new Set();
+
+      reviewers.forEach((reviewer, reviewerIndex) => {
+        if (reviewer?.anonymizedGroup) {
+          signatureToReviewerIndex.set(reviewer.anonymizedGroup, reviewerIndex);
+        }
+        if (reviewer?.preferredId) {
+          signatureToReviewerIndex.set(reviewer.preferredId, reviewerIndex);
+        }
+      });
+
+      replies.forEach(reply => {
+        const replySignature = reply?.signatures?.[0];
+        const isEmergencyDeclaration = (reply?.invitations ?? []).some(invitation => invitation.includes('Emergency_Declaration'));
+
+        if (!isEmergencyDeclaration || !signatureToReviewerIndex.has(replySignature)) {
+          return;
+        }
+
+        reviewersWithEmergencyDeclaration.add(signatureToReviewerIndex.get(replySignature));
+      });
+
+      return Math.max(0, reviewers.length - reviewersWithEmergencyDeclaration.size);
+      `,


The SAC console uses assignedReviewersAfterEmergencyDeclarationsCount while the PC console uses assignedReviewersMinusEmergencyDeclarationsCount for the same concept (assigned reviewers minus emergency declarations). This inconsistency makes the query language harder to use across consoles and diverges from the PR description terminology; consider standardizing the parameter name across both webfields.

Copilot · 2026-04-24T13:14:39Z

+      completedReviewsOrDelayNotificationsCount: `
+      const replies = row.note?.details?.replies ?? [];
+      const reviewers = row.reviewers ?? [];
+      const officialReviews = row.officialReviews ?? [];
+      const signatureToReviewerIndex = new Map();
+      const reviewersWithDelayNotification = new Set();
+      const reviewersWithOfficialReview = new Set(
+        officialReviews
+          .map(review => review?.anonymousId)
+          .filter(Boolean)
+      );
+
+      reviewers.forEach((reviewer, reviewerIndex) => {
+        if (reviewer?.anonymizedGroup) {
+          signatureToReviewerIndex.set(reviewer.anonymizedGroup, reviewerIndex);
+        }
+        if (reviewer?.preferredId) {
+          signatureToReviewerIndex.set(reviewer.preferredId, reviewerIndex);
+        }
+      });
+
+      replies.forEach(reply => {
+        const replySignature = reply?.signatures?.[0];
+        const isDelayNotification = (reply?.invitations ?? []).some(invitation => invitation.includes('Delay_Notification'));
+
+        if (!isDelayNotification || !signatureToReviewerIndex.has(replySignature)) {
+          return;
+        }
+
+        reviewersWithDelayNotification.add(signatureToReviewerIndex.get(replySignature));
+      });
+
+      return reviewers.filter((reviewer, reviewerIndex) => {
+        return reviewersWithOfficialReview.has(reviewer?.anonymousId) || reviewersWithDelayNotification.has(reviewerIndex);
+      }).length;


Similar to the PC console’s completedReviewsPlusDelayNotificationsCount, this function counts unique reviewers who submitted an official review OR a delay notification (union), not a literal sum of the two counts. If the intended behavior is a true sum (per PR description wording), the logic/name should be adjusted; otherwise consider renaming for clarity and consistency across consoles.

Copilot · 2026-04-24T13:14:39Z

+        assert_submission_query(
+            f'{venue_id}/Program_Chairs',
+            pc_console_client,
+            '+number=2 AND areaChairEmergencyDeclarationCount=0',
+            [2]
+        )
+        assert_submission_query(
+            f'{venue_id}/Program_Chairs',
+            pc_console_client,
+            '+number=2 AND reviewerDelayNotificationCount=1',
+            [2]
+        )
+        assert_submission_query(
+            f'{venue_id}/Program_Chairs',
+            pc_console_client,
+            '+number=2 AND areaChairDelayNotificationCount=0',
+            [2]


The new area-chair query params are only asserted with =0 (e.g., areaChairEmergencyDeclarationCount=0, areaChairDelayNotificationCount=0), so these tests wouldn’t catch an implementation that always returns 0 (e.g., due to a wrong data path). Add at least one positive test case where an area chair submits an Emergency Declaration / Delay Notification and assert the count is 1.

Copilot · 2026-04-24T13:14:40Z

+      completedReviewsPlusDelayNotificationsCount: `
+      const replies = row.note?.details?.replies ?? [];
+      const reviewers = row.reviewers ?? [];
+      const officialReviews = row.officialReviews ?? [];
+      const signatureToReviewerIndex = new Map();
+      const reviewersWithDelayNotification = new Set();
+      const reviewersWithOfficialReview = new Set(
+        officialReviews
+          .map(review => review?.anonymousId)
+          .filter(Boolean)
+      );
+
+      reviewers.forEach((reviewer, reviewerIndex) => {
+        if (reviewer?.anonymizedGroup) {
+          signatureToReviewerIndex.set(reviewer.anonymizedGroup, reviewerIndex);
+        }
+        if (reviewer?.preferredId) {
+          signatureToReviewerIndex.set(reviewer.preferredId, reviewerIndex);
+        }
+      });
+
+      replies.forEach(reply => {
+        const replySignature = reply?.signatures?.[0];
+        const isDelayNotification = (reply?.invitations ?? []).some(invitation => invitation.includes('Delay_Notification'));
+
+        if (!isDelayNotification || !signatureToReviewerIndex.has(replySignature)) {
+          return;
+        }
+
+        reviewersWithDelayNotification.add(signatureToReviewerIndex.get(replySignature));
+      });
+
+      return reviewers.filter((reviewer, reviewerIndex) => {
+        return reviewersWithOfficialReview.has(reviewer?.anonymousId) || reviewersWithDelayNotification.has(reviewerIndex);
+      }).length;


The completedReviewsPlusDelayNotificationsCount implementation counts unique assigned reviewers who either submitted an official review OR submitted a delay notification (it does not add the two counts, and it won’t double-count a reviewer who did both). This behavior doesn’t match the PR description wording (“plus”), and the Plus name is misleading given the OR/union semantics. Consider either renaming to reflect OR/union semantics (and aligning with the SAC console param name), or adjusting the computation if a true sum is required.

haroldrubio added 8 commits March 19, 2026 08:15

Add new query params

f0742b8

Change webfield code

1c20b2b

Update names

d3a16bd

Merge branch 'master' into fix/arr-query-params

4367d92

Fix brackets

a671399

Modify tests

0243c84

Merge branch 'master' into fix/arr-query-params

2713a84

Change all to AC filters

11eb038

haroldrubio self-assigned this Apr 2, 2026

melisabok added the arr Issues related to the ARR workflow label Apr 8, 2026

haroldrubio added 3 commits April 24, 2026 15:06

Remove data assertions

12b6d48

Merge branch 'master' into fix/arr-query-params

8e483b8

Remove new lines

3e5aaa7

haroldrubio marked this pull request as ready for review April 24, 2026 13:09

Copilot AI review requested due to automatic review settings April 24, 2026 13:09

Copilot started reviewing on behalf of haroldrubio April 24, 2026 13:09 View session

Copilot AI reviewed Apr 24, 2026

View reviewed changes

Address Copilot

227f674

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARR: Add additional submission query params#2996

ARR: Add additional submission query params#2996
haroldrubio wants to merge 12 commits intomasterfrom
fix/arr-query-params

haroldrubio commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

haroldrubio commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants