Skip to content

Adds metric for disconnects during upload #1117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aagbsn wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Adds metric for disconnects during upload #1117

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1a66789
Adds metric for disconnects during upload
aagbsn Mar 12, 2026
78368ea
Merge remote-tracking branch 'origin/master' into fix_1116_uncaught_exc
aagbsn Mar 17, 2026
0f3cde8
Merge remote-tracking branch 'origin/master' into fix_1116_uncaught_exc
aagbsn Mar 18, 2026
d123d8c
use error() util to raise HTTPException
aagbsn Mar 24, 2026
515b5f6
Merge remote-tracking branch 'origin/master' into fix_1116_uncaught_exc
aagbsn Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ooniapi/services/ooniprobe/src/ooniprobe/metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,7 @@ class Metrics:
)

TEST_LIST_URLS_COUNT = Gauge("test_list_urls_count", "Size of reported test list")

CLIENT_DISCONNECT = Counter(
"client_upload_disconnect", "Client disconnected while awaiting report"
)
11 changes: 10 additions & 1 deletion ooniapi/services/ooniprobe/src/ooniprobe/routers/reports.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,16 @@ async def receive_measurement(
Metrics.MSMNT_DISCARD_CC_ZZ.inc()
return empty_measurement

data = await request.body()
try:
data = await request.body()
except ClientDisconnect:
log.info(f"Client disconnected mid-upload")
Metrics.CLIENT_DISCONNECT.inc()
return empty_measurement
Copy link
Copy Markdown
Member

@hellais hellais Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should verify how clients behave when we return empty_measurement instead of raising an exception, since this might affect changes in the client behavior.

I would actually suggest we preserve the original behavior by re-rasing the exception

Copy link
Copy Markdown
Contributor Author

@aagbsn aagbsn Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we re-raise the exception it pollutes the logs with a Traceback; but if the client has disconnected - how would it receive anything?

Copy link
Copy Markdown
Member

@hellais hellais Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think for the ClientDisconnect you are right. In the case of the catch-all exception, we ought to though return something that's an error to the client, either a 5xx error (by just re-raising the exception) or a specific 5xx/4xx status code.

except Exception as e:
log.error(f"Uncaught exception {e}")
return empty_measurement
Copy link
Copy Markdown
Contributor Author

@aagbsn aagbsn Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should return something that indicates that this was not a successful submission; verify what repsonse should be sent. IE, should re-raise as a wrapped HTTPException?


if content_encoding == "zstd":
try:
compressed_len = len(data)
Expand Down
Loading