fix: sync core-data delete - #5378
Conversation
Signed-off-by: remo-lab <remopanda7@gmail.com>
|
|
@judehung @weichou1229 @cloudxxx8 , please let me know your thoughts on this! |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5378 +/- ##
==========================================
+ Coverage 21.21% 21.33% +0.11%
==========================================
Files 285 285
Lines 20979 20966 -13
==========================================
+ Hits 4451 4473 +22
+ Misses 16174 16134 -40
- Partials 354 359 +5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@cloudxxx8 , shall i update anything on this pr? |
|
It's as design for the performance consideration, not a bug. |
|
Hi @cloudxxx8, thanks for the feedback! I understand the mark_deleted pattern is intentional for performance. The concern is: if updateDeviceInfosDeletableByDeviceName succeeds but the background delete transaction fails, the data becomes invisible (filtered by mark_deleted = false) yet physically remains in the DB — with no rollback or retry. Would a smaller fix be acceptable instead — e.g., rolling back mark_deleted = false on the affected rows if the background delete fails? That preserves the performance optimization while closing the silent-failure window. Happy to update the PR accordingly! |
|
@remo-lab The logic in The reason we use async call for event deletion is that is really time consuming when the data volume is large. However, your concern is valid. How about we add a periodical check to remove the data with |



Summary
This PR fixes a core-data consistency bug where deleting telemetry by device name could make events and readings disappear from queries before the database delete had actually succeeded.
In event.go and deviceinfo.go, I removed the preemptive mark_deleted update on device_info, changed the PostgreSQL delete path to execute readings, events, and device_info cleanup in a
single transaction, and kept cache eviction only after commit. The async boundary stays in the application layer, so the API behaviors unchanged, but database failures now surface correctly instead of silently hiding still-present telemetry.
I also added a regression test in event_test.go for the transactional delete path.
Previously, DeleteEventsByDeviceName() marked matching device_info rows as deleted before the real delete work ran. Since event and reading queries filter on mark_deleted = false, any later failure
could leave data physically present in PostgreSQL but invisible to operators, while the delete request had already returned success.This change removes that silent-failure mode.
There are no API or schema changes.
Validation:
Passed:
infrastructure/postgres