Skip to content

fix(core): avoid holding pool registry locks during database I/O - #7783

Merged
t8y2 merged 3 commits into
t8y2:mainfrom
CN-Scars:fix/pool-registry-lock-boundary
Sep 2, 2026
Merged

fix(core): avoid holding pool registry locks during database I/O#7783
t8y2 merged 3 commits into
t8y2:mainfrom
CN-Scars:fix/pool-registry-lock-boundary

Conversation

@CN-Scars

@CN-Scars CN-Scars commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

变更说明

修复长时间数据库操作持有全局连接池注册表锁,导致其他无关连接无法打开或重连的问题。以 MongoDB 4.2 Legacy Agent 执行 CreateIndex 为例,索引创建阻塞时不再连带阻塞其他 MongoDB、SQL 或 Redis 数据源。

  • 将连接池注册表封装为同步检查、原子修改和持有自有句柄的访问接口,禁止业务操作将注册表锁带入异步数据库 I/O
  • 为所有连接池类型提供可克隆的自有句柄,并使用 Arc 共享 Redis 连接状态,保留原有串行执行与集群扫描状态
  • 迁移 MongoDB、Redis、查询、元数据、导入导出、数据传输和 Agent 等路径,确保数据库请求均在注册表锁释放后执行
  • 保留连接替换、断开、健康检查、空闲回收、手动事务及 Agent 共享运行时的身份校验,避免旧操作误删新连接池
  • 新增 Legacy MongoDB 阻塞建索引、连接池快照、Redis 句柄生命周期与真实 Redis 阻塞命令回归测试

变更类型

  • 新功能
  • Bug 修复
  • 性能优化
  • 代码重构
  • 文档更新
  • CI / 构建

涉及前端

  • 本 PR 涉及前端改动,已附截图/录屏(见下方)

本 PR 不涉及前端改动。

验证

  • make check 通过
  • make cargo-check-fast 通过
  • 相关测试通过

聚焦验证已通过:

  • cargo check -p dbx-core --lib
  • cargo check -p dbx-web
  • cargo test -p dbx-core --lib pool_handle(2 项)
  • cargo test -p dbx-core --lib stale_redis_generation_cannot_remove_replacement
  • cargo test -p dbx-web --no-run
  • Legacy MongoDB 阻塞 CreateIndex 注册表写入回归测试
  • 真实 Redis 阻塞命令与连接池注册表并发回归测试

手动 E2E 已完成:

  • MongoDB 4.2.24 Legacy Agent:通过 failpoint 阻塞 CreateIndex,期间其他 MongoDB 与 MySQL 连接均可正常展开
  • MongoDB 5.0.18 Native:阻塞 CreateIndex 期间,MySQL、PostgreSQL 与其他 MongoDB 连接均可正常展开
  • Redis 7.4:执行阻塞式 BLPOP 期间,MongoDB、MySQL 与 PostgreSQL 连接均可正常展开
  • Redis 连接生命周期:BLPOP 阻塞期间断开并重新连接,新的 DB0 控制台执行 PING 正常返回 PONG

关联 Issue

Close #7720

@github-actions github-actions Bot added area/core Shared DBX core runtime area/desktop Desktop application or Tauri shell area/web Web backend or web API bug Something isn't working db/mongodb Database: MongoDB (Legacy) db/redis Database: Redis db/sqlite Database: SQLite labels Sep 1, 2026

@t8y2 t8y2 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes.

remove_stale_connection_pool probes outside the registry lock, but it retains identity only for MySQL and Redis. For the other pool kinds, a failed probe reaches connection.rs:3704-3708 and removes by pool_key alone. If a reconnect or replacement inserts a fresh pool under the same key while the probe is in flight, the stale-probe cleanup can delete the new healthy pool.

Please retain an identity handle for every pool kind that can be detached by this path, and remove the entry only if it is still the exact pool that was probed. Add a concurrent replacement regression test for at least one non-MySQL/non-Redis pool.

@CN-Scars
CN-Scars force-pushed the fix/pool-registry-lock-boundary branch from 462380c to 75c89e9 Compare September 2, 2026 04:56
@github-actions github-actions Bot added db/multiple Touches more than three database integrations and removed db/redis Database: Redis db/sqlite Database: SQLite db/mongodb Database: MongoDB (Legacy) labels Sep 2, 2026
@CN-Scars

CN-Scars commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

已按建议为所有连接池类型统一增加发布代际标识,并补充了非 MySQL/Redis 场景的并发替换测试。相关修改已推送。

@t8y2 t8y2 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

维护者补丁已推送:fd11fd7c8(原 head 75c89e9,先并入 main@7e24f4007 —— merge 提交 ff8d373)。冲突仅一处:crates/dbx-core/src/agent_kv.rs(main 的 72dcd83 etcd 重写新增了 agent_key 查找,与本分支改写的注册表读取相邻)。解法:完整保留 main 的 agent_key 块 + 采用本 PR 的 pool_handle 形态,删除被 pool_handle 取代的旧 connections.read().await.get(...) 行 —— etcd2 感知的错误提示与「I/O 期间不持注册表锁」两个行为同时保留;文件中已无残留 guard 读取。range-diff 确认 PR 补丁逐 hunk 完整保留,无被 base 吸收的改动。

附带一个清晰化修复:close_pool_with_timeoutself.connections.read().await.clone() 实际经 Deref 解析为内部 HashMap 的 clone(注册表类型本身不是 Clone),改为显式 registry.pools.clone(),锁的持有范围不变。

本地验证:rustfmt 干净、git diff --check 干净;完整 dbx-core 测试由本 head 的 Rust CI 作为合并门槛(本地长测试因磁盘环境受限中断,非代码问题)。

@t8y2
t8y2 merged commit d6fc1fe into t8y2:main Sep 2, 2026
14 checks passed
@t8y2

t8y2 commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Thanks for the contribution! Merged in d6fc1fe, will be released in the next version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Shared DBX core runtime area/desktop Desktop application or Tauri shell area/web Web backend or web API bug Something isn't working db/multiple Touches more than three database integrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] mongo会话执行创建索引任务,所有数据源db都点不开,转圈

2 participants