Commit Graph
10 Commits
Author SHA1 Message Date
qing d883dc620e refactor(pymsg): 新增AsyncRpcQueue通用异步队列,log_client基于此重构
- pymsg新增AsyncRpcQueue:提供内存队列+后台线程+攒批处理的通用异步基础设施,
  支持两种模式:简单RPC模式(client+method+serialize)和自定义handler模式
- confs/dnfs/static的log_client.py重构为基于AsyncRpcQueue的薄封装,
  保持API完全向后兼容(push/info/error/warning/debug/critical + 全局logs单例)
- 消除了三份log_client副本中重复的队列/线程/攒批逻辑(约150行/份),
  代码量净减少约50行
- LogClient的行为与旧版一致:逐条INSERT到db、fail-fast网络错误、队列满丢最旧
2026-08-01 12:59:33 +08:00
qing 1b9c727e80 fix(log_client): harden background flush thread against exceptions
- Wrap _flush_loop main body in outer try/except to prevent any
  unexpected exception (network errors, serialization errors, etc.)
  from killing the background thread (which would cause unbounded
  queue growth and eventual OOM)
- Wrap _send() call in inner try/except as final safety net
- Wrap RpcClient() construction in try/except (connection failures
  during client creation were previously unhandled)
- Add _safe_json() helper: unserializable extra fields fall back to
  repr()/str() instead of raising TypeError that could kill thread
- Generate timestamp at push() (enqueue time) instead of _send() time,
  so timestamps reflect actual event time even when queue is backed up
- Replace unreliable time.sleep(0.6) in flush() with _flush_done Event
  for proper synchronization between caller and background thread
- Add _QUEUE_MAXSIZE=10000 with drop-oldest policy to prevent unbounded
  memory growth even in pathological cases
- Use put_nowait with Full handling to ensure push() never blocks
  business threads
- Sync fix to all 4 copies: confs/static/smtp/dnfs
2026-08-01 03:23:19 +08:00
qing 3f97dd92ef update 2026-07-30 21:57:04 +08:00
qing 167ef235e6 fixbug 100% cpu 2026-07-27 14:47:12 +08:00
qing 08021015b1 更新msg客户端 2026-07-27 09:21:06 +08:00
qing 4fe0689cf6 修复db客户端无法推日志的问题 2026-07-26 21:26:17 +08:00
qing 1a39e87c8c 更新pymsg 2026-07-26 21:04:42 +08:00
qing 8ec42df779 初步开发了新的pymsg,并且应用在了dnfs上面 2026-07-26 17:57:27 +08:00
qing 060b7fc7ad 修复CPU占用100%的问题 2026-07-26 01:22:23 +08:00
qing 013e130a08 logs in db develop 2026-07-25 21:01:46 +08:00