Commit Graph
56 Commits
Author SHA1 Message Date
qing de58c0487a fix(dnfs): validate token via gateway /whoami with RSA signature verification
- Replace unsafe base64-only decode_token() with HTTP POST to gateway /whoami,
  which performs RSA signature verification and expiry checks internally
- Fail-closed on gateway unreachable: return 503 instead of allowing access
- Add [gateway] config section (url/timeout) with GATEWAY_URL env override
- Add missing @_handle_exp decorator to delete_note and update_note views
- Fix Flask endpoint name collision using @functools.wraps on _handle_exp
- Remove unused base64 import
- Update dnfs.ini with gateway config (http://127.0.0.1:8443/whoami, timeout=5)
- Update docker-compose.yml with GATEWAY_URL and host.docker.internal for containers
2026-08-01 13:49:28 +08:00
qing e7a2877422 feat(dnfs): delegate Token verification to gateway /whoami
Replace the insecure local base64 decode (with FIXME comment about
missing signature verification) with proper remote RSA signature
verification through gateway's /whoami endpoint:

- Add [gateway] config section (url, timeout) to dnfs.ini with defaults.
- Support GATEWAY_URL environment variable override for Docker.
- decode_token() now POSTs to gateway /whoami with Authorization: Bearer
  <token>; gateway verifies RSA signature and expiry, returns login/user/
  is_admin. Network/gateway errors fail-closed with 503.
- Add @_handle_exp decorators to delete_note() and update_note() which
  were missing exception handling wrappers (consistent with other routes).
- docker-compose.yml: add GATEWAY_URL, CONFS_HOST, CONFS_PORT env vars
  plus host.docker.internal extra_hosts so dnfs container can reach
  host-side gateway and confs services.
2026-08-01 13:42:59 +08:00
qing b30ce1424b docs(dnfs): update README with security audit findings and architecture corrections
- Fix architecture diagram to reflect pymsg TCP RPC (not HTTP), add confs dependency
- Update security headers to match actual security_middleware.py values
- Add complete P0/P1/P2 security audit findings (16 issues total):
  - P0: Token without RSA signature verification (forgery risk)
  - P0: review_content() fail-closed broken due to exception handling gaps
  - P0: delete_note/update_note missing @_handle_exp decorator
  - P0: Parent nodes with children can be deleted (no DB foreign key + TOCTOU race)
  - P1: No token expiry check, no request body size limit, log_client config mismatch
  - P2: Error message info leakage, overly permissive CORS, legacy commented code
- Correct file paths for frontend (static/public/), dependencies (pymsg, not nuitka)
- Fix outdated code line references
2026-08-01 13:28:41 +08:00
qing d883dc620e refactor(pymsg): 新增AsyncRpcQueue通用异步队列,log_client基于此重构
- pymsg新增AsyncRpcQueue:提供内存队列+后台线程+攒批处理的通用异步基础设施,
  支持两种模式:简单RPC模式(client+method+serialize)和自定义handler模式
- confs/dnfs/static的log_client.py重构为基于AsyncRpcQueue的薄封装,
  保持API完全向后兼容(push/info/error/warning/debug/critical + 全局logs单例)
- 消除了三份log_client副本中重复的队列/线程/攒批逻辑(约150行/份),
  代码量净减少约50行
- LogClient的行为与旧版一致:逐条INSERT到db、fail-fast网络错误、队列满丢最旧
2026-08-01 12:59:33 +08:00
qing 101aed48be feat(pymsg): RpcClient 新增 retry_on_refused 阻塞等待模式,解决微服务启动顺序依赖
pymsg 客户端:
- MsgClient.call() 新增 retry_on_refused/retry_interval 参数
  - retry_on_refused=True 时,ConnectionRefusedError 不抛出,按 retry_interval(默认1s)阻塞重试
  - 重试期间每5秒向 stderr 打印等待提示,避免日志刷屏
  - 仅对 ConnectionRefusedError 重试;TimeoutError/ConnectionError/RpcError 仍立即抛出
  - 默认 retry_on_refused=False,完全向后兼容
- RpcClient 构造函数新增同名关键字参数,call() 支持单次调用覆盖
- 新增 time 模块导入
- 更新 README 文档(新增"启动等待模式"章节和参数表)

各服务(db/dnfs/static):
- _fetch_from_confs() 改用 retry_on_refused=True,移除手动 ConnectionRefusedError 捕获和 sys.exit
- 保留 RpcError(如配置文件不存在)的友好错误处理和 sys.exit(1)
- 启动时不再因 confs 未就绪而退出:pymsg 自动阻塞等待 confs 启动后自动连接成功
- 启动顺序从"刚性要求 confs 先启动"变为"不严格要求启动顺序"
- log_client 保持默认 retry_on_refused=False,后台线程不阻塞(fail-safe 丢弃日志)
- 更新各服务 README 和 confs README 的启动顺序描述

测试验证:
- 基本 call() 正常工作
- retry_on_refused=False 连接拒绝立即抛 ConnectionRefusedError
- retry_on_refused=True 阻塞等待(延迟1.5s启动服务后成功连接,重试日志正常输出)
2026-08-01 12:47:08 +08:00
qing 84adec4722 refactor(dnfs): 删除默认配置逻辑,confs 改为强依赖
- 配置加载从三层(默认值→confs→环境变量)改为两层(confs 强依赖→环境变量覆盖)
- confs 连接失败直接 sys.exit(1),不再静默使用内置默认值
- _dnfs_config 初始值改为空字符串白名单模板,移除内置默认值
- 新增配置完整性校验:confs 返回后检查所有必需 section/key 非空
- 区分错误类型:连接拒绝/RpcError/通用异常分别打印明确错误信息到 stderr
- confs 连接地址改为引导配置(环境变量优先,默认 127.0.0.1:8010),移除 dnfs.ini 中 [confs] 段
- 启动顺序调整为 confs → db → dnfs → ai_review/...(dnfs 同时依赖 confs 和 db)
- 更新 dnfs/README.md 与 confs/README.md 文档
2026-08-01 12:33:51 +08:00
qing cebef9fe70 docs: 各模块安全审计报告 + 补充 .gitignore 忽略规则 2026-08-01 05:21:23 +08:00
qing e007990594 feat(dnfs,static): 新增管理员特殊节点(链接/网页代码/图片/文件)
- 信封格式:content 为 {"_sp":...} JSON,普通 Markdown 节点不受影响
- dnfs:add_note/update_note 按 is_admin 强制特殊节点发布权(非管理员 403),
  字段与大小校验(url≤2048、code≤512KB、base64≤3.5M 字符),AI 审核剔除 base64 负载
- dnfs:修复存量 bug —— 模块级 _sys.path.insert 未定义导致 import 即 NameError
- static:新增 GET /api/favicon 服务端代理(SSRF 防护:内网地址拦截 + 重定向校验 + 512KB 上限)
- 前端:管理员可见节点类型选择器;链接 favicon 卡片 / 网页可审计可运行 / 图片直显 / 文件下载
- theme.css 补充 select 与特殊节点样式;两个 README 同步更新
2026-08-01 04:34:23 +08:00
qing 4e190953c0 feat(confs): integrate dnfs service with centralized config management
- Add dnfs.ini to confs/configs/ with server, database, ai_review, confs sections
- Refactor dnfs.py to use three-layer fail-safe config loading (defaults → confs → env vars)
- Unify timeouts for DB RPC and AI review calls via configuration
- Add _merge_ini() whitelist merging to prevent unexpected config injection
- Environment variables have highest priority for Docker networking (db/ai_review hostnames)
- Document config loading mechanism in both confs/README.md and dnfs/README.md

Follows the same pattern as static service integration; confs failure falls back
gracefully to built-in defaults + environment variables.
2026-08-01 04:13:39 +08:00
qing 1b9c727e80 fix(log_client): harden background flush thread against exceptions
- Wrap _flush_loop main body in outer try/except to prevent any
  unexpected exception (network errors, serialization errors, etc.)
  from killing the background thread (which would cause unbounded
  queue growth and eventual OOM)
- Wrap _send() call in inner try/except as final safety net
- Wrap RpcClient() construction in try/except (connection failures
  during client creation were previously unhandled)
- Add _safe_json() helper: unserializable extra fields fall back to
  repr()/str() instead of raising TypeError that could kill thread
- Generate timestamp at push() (enqueue time) instead of _send() time,
  so timestamps reflect actual event time even when queue is backed up
- Replace unreliable time.sleep(0.6) in flush() with _flush_done Event
  for proper synchronization between caller and background thread
- Add _QUEUE_MAXSIZE=10000 with drop-oldest policy to prevent unbounded
  memory growth even in pathological cases
- Use put_nowait with Full handling to ensure push() never blocks
  business threads
- Sync fix to all 4 copies: confs/static/smtp/dnfs
2026-08-01 03:23:19 +08:00
qing e7f435dc4e refix 2026-07-31 21:07:41 +08:00
qing ada2af47af 增加handle_exp包装器 2026-07-31 19:47:27 +08:00
qing 5839172dae 迁移/whoami视图 2026-07-30 22:05:12 +08:00
qing 3f97dd92ef update 2026-07-30 21:57:04 +08:00
qing 6649b31e17 feat(static): add frontend token auto-refresh for dnfs page
- New token_manager.js: silent refresh timer (80% lifetime) + global
  fetch 401 interceptor with one-shot retry.
- dnfs.py /whoami now returns iat, created_at, keep_time for expiry
  estimation.
- Dnfs.html loads token_manager.js before dnfs.js (zero change to dnfs.js).
- Graceful fallback: stale tokens without new fields, refresh failures,
  and /refresh_token 401 itself all degrade to the existing 401→login flow.
2026-07-30 12:25:11 +08:00
qing 8c1956cdeb feat(dnfs): 左上角用户信息面板(默认折叠),新增 /whoami 身份接口
dnfs 新增 POST /whoami:解 Token payload 返回 user/is_admin,无或畸形
Token 返回 login=false(HttpOnly Cookie 模式下 JS 无法读 Token,身份
必须由后端下发);static 加 /whoami 代理(Cookie 转 Bearer);前端
dnfs.html 左上新增 userpanel,镜像 minimap 样式默认折叠,展开显示
用户名/角色/退出登录(POST /logout 清 Cookie 后跳登录页)。
2026-07-30 09:31:28 +08:00
qing b4e1cf462a feat(dnfs): get_notes 按当前用户注入 can_operate,前端按权显隐编辑/删除按钮
dnfs get_notes 可选解析 Authorization(畸形 Token try 兜底),为每个节点
附带 can_operate 标记(is_admin 或 owner,口径与 delete/update 一致);
dnfs.js 卡片编辑/删除按钮仅 can_operate === true 时渲染,未登录或无权隐藏。
2026-07-30 09:17:28 +08:00
qing 46c3a197be refactor(auth): 管理员判定上移至 gateway,经 Token payload 下发 is_admin
gateway 登录签发 Token 时,账号命中 [Admin] users 白名单即在 payload
写入 is_admin=true(随 RSA 签名保护);dnfs 移除本地 ADMIN_USERS 硬编码,
delete_note/update_note 豁免判断改读 payload 的 is_admin 字段。
旧 Token 无此字段按非管理员处理(安全方向)。
2026-07-30 09:08:33 +08:00
qing e9f9860fe1 fix(dnfs): review_content 兼容布尔/字符串审核结果,修复安全内容误拦
仅明确判定安全(布尔 false 或字符串 "false")才放行,其余一律拦截,
保持 fail-closed;同时消除 invalid 字段缺失时的 KeyError。
2026-07-30 08:38:14 +08:00
qing 751866d9c1 refix 2026-07-29 11:52:46 +08:00
qing ace98caff6 feat: 卡片编辑/删除按钮 + dnfs 管理员豁免
- dnfs.js: 卡片底部新增编辑/删除按钮,复用下方表单支持编辑模式
  (startEdit/cancelEdit/updateNode/deleteChild),stopPropagation 防误触
- theme.css: 新增 card-actions/act-btn 按钮样式,编辑橙色/删除红色软底
- dnfs.py: ADMIN_USERS 统一管理豁免名单(admin, admin@qsont.me),
  update_note 新增管理员豁免,delete_note 同步改用名单
- dnfs/README.md: 同步更新所有权豁免说明
2026-07-29 11:22:30 +08:00
qing 50f209e84a fix: X-Frame-Options/frame-ancestors 改为同源以修复条例弹窗空白 2026-07-28 11:53:35 +08:00
qing a5e581ef24 fix: CSP img-src 放行 blob:,修复验证码图片被拦截
登录页验证码用 URL.createObjectURL 生成的 blob: URL 显示,被 img-src 'self' data: 拦截导致图片无法加载;五处 CSP 定义(waf 边缘 + 四个 Flask 中间件副本)同步放宽。
2026-07-28 10:19:47 +08:00
qing 13f3eb8f88 DEV 2026-07-27 21:43:56 +08:00
qing 167ef235e6 fixbug 100% cpu 2026-07-27 14:47:12 +08:00
qing 08021015b1 更新msg客户端 2026-07-27 09:21:06 +08:00
qing 4fe0689cf6 修复db客户端无法推日志的问题 2026-07-26 21:26:17 +08:00
qing 1a39e87c8c 更新pymsg 2026-07-26 21:04:42 +08:00
qing 8ec42df779 初步开发了新的pymsg,并且应用在了dnfs上面 2026-07-26 17:57:27 +08:00
qing 060b7fc7ad 修复CPU占用100%的问题 2026-07-26 01:22:23 +08:00
qing 013e130a08 logs in db develop 2026-07-25 21:01:46 +08:00
qing b15745f35a 更新文档 2026-07-25 14:34:39 +08:00
qing 17c22a8cd3 管理员也不能更新别人的节点 2026-07-25 14:25:06 +08:00
qing 3e9ff99eaa 只能更新自己的节点 2026-07-25 14:24:36 +08:00
qing efbe2b36f3 节点所有权 2026-07-25 14:20:08 +08:00
qing c028e46d00 fixbug 2026-07-25 11:15:42 +08:00
qing c5fc6f1076 fix up waf and gateway 2026-07-25 01:54:43 +08:00
qing fbccc5d909 网友修改版 2026-07-25 00:21:16 +08:00
qing 290332c70c 更新README.md 2026-07-24 22:17:20 +08:00
shufeng2012 5fe2e2ed65 refix 2026-07-24 16:50:15 +08:00
shufeng2012 f1201c692b prepare fix bug 2026-07-24 16:42:08 +08:00
shufeng2012 cd7293b791 dnfs server calling ai_review 2026-07-24 15:47:25 +08:00
shufeng2012 468d420746 解决冲突(这啥呀*2) 2026-07-24 15:00:34 +08:00
shufeng2012 80d20f5015 解决冲突(这啥呀) 2026-07-24 14:46:26 +08:00
shufeng2012 b431aba35d refix 2026-07-24 12:03:33 +08:00
shufeng2012 a6ff35260b 使用db服务 2026-07-24 10:02:46 +08:00
shufeng2012 7bb0124bbe 有子节点不可删除 2026-07-24 08:54:22 +08:00
shufeng2012 cc08d171c6 refix 2026-07-24 07:05:46 +08:00
qingss0 f0a43f1da2 update: 更新文件 LICENSE
Signed-off-by: qingss0 <qingss0@noreply.gitcode.com>
2026-07-24 06:15:14 +08:00
qing 5edb6ad0a4 dnfs不再托管本地前端 2026-07-23 09:20:04 +08:00