Commit Graph
49 Commits
Author SHA1 Message Date
qing aa3741b861 feat(liblogc): 统一集中式日志客户端库 + 全量迁移
新增:
- liblogc/: C++ 统一日志库 (libus_logc.so),支持 set_endpoint/set_service_name/set_level
- pymsg/pymsg/logc.py: Python 统一日志模块,基于 AsyncRpcQueue

C++ 迁移 (删除 6 份本地 LogC.hpp 副本):
- ca, gateway: <us/LogC.hpp> + set_service_name
- captcha2, confs2, smtp3: using qing::LogC + set_service_name
- core: ConfsLogC → qing::LogC (ConfsTask.cpp)
- 所有 Makefile 添加 -lus_logc

Python 迁移 (6 个 log_client.py → 4 行薄包装):
- smtp, redirect80, confs, captcha, dnfs, static
- 全部委托到 pymsg.logc.LogClient

Go (waf): 保持现状,自成体系
2026-08-10 04:51:43 +00:00
qing 8eb14cad4a gateway: CLI 标准化(--host/--port/--config/--log-level/--instance + LogC 级别过滤)
- 新增 --host/--port 覆盖 [Server] address/port
- 新增 --config 覆盖 confs 拉取的配置名(默认 gateway.ini)
- 新增 --log-level debug|info|warn|error(LogC 加级别过滤)
- 新增 --instance 实例名(注册中心用)
- LogC.hpp: 新增 set_level() + push() 级别过滤(atomic 阈值,大写兼容)
- 配置优先级: CLI > ENV > confs > 默认值
- services.yaml: gateway 声明 --instance 和 --log-level
2026-08-10 11:55:54 +08:00
qingandWorkBuddy dbc79c52f3 docs(security): 合并 gateway 审计文档并新建 ca 审计索引
- 三份 gateway 审计(20260801/0807/0809)合并为单一 docs/security/gateway_SECURITY_AUDIT.md
- 已完成修复已合入 gateway/src/main.cpp 注释,认证相关项标注迁 ca
- 新建 docs/security/ca_SECURITY_AUDIT.md,同步迁入 P2-3/P1-7/A2 等开放项
- 更新 docs/INDEX.md 审计表、ca/README.md 引用、auth_service_plan/libhttp 文档链接

Co-Authored-By: WorkBuddy <workbuddy@tencent.com>
2026-08-10 00:56:21 +08:00
qing 407ce3c366 feat(ca): 认证独立化 CA 模块 v1.0.0 + gateway v0.9.0 纯转发瘦身
拓扑:waf:443 → ca:8443(认证+前置转发)→ gateway:8445(仅 loopback 纯转发),
WAF upstream 不变,core/dnfs2/前端零改动。

ca v1.0.0(新模块):
- 认证资产逐字搬迁自 gateway v0.8.0(9 条认证路由、Token 双轨吊销、
  PBKDF2、限流、WS 代理鉴权、wrapperHttp、优雅关闭)
- handleForward 前置转发层:固定上游 gateway:8445;无 Target-Url 公开直转,
  有则本地验签后注入 X-Auth-User/X-Auth-Admin/Authorization(strip-then-set)
- is_special_header 新增剥离 x-auth-user/x-auth-admin;上游不可达统一 502
- confs/configs/ca.ini 新增([Upstream] gateway + UPSTREAM_GATEWAY env)

gateway v0.9.0(2470→约 560 行):
- 删除全部认证代码,退居 127.0.0.1:8445
- 启动 loopback 强制自检(非 loopback 拒绝启动)
- X-Auth-User 存在性不变量:显式 Target-Url 无该头一律 401
- gateway.ini 精简至 [Server]/[Proxy]/[Downstream db]

验收:黑盒 34 断言对 8443 原样全绿(32 passed/2 skipped,与基线一致);
探针实测伪造 X-Auth-User 经 CA 被剥离(echo 实证)、直连 8445 无头 401、
白名单外 403、gateway 宕机 CA 502。core/init.txt 加入 ca/_run.sh。
2026-08-09 12:15:35 +00:00
qing 546394b681 feat: captcha C++20 重写 + 安全审计修复 (P0/P1/P2/P3, 10/11 已修复)
captcha2 — C++20 验证码服务(FreeType + stb_image_write):
- 完整实现: 图片生成、CaptchaStore、RateLimiter、pymsg RPC 路由
- 协议完全兼容 Python 原版,gateway 零改动

审计修复 (captcha2_SECURITY_AUDIT_20260809.md):
- P0-1: 存储答案小写化 (对齐 Python 版语义)
- P1-1: gateway 按 IP 限流 10/min + captcha2 全局限流 600/min 兜底
- P2-1: generate_text mt19937 → std::random_device
- P2-2: ::tolower UB → unsigned char lambda
- P3-1: parse_int 辅助函数, stoi 异常可诊断
- P3-2: 显式 #include <iomanip>
- P3-3: 补齐 CAPTCHA_LEN/TTL/RATE_LIMIT 环境变量
- P3-4: LogC 日志超时 5.0→1.0s
- P3-5: 限流/满容抛异常, gateway 映射 429/503
- P3-6: 清理死代码

集成:
- core/init.txt: captcha → captcha2
- confs configs/captcha.ini: 新增 max_store/rate_limit_per_min
- docs: 开发方案、安全审计报告、交叉引用更新
2026-08-09 04:26:09 +00:00
qing e815f0ec40 fix(gateway,libhttp): 修复进程无法退出bug,完成P0安全问题修复
安全修复(P0):
- 修复4处错误消息泄露内部步骤问题,替换为用户友好文案
- 修正拼写错误 "Registe" -> "Registration"
- 修正注册失败HTTP状态码 400 -> 500
- 所有P0严重问题100%修复,安全评级提升至A-

稳定性修复(关键):
- 修复信号处理非async-signal-safe问题:仅设置原子标志,杜绝死锁
- 新增SIGPIPE忽略处理,防止写关闭socket时进程意外终止
- 修复Listener::doAccept()关闭后无限递归导致CPU 100%空转bug
- 新增Listener::close()方法,stop()先关闭acceptor再停止ioc
- 改进stop()流程:acceptor->WS客户端->ioc->worker线程 有序关闭
- 新增双重超时保护:stop() 8秒、日志flush 3秒,超时强制exit保证进程绝不挂死
- 服务器事件循环改为独立线程运行,主线程负责信号检测和关闭协调

文档:
- 新增CHANGELOG.md更新日志
- 更新gateway安全审计文档,添加稳定性修复记录
- 更新docs/INDEX.md文档索引
2026-08-07 23:07:35 +08:00
qing b33c5e6d1d fix(gateway): 修复P0安全问题,错误消息脱敏
- 将4处暴露内部处理步骤的错误消息替换为用户友好文案
- 修正拼写错误 "Registe" -> "Registration"
- 修正注册失败HTTP状态码 400 -> 500
- 更新安全审计文档至2026-08-07版本
- P0严重问题全部修复,安全评级提升至A-

修复内容:
- "Login (Query Step) Failed" -> "Login failed, please try again later"
- "Login (Update Step) Failed" -> "Login failed, please try again later"
- "Registe Failed" -> "Registration failed, please try again later"
- "Reset Failed" -> "Password reset failed, please try again later"
2026-08-07 22:53:26 +08:00
qing 5dc8956b55 fix(gateway): implement dual-track token revocation mechanism (P0-1)
Add thread-safe in-memory token revocation to fix the critical issue
where logged-out or password-reset tokens remain valid until expiry.

- Add jti (uuid v4) to all issued tokens for single-token revocation
- Add per-user version number for bulk revocation on password reset
- Add jti blacklist with TTL-based auto-cleanup to prevent memory leaks
- Introduce unified verify_token() replacing 4 copies of inline logic
- Refactor all handlers (login/logout/refresh/reset_password/proxy/whoami/
  admin_verify/ws_admin) to use the new verification pipeline
- Refresh token now revokes old jti (prevents replay after refresh)
- Bump version to 0.8.0
2026-08-07 02:35:08 +08:00
qing 43c5821d3d fix(gateway): 修复代理 URL 双斜杠导致下游 404
handleProxy 拼接转发 URL 时,target_path 以 '/' 开头,
url = t_url + '/' + target_path 会产生双斜杠(如 //list)。
core 等使用精确路径匹配的下游服务返回 404。

修复:仅当 target_path 不以 '/' 开头时才补斜杠。
2026-08-04 21:20:24 +08:00
qing 34cd295005 feat(core): mgmt 接入 core 进程管理 + 管理员认证
- gateway 新增 POST /admin/verify 验签接口,供内部服务校验管理员 Token
- core 的 /list /exec /kill 接入管理员认证(调用 gateway 验签,fail-closed)
- gateway 代理 opened 白名单加入 core:5200
- mgmt 管理页显示当前用户与角色,进程页接入 core 进程管理接口
2026-08-04 20:00:24 +08:00
qing fb5920d89f feat(gateway): 新增重置密码接口并改造前端 forgot 页对接
- gateway 新增 POST /reset_password:复用注册校验链(限流/密码强度/邮箱格式/CAPTCHA/邮箱验证码),最后一步由创建用户改为 UPDATE Users SET password_hash,用户不存在时统一返回成功防枚举
- gateway 新增 POST /reset/send_code:CAPTCHA 强制 + 邮箱双窗口限流,定制重置密码文案
- 前端 forgot.html 弃用 static 被禁用的 /api/proxy_* 接口,对接 gateway 新接口,并改为中文 register.html 风格
2026-08-03 22:40:43 +08:00
qing b935fefb82 refactor(gateway): remove local ini fallback, configs only via confs
Follow the same convention as the static module: remove local config.ini
fallback and --config/-i command-line argument entirely. All configuration
is now managed exclusively through confs service.

- Delete gateway/example.ini (moved to confs/configs/gateway.ini).
- Remove -i/--input option from boost::program_options; help (-h) and
  version (-v) are the only CLI flags remaining.
- Simplify main.cpp config loading: remove local-file branch, always fetch
  gateway.ini from confs with retry_on_refused=true waiting for confs ready.
- Update _run.sh: no longer passes any arguments; config comes from confs.
- Update README: remove example.ini references, -i flag docs, local file
  mode description, and example.ini from directory structure tree.
- Clarify changelog entry for v0.7.0.
2026-08-01 14:00:26 +08:00
qing 9ccb9cbdca feat(gateway): migrate configuration to confs service (v0.7.0)
Migrate gateway from local -i ini file to the unified confs configuration
service, following the same two-layer pattern as Python services (dnfs/db/static):

- Add confs/configs/gateway.ini with all existing settings plus new
  [Downstream] section for db/captcha/smtp host:port (previously hardcoded).
- Conf.hpp: add fromString() to parse ini from text (confs response),
  env()/env_int() static helpers for environment variables, public default
  constructor, and empty-filename guard in file constructor.
- DbC.hpp: add init(host, port) static method and static _host/_port members,
  replacing hardcoded "127.0.0.1":8005. Defaults preserved for backward compat.
- main.cpp:
  * Rewrite config loading: bootstrap CONFS_HOST/CONFS_PORT from env
    (default 127.0.0.1:8010), fetch gateway.ini via pymsg from confs
    (retry_on_refused=true), parse with ConfIni::fromString().
  * -i flag is now optional; when provided, reads local ini (dev/debug mode).
  * Add downstream host/port globals (_DB_HOST/_DB_PORT, _CAPTCHA_HOST/_CAPTCHA_PORT,
    _SMTP_HOST/_SMTP_PORT, _CONFS_HOST/_CONFS_PORT), populated from [Downstream].
  * Environment variable overrides (highest priority): LISTEN_ADDR/LISTEN_PORT,
    DB_HOST/DB_PORT, CAPTCHA_HOST/CAPTCHA_PORT, SMTP_HOST/SMTP_PORT.
  * Config integrity validation: required fields (Server.address/port,
    Proxy.default, Downstream.db_host/db_port) cause startup failure if missing.
  * Initialize DbC::init() and logs().set_endpoint() after config load.
  * Replace all hardcoded "127.0.0.1":8005/8008/8009 in captcha/smtp/db calls
    with configurable globals.
  * Add downstream service info to startup log output.
  * Fix: -v/--version flag now actually prints version and exits.
  * Bump version to 0.7.0.
- _run.sh: remove -i example.ini default; now starts without args (confs mode),
  passes "$@" through for optional -i override.
- Update README with new config loading docs, Downstream config section,
  updated directory structure, and v0.7.0 changelog.
2026-08-01 13:55:18 +08:00
qing 7d6ba6de93 feat(gateway): add retry_on_refused for pymsg MsgClient (v0.6.1)
- Add MsgConnectionRefusedException to distinguish ECONNREFUSED from other
  connection errors; openConnection() now throws it when connect() returns
  ECONNREFUSED.
- MsgClient::call() adds retry_on_refused/retry_interval parameters (defaults
  false/1.0s), matching Python pymsg.MsgClient.call(retry_on_refused=True).
- Retry loop: only retries on ECONNREFUSED; other exceptions (timeout,
  connection reset, business errors) propagate immediately. Throttled logging:
  first failure prints immediately, subsequent logs every 5 seconds.
- DbC::call_msg() enables retry_on_refused=true (db is a hard dependency;
  gateway blocks waiting for db on startup instead of crashing).
- LogC::_send() explicitly passes retry_on_refused=false (fail-safe; never
  block the background log thread when db is unavailable).
- captcha/smtp auxiliary services keep default retry_on_refused=false
  (fail-fast; blocking in request threads would exhaust the thread pool).
- Update README with pymsg protocol documentation, downstream retry strategy
  table, and v0.6.1 changelog entry.
2026-08-01 13:42:24 +08:00
qing 12ece9020c feat(libhttp,gateway): add admin-only WebSocket reverse proxy
libhttp v3.1:
- add ws_proxy(path, handshake) API with WsProxyHandshakeHandler callback
- implement full-duplex async bidirectional bridging (text+binary frames)
- support ws:// and wss:// backends with SNI, write queues for ordering
- proxy routes take priority over regular ws() routes
- 403 response on handshake rejection, graceful close propagation

gateway v0.6.0:
- add [WsProxy] config section (routes = /path=ws://backend;...)
- handshake callback verifies Token (Authorization/Cookie) is_admin=true
- non-admin / expired / missing token → 403, admin → transparent bridge
- fix LocalTime.hpp C++20 tzdb incompatibility on Apple clang (use strftime/get_time/mktime)
- update example.ini and README docs for both modules
2026-08-01 05:20:14 +08:00
qing 49f0698b9d fix(gateway,smtp): 修复邮件轰炸漏洞(审计 P0-1)
gateway v0.5.3 → v0.5.4:

1. gateway/src/main.cpp
   - /register/send_code 强制 CAPTCHA 校验(经 captcha:8009,Token 一次性消费,fail-closed)
   - 新增邮箱级双窗口限流(3 次/小时 + 10 次/天,CAPTCHA 通过后才计入配额)
   - 新增 _EMAIL_ADDR_RATE_HOUR / _EMAIL_ADDR_RATE_DAY 限流器实例

2. smtp/smtp_v2.py
   - 新增邮箱级长期限流兜底(EMAIL_HOURLY_LIMIT / EMAIL_DAILY_LIMIT,默认 3/10)
   - 新增全局熔断(SMTP_HOURLY_LIMIT,默认 100 封/小时,防 smtp 账号被封)
   - 预占/回滚与 60s 冷却同模式,发送失败不计入配额

3. gateway/docs/SECURITY_AUDIT_20260801.md
   - 新增带日期的安全审计报告
   - P0-1 标记为已修复 v0.5.4

4. gateway/README.md
   - 更新 /register/send_code 接口文档(CAPTCHA 必填 + 5 层防护)
   - 更新版本变更日志

防护层次:IP 限流 → CAPTCHA → 邮箱级限流(gateway)→ 60s 冷却 → 邮箱级限流(smtp)→ 全局熔断
2026-08-01 02:37:15 +08:00
qing 48b45cb44a 修复网关bug 2026-07-31 22:19:56 +08:00
qing 5839172dae 迁移/whoami视图 2026-07-30 22:05:12 +08:00
qing a8a190571c feat(gateway): add /refresh_token API with grace window + refresh limit
- POST /refresh_token: accept current Bearer token (or Cookie fallback),
  verify RSA signature, check grace window (iat + keep_time * multiplier)
  and refresh count limit, then issue a new token.
- Token payload extended with iat, refresh_count, max_refresh fields.
- iat preserves original issue time; created_at updates on each refresh.
- Rate limited: 5 req / 5 min / IP via RateLimiter.
- Config: Token.refresh_grace_multiplier (default 2), Token.max_refresh (default 10).
- Backward compatible: old tokens without new fields still pass proxy checks.
2026-07-30 11:54:31 +08:00
qing 46c3a197be refactor(auth): 管理员判定上移至 gateway,经 Token payload 下发 is_admin
gateway 登录签发 Token 时,账号命中 [Admin] users 白名单即在 payload
写入 is_admin=true(随 RSA 签名保护);dnfs 移除本地 ADMIN_USERS 硬编码,
delete_note/update_note 豁免判断改读 payload 的 is_admin 字段。
旧 Token 无此字段按非管理员处理(安全方向)。
2026-07-30 09:08:33 +08:00
qing c5dd8d198d feat(smtp): 同一邮箱 60 秒发送冷却,gateway 429 透传 + 前端重发倒计时(gateway v0.5.3) 2026-07-29 20:54:56 +08:00
qing f0cdaacd85 feat(gateway): 运行日志接入 db 集中式日志(新增 C++ 版 LogC,pymsg 推送 Logs 表)
- 新增 gateway/inc/LogC.hpp:异步批量日志客户端(内存队列+后台线程,50条/2秒 flush),
  对齐 Python 版 log_client.py,fail-safe 设计(db 不可用静默丢弃)
- main.cpp 全部日志输出迁移:wrapperHttp 双异常分支(>=500 ERROR / 其余 WARNING)、
  启动/关闭、登录成功、注册成功、验证码邮件发送成功、代理白名单拒绝
- Makefile 增加 -pthread(LogC 使用 std::thread)
2026-07-29 08:34:14 +08:00
qing 6a0551ab34 fix(gateway): 限流错误码统一429、登录用户不存在改401并拉平时序、pymsg超时常量化(v0.5.2)
- 三处限流错误码统一为 429(login 400→429 并修 attampts 拼写;register/send_code 403→429)
- 登录用户不存在不再兜底 500:空结果统一走 401,dummy PBKDF2 校验拉平时序防用户枚举
- db 查询失败消息误名修正:Login (Update Step) Failed → (Query Step)
- 超时常量化:DbC::DEFAULT_TIMEOUT=5.0;_TIMEOUT_CAPTCHA/_SMTP_VERIFY=5.0、_SMTP_SEND=20.0
- 文档:gateway/README 数据库节 email→username 重写、建表归 db/create_sqlite3.sql;根 README 版本号与自动建表描述同步
2026-07-29 00:21:13 +08:00
qing b07dbf2da5 feat(gateway): 登录成功后更新 Users.last_login(v0.5.1)
启用 handleLogin 中被注释的 last_login 更新:经 DbC::call_msg 以
"$now" 哨兵(db 日期适配层转为服务端 UTC 时间)写入,失败按惯例
THROW_EX_CUSTOM 抛 500 由 wrapper 统一处理。README 版本节同步
(原仍停留在 0.3.0-security)。
2026-07-28 23:31:20 +08:00
qing df518d1adb feat: 注册强制邮箱验证码(gateway v0.5.0 新增 /register/send_code,pymsg 调 smtp) 2026-07-28 17:29:52 +08:00
qing 6fcb70d235 feat: 登录页接入图形验证码,登录接口强制校验
登录页展示 captcha 图形码(点击刷新),登录请求携带 captcha_token/captcha_answer;gateway 的 /login 启用验证码校验,经 pymsg 调 captcha verify 路由(fail-closed),补齐防爆破短板。
2026-07-28 10:15:01 +08:00
qing 44c1e5ef72 feat: captcha 改造为 pymsg/TCP 服务,gateway 新增图形码代理路由
captcha 参照 dbv2 统一为 pymsg 协议(image/verify/health 路由),与主干 RPC 栈对齐;gateway 新增 GET /captcha/image 做 HTTP→pymsg 转换,解码 base64 PNG 并以原契约(image/png + X-Captcha-Token)返回浏览器。
2026-07-28 09:54:08 +08:00
qing 4ad3fbfca8 fix: 修复公开页面因旧 Cookie 验签失败误返回 401
浏览器会自动给每个请求携带 q4_token,gateway 重启轮换内存 RSA 密钥后旧凭证验签失败,导致无需登录的公开页面也 401。代理层改为仅对需鉴权目标校验凭证,公开目标跳过;前端遇 401 自动跳转登录页自愈。
2026-07-28 09:31:10 +08:00
qing e89578fa82 security: Token 改为 HttpOnly Cookie 下发,消除 XSS 窃取面
登录时网关签发 HttpOnly+Secure+SameSite=Lax 的 q4_token Cookie,前端不再经手 Token;网关代理时支持从 Cookie 验签并向下游注入 Authorization(dnfs 零改动),新增 /logout 清除 Cookie,static 代理路由兼容 Cookie 转 Bearer。
2026-07-28 09:07:12 +08:00
qing 13f3eb8f88 DEV 2026-07-27 21:43:56 +08:00
qing 8c74d93a76 security fixup 2026-07-27 17:28:15 +08:00
qing 6efb792133 C++版本的MsgClient 2026-07-26 21:48:14 +08:00
qing 0e64b3ca26 refix 2026-07-26 14:41:38 +08:00
qing 303d151fce fix without test 2026-07-26 14:33:30 +08:00
qing 96213d58bd 迁移数据库到db服务 2026-07-26 01:10:21 +08:00
qing 46b8f750ae fixbug 2026-07-25 13:07:18 +08:00
qing 3d8cf52503 白名单域名:硬编码 https://qs***.me 、https://www.qs***.me ,加两个本地开发域名(127.0.0.1:8443、localhost:8443)。
动态回填 Origin:仅当请求 Origin 命中白名单时才设置 Access-Control-Allow-Origin,否则不返回该头(浏览器会拒绝跨域请求)。
追加 Vary: Origin:告诉缓存该响应因 Origin 而异,避免缓存误用。
移除 Access-Control-Allow-Credentials: true:当前用 Bearer token(请求头),非 cookie,不需要此头;保留反而增加风险面。
2026-07-25 10:22:58 +08:00
qing 7927d87c0a 在循环内先将 char c 转为 unsigned char uc,再传入 std::isupper / std::islower / std::isdigit 2026-07-25 10:20:21 +08:00
qing 6f44d9497b 修复special_headers过滤,修复XFF,添加is_safe_path 2026-07-25 10:10:57 +08:00
qing bfc361ddfb 之前的修复有遗漏 2026-07-25 10:08:02 +08:00
qing efae81b230 修复源码信息泄漏以及token无限期问题 2026-07-25 10:05:24 +08:00
qing 13be4bd8ca 修复线程安全问题,以及一个别的 2026-07-25 09:58:25 +08:00
qing 202f2804b2 增加waf模块 2026-07-25 01:27:08 +08:00
qing fbccc5d909 网友修改版 2026-07-25 00:21:16 +08:00
qing 419ae82858 修改HTTP LIBRARY,增加OPTIONS支持
修改GATEWAY,修复转发
修改dnfs,暂时不编译(找不到模板)
修改static,能够获取节点
2026-07-22 12:47:13 +08:00
qing 798d5afc4d 加入token时间验证 2026-07-21 23:03:27 +08:00
qing 0399715008 修改后端登陆响应为json,修复登陆页面 2026-07-21 22:27:48 +08:00
qing aaac8fff12 代理地址限制 2026-07-21 11:49:45 +08:00
qing 70d467d052 init 2026-07-20 11:12:29 +08:00