diff --git a/docs/INDEX.md b/docs/INDEX.md index f0f4fcf..877adcd 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -7,7 +7,6 @@ ``` docs/ ├── INDEX.md # 本文件（文档总索引） -├── SECURITY_AUDIT_20260801.md # 全局安全审计总览 ├── security/ # 安全审计与加固相关 ├── reviews/ # 代码审查日志 └── reports/ # 其他分析报告 @@ -29,7 +28,6 @@ docs/ | 文档 | 说明 | |------|------| -| [SECURITY_AUDIT_20260801.md](security/SECURITY_AUDIT_20260801.md) | **全局总览** — 全模块安全审计汇总（问题等级定义、各模块问题汇总、修复跟踪） | | [DEPLOY_SECURITY_AUDIT_20260801.md](security/DEPLOY_SECURITY_AUDIT_20260801.md) | 部署配置安全审计（docker-compose、systemd、防火墙、TLS 等） | | [DEPS_SECURITY_AUDIT_20260801.md](security/DEPS_SECURITY_AUDIT_20260801.md) | 依赖安全审计（第三方库版本、已知漏洞、许可证） | | [SECURITY_HARDENING_PLAN_20260803.md](security/SECURITY_HARDENING_PLAN_20260803.md) | 安全加固计划（内网通信安全加固开发计划） | @@ -49,8 +47,8 @@ docs/ | smtp | [smtp_SECURITY_AUDIT_20260801.md](security/smtp_SECURITY_AUDIT_20260801.md) | Python 邮件服务（验证码发送/校验） | | captcha | [captcha_SECURITY_AUDIT_20260801.md](security/captcha_SECURITY_AUDIT_20260801.md) | Python 验证码服务 | | confs | [confs_SECURITY_AUDIT_20260801.md](security/confs_SECURITY_AUDIT_20260801.md) | Python 统一配置服务 | +| libhttp | [libhttp_SECURITY_AUDIT_20260803.md](security/libhttp_SECURITY_AUDIT_20260803.md) | C++ HTTP/WebSocket 库（请求/响应/WS/静态文件） | | aibox | [aibox_SECURITY_AUDIT_20260801.md](security/aibox_SECURITY_AUDIT_20260801.md) | Python AI 智能体系统（WebSocket） | -| aibox (详版) | [aibox_SECURITY_AUDIT_full_20260801.md](security/aibox_SECURITY_AUDIT_full_20260801.md) | aibox 详细安全审计（含具体漏洞位置与修复建议） | | BotOps | [BotOps_SECURITY_AUDIT_20260801.md](security/BotOps_SECURITY_AUDIT_20260801.md) | BotOps 运维控制面板 | --- diff --git a/docs/security/libhttp_SECURITY_AUDIT_20260803.md b/docs/security/libhttp_SECURITY_AUDIT_20260803.md index 999caed..6868107 100644 --- a/docs/security/libhttp_SECURITY_AUDIT_20260803.md +++ b/docs/security/libhttp_SECURITY_AUDIT_20260803.md @@ -11,9 +11,9 @@ | 优先级 | 总数 | 已修复 | 部分修复 | 未修复 | |--------|------|--------|----------|--------| | P0 | 0 | 0 | 0 | 0 | -| P1 | 6 | 2 | 0 | 4 | +| P1 | 6 | 3 | 0 | 3 | | P2 | 4 | 0 | 0 | 4 | -| **合计** | **10** | **4** | **0** | **6** | +| **合计** | **10** | **5** | **0** | **5** | --- @@ -22,7 +22,7 @@ | # | 问题 | 状态 | 说明 | |---|------|------|------| | P1-1 | 请求体大小限制不可配置 + 超限无 413 | ✅ 已修复 | `Session::doRead`（`HttpServer_Request.cpp` L167-186）改用 `http::request_parser` + `body_limit()`（默认 10MB，与 WAF 对齐）；`HttpServer` 新增 `setMaxBodySize()`/`getMaxBodySize()`（`HttpServer.hpp` L118-120）。超限时 `handleRead` 返回 **413 Payload Too Large** 并关闭连接（L256-265），而非静默断连 | -| P1-2 | HttpClient 响应无大小限制 | ❌ 未修复 | `HttpClient::write_callback`（`HttpClient.cpp` L329-334）把下游响应 `append` 进 `full_response` 无上限，超大响应可导致 OOM（`request_stream_sync` L63-81） | +| P1-2 | HttpClient 响应无大小限制 | ✅ 已修复 | `HttpClient.cpp` 新增 `WriteCtx`（流式回调 + 累计字节 + 上限），`write_callback` 超限返回 0 触发 `CURLE_WRITE_ERROR`，`request_stream_sync` 抛异常；`request_stream_sync` 改为**真流式**（callback 非空时每块即回调，不二次累积），`HttpClient` 新增 `set_max_response_size()`/`get_max_response_size()`（默认 10MB，对齐 WAF，0 表示不限制） | | P1-3 | WebSocket 消息无大小限制 | ❌ 未修复 | `Session::onWsRead`（`HttpServer_WsSession.cpp` L88-111）把整条 WS 消息读入内存，依赖 Beast 默认 `read_message_max`（16MB）不可配置；超大消息可被恶意客户端推送导致 OOM | | P1-4 | 服务器无读写超时（Slowloris） | ❌ 未修复 | 入站 `stream_`（`beast::tcp_stream`）**从未设置 `expires_after`**（仅代理后端设了超时）。攻击者建立连接后缓慢发送 HTTP 头，可无限占用连接与线程，Slowloris/连接耗尽（对应全局审计 P2-4） | | P1-5 | 静态文件目录路径遍历 | ✅ 已修复 | 新增 `isSafeStaticPath()`（`HttpServer_Request.cpp` L139-156），静态文件服务前校验 target_path（拒绝 `..`、空字节、非 `/` 开头、URL 编码 `%2e`/`%2f`/`%5c`），逻辑与 gateway `is_safe_target()` 对齐（`HttpServer.cpp` L94） | @@ -68,7 +68,7 @@ ### 立即（24小时） - [x] P1-1 请求体大小限制：`doRead` 改用 `http::request_parser` + `body_limit()`，`HttpServer` 增加 `setMaxBodySize()`（对齐 WAF 10MB），超限返回 413 而非断开 → ✅ 已实现（默认 10MB，`setMaxBodySize()` 可覆盖） -- [ ] P1-2 HttpClient 响应上限：`write_callback` 累计超过上限即失败，返回错误 +- [x] P1-2 HttpClient 响应上限：`write_callback` 累计超过上限即失败，返回错误 → ✅ 已实现（`WriteCtx` 流式回调 + 上限，默认 10MB，`set_max_response_size()` 可覆盖） - [ ] P1-4 服务器读超时：入站 `stream_` 设置 `expires_after`（首字节/头+体超时），防 Slowloris ### 短期（72小时） diff --git a/libhttp/README.md b/libhttp/README.md index f96a371..086e4cd 100644 --- a/libhttp/README.md +++ b/libhttp/README.md @@ -16,6 +16,8 @@ C++20 network library for the UniServe microservice framework, providing unified ### HttpClient - **HTTP Client**: Synchronous HTTP requests via libcurl (GET/POST/PUT/DELETE/PATCH/HEAD/OPTIONS) +- **Streaming Response**: `request_stream_sync` streams response body chunk-by-chunk to the callback (no full-body buffering) +- **Response Size Limit**: `set_max_response_size()` caps downstream response body (default 10MB, aligned with WAF; 0 = unlimited) to prevent OOM on oversized responses - **WebSocket Client**: Synchronous WebSocket client via Boost.Beast - **Dual Protocol**: Same client instance handles both HTTP and WebSocket - **ws:// and wss://**: Supports both plain and TLS-encrypted WebSocket connections @@ -221,6 +223,10 @@ void put_stream_sync(const std::string& url, const std::string& data, ResponseCa void delete_stream_sync(const std::string& url, ResponseCallback cb); void request_stream_sync(HttpMethod method, const std::string& url, const std::string& data, ResponseCallback cb); + +// Response size limit (防 OOM). Default 10MB (aligned with WAF); 0 = unlimited. +void set_max_response_size(size_t bytes); +size_t get_max_response_size() const; ``` **WebSocket Methods** @@ -277,6 +283,7 @@ using WebSocketClientBase = HttpClient; ## Version History +- **v3.4.0**: HttpClient 响应大小限制（防 OOM）。`request_stream_sync` 改为真流式（按块回调，不二次缓冲）；新增 `set_max_response_size()`/`get_max_response_size()`（默认 10MB，对齐 WAF，0 表示不限制），超限时 `write_callback` 返回 0 中止传输并抛异常。 - **v3.3.0**: Split HttpServer into 6 modular translation units by functional area (core/lifecycle, listener, routes, request dispatch, WS management, WS session/proxy) for dramatically faster incremental builds. Added internal detail header `inc/us/detail/HttpServerImpl.hpp` for shared definitions across TU boundaries. - **v3.2.0**: Split into two shared libraries — `libus_http.so` (server only, no curl) and `libus_httpclient.so` (HttpClient, depends on curl). HttpClient implementation moved to `src/HttpClient.cpp`; `HttpClient.hpp` is declaration-only. - **v3.1.0**: Added WebSocket reverse proxy (`ws_proxy()`) with bidirectional async bridging, ws:// and wss:// backend support, handshake-time authorization callback. diff --git a/libhttp/inc/us/HttpClient.hpp b/libhttp/inc/us/HttpClient.hpp index 6d1df70..d3da319 100644 --- a/libhttp/inc/us/HttpClient.hpp +++ b/libhttp/inc/us/HttpClient.hpp @@ -59,6 +59,12 @@ public: void put_stream_sync(const std::string& url, const std::string& data, std::function callback); void delete_stream_sync(const std::string& url, std::function callback); + // ===== 响应体大小限制（防 OOM）===== + // 设置下游响应体大小上限（字节）。0 表示不限制（默认 10MB，与 WAF 对齐）。 + // 超限时 write_callback 返回 0 中止传输，request_stream_sync 抛异常。 + void set_max_response_size(size_t bytes); + size_t get_max_response_size() const; + // ===== WebSocket Methods (Synchronous, Boost.Beast-based) ===== bool ws_connect(const std::string& url, int timeout_ms = 5000); bool ws_connect(const std::string& host, const std::string& port, @@ -77,6 +83,7 @@ private: struct curl_slist* headers_ = nullptr; std::map response_headers_; long response_code_ = 0; + size_t max_response_size_ = 10 * 1024 * 1024; // 默认 10MB，对齐 WAF；0 表示不限制 struct WsState { asio::io_context ioc; diff --git a/libhttp/src/HttpClient.cpp b/libhttp/src/HttpClient.cpp index 3b31389..514a2a3 100644 --- a/libhttp/src/HttpClient.cpp +++ b/libhttp/src/HttpClient.cpp @@ -4,6 +4,19 @@ namespace pmc::net { +namespace { + +// write_callback 的用户态上下文：流式回调 + 响应大小上限 +struct WriteCtx { + std::function* callback = nullptr; // 流式回调（非空时每块回调） + std::string* full_response = nullptr; // 退化累积（callback 为空时） + size_t received = 0; // 已接收字节数 + size_t max = 0; // 响应上限（0 表示不限制） + bool overflow = false; // 是否超限 +}; + +} // namespace + HttpClient::HttpClient() { curl_ = curl_easy_init(); if (!curl_) { @@ -21,6 +34,14 @@ HttpClient::~HttpClient() { } } +void HttpClient::set_max_response_size(size_t bytes) { + max_response_size_ = bytes; +} + +size_t HttpClient::get_max_response_size() const { + return max_response_size_; +} + void HttpClient::set_header(const std::string& header) { headers_ = curl_slist_append(headers_, header.c_str()); } @@ -61,8 +82,12 @@ void HttpClient::request_stream_sync( curl_easy_setopt(curl_, CURLOPT_HEADERDATA, this); std::string full_response; + WriteCtx write_ctx; + write_ctx.callback = &callback; + write_ctx.full_response = &full_response; + write_ctx.max = max_response_size_; curl_easy_setopt(curl_, CURLOPT_WRITEFUNCTION, write_callback); - curl_easy_setopt(curl_, CURLOPT_WRITEDATA, &full_response); + curl_easy_setopt(curl_, CURLOPT_WRITEDATA, &write_ctx); curl_easy_setopt(curl_, CURLOPT_TIMEOUT, 30L); curl_easy_setopt(curl_, CURLOPT_CONNECTTIMEOUT, 10L); @@ -70,11 +95,16 @@ void HttpClient::request_stream_sync( CURLcode res = curl_easy_perform(curl_); if (res != CURLE_OK) { + if (res == CURLE_WRITE_ERROR && write_ctx.overflow) { + throw std::runtime_error("HTTP response exceeded max size (" + + std::to_string(max_response_size_) + " bytes)"); + } throw std::runtime_error(std::string("CURL error: ") + curl_easy_strerror(res)); } curl_easy_getinfo(curl_, CURLINFO_RESPONSE_CODE, &response_code_); + // 退化路径：调用方未传 callback 时，整块累积后一次性回调 if (!full_response.empty()) { callback(full_response); } @@ -328,8 +358,20 @@ size_t HttpClient::header_callback(char* buffer, size_t size, size_t nitems, voi size_t HttpClient::write_callback(char* ptr, size_t size, size_t nmemb, void* userdata) { size_t total_size = size * nmemb; - std::string* response = static_cast(userdata); - response->append(ptr, total_size); + auto* ctx = static_cast(userdata); + // 响应上限：超限返回 0，让 curl 以 CURLE_WRITE_ERROR 中止传输 + if (ctx->max > 0 && ctx->received + total_size > ctx->max) { + ctx->overflow = true; + return 0; + } + ctx->received += total_size; + // 流式：callback 非空时每块即回调（不累积，避免二次内存） + if (ctx->callback && *ctx->callback) { + (*ctx->callback)(std::string(ptr, total_size)); + } else if (ctx->full_response) { + // 退化：callback 为空时整块累积，供一次性回调 + ctx->full_response->append(ptr, total_size); + } return total_size; }