fix(libhttp): HttpClient 响应大小限制防 OOM

- request_stream_sync 改为真流式,按块回调不二次缓冲
- 新增 set_max_response_size/get_max_response_size(默认 10MB,对齐 WAF)
- write_callback 超限返回 0 中止传输并抛异常
- 更新审计文档 P1-2 状态、README v3.4.0、INDEX 移除不存在的文档
This commit is contained in:
qing committed 2026-08-04 15:12:06 +08:00
1 parent dc52ec13ad
commit 85e0d0740b
5 files changed
+64 -10

No files matched your search

+7
View File
@@ -16,6 +16,8 @@ C++20 network library for the UniServe microservice framework, providing unified
### HttpClient
- **HTTP Client**: Synchronous HTTP requests via libcurl (GET/POST/PUT/DELETE/PATCH/HEAD/OPTIONS)
- **Streaming Response**: `request_stream_sync` streams response body chunk-by-chunk to the callback (no full-body buffering)
- **Response Size Limit**: `set_max_response_size()` caps downstream response body (default 10MB, aligned with WAF; 0 = unlimited) to prevent OOM on oversized responses
- **WebSocket Client**: Synchronous WebSocket client via Boost.Beast
- **Dual Protocol**: Same client instance handles both HTTP and WebSocket
- **ws:// and wss://**: Supports both plain and TLS-encrypted WebSocket connections
@@ -221,6 +223,10 @@ void put_stream_sync(const std::string& url, const std::string& data, ResponseCa
void delete_stream_sync(const std::string& url, ResponseCallback cb);
void request_stream_sync(HttpMethod method, const std::string& url,
const std::string& data, ResponseCallback cb);
// Response size limit (防 OOM). Default 10MB (aligned with WAF); 0 = unlimited.
void set_max_response_size(size_t bytes);
size_t get_max_response_size() const;
```
**WebSocket Methods**
@@ -277,6 +283,7 @@ using WebSocketClientBase = HttpClient;
## Version History
- **v3.4.0**: HttpClient 响应大小限制(防 OOM)。`request_stream_sync` 改为真流式(按块回调,不二次缓冲);新增 `set_max_response_size()`/`get_max_response_size()`(默认 10MB,对齐 WAF,0 表示不限制),超限时 `write_callback` 返回 0 中止传输并抛异常。
- **v3.3.0**: Split HttpServer into 6 modular translation units by functional area (core/lifecycle, listener, routes, request dispatch, WS management, WS session/proxy) for dramatically faster incremental builds. Added internal detail header `inc/us/detail/HttpServerImpl.hpp` for shared definitions across TU boundaries.
- **v3.2.0**: Split into two shared libraries — `libus_http.so` (server only, no curl) and `libus_httpclient.so` (HttpClient, depends on curl). HttpClient implementation moved to `src/HttpClient.cpp`; `HttpClient.hpp` is declaration-only.
- **v3.1.0**: Added WebSocket reverse proxy (`ws_proxy()`) with bidirectional async bridging, ws:// and wss:// backend support, handshake-time authorization callback.
+7
View File
@@ -59,6 +59,12 @@ public:
void put_stream_sync(const std::string& url, const std::string& data, std::function<void(const std::string&)> callback);
void delete_stream_sync(const std::string& url, std::function<void(const std::string&)> callback);
// ===== 响应体大小限制(防 OOM)=====
// 设置下游响应体大小上限(字节)。0 表示不限制(默认 10MB,与 WAF 对齐)。
// 超限时 write_callback 返回 0 中止传输,request_stream_sync 抛异常。
void set_max_response_size(size_t bytes);
size_t get_max_response_size() const;
// ===== WebSocket Methods (Synchronous, Boost.Beast-based) =====
bool ws_connect(const std::string& url, int timeout_ms = 5000);
bool ws_connect(const std::string& host, const std::string& port,
@@ -77,6 +83,7 @@ private:
struct curl_slist* headers_ = nullptr;
std::map<std::string, std::string> response_headers_;
long response_code_ = 0;
size_t max_response_size_ = 10 * 1024 * 1024; // 默认 10MB,对齐 WAF;0 表示不限制
struct WsState {
asio::io_context ioc;
+45 -3
View File
@@ -4,6 +4,19 @@
namespace pmc::net {
namespace {
// write_callback 的用户态上下文:流式回调 + 响应大小上限
struct WriteCtx {
std::function<void(const std::string&)>* callback = nullptr; // 流式回调(非空时每块回调)
std::string* full_response = nullptr; // 退化累积(callback 为空时)
size_t received = 0; // 已接收字节数
size_t max = 0; // 响应上限(0 表示不限制)
bool overflow = false; // 是否超限
};
} // namespace
HttpClient::HttpClient() {
curl_ = curl_easy_init();
if (!curl_) {
@@ -21,6 +34,14 @@ HttpClient::~HttpClient() {
}
}
void HttpClient::set_max_response_size(size_t bytes) {
max_response_size_ = bytes;
}
size_t HttpClient::get_max_response_size() const {
return max_response_size_;
}
void HttpClient::set_header(const std::string& header) {
headers_ = curl_slist_append(headers_, header.c_str());
}
@@ -61,8 +82,12 @@ void HttpClient::request_stream_sync(
curl_easy_setopt(curl_, CURLOPT_HEADERDATA, this);
std::string full_response;
WriteCtx write_ctx;
write_ctx.callback = &callback;
write_ctx.full_response = &full_response;
write_ctx.max = max_response_size_;
curl_easy_setopt(curl_, CURLOPT_WRITEFUNCTION, write_callback);
curl_easy_setopt(curl_, CURLOPT_WRITEDATA, &full_response);
curl_easy_setopt(curl_, CURLOPT_WRITEDATA, &write_ctx);
curl_easy_setopt(curl_, CURLOPT_TIMEOUT, 30L);
curl_easy_setopt(curl_, CURLOPT_CONNECTTIMEOUT, 10L);
@@ -70,11 +95,16 @@ void HttpClient::request_stream_sync(
CURLcode res = curl_easy_perform(curl_);
if (res != CURLE_OK) {
if (res == CURLE_WRITE_ERROR && write_ctx.overflow) {
throw std::runtime_error("HTTP response exceeded max size ("
+ std::to_string(max_response_size_) + " bytes)");
}
throw std::runtime_error(std::string("CURL error: ") + curl_easy_strerror(res));
}
curl_easy_getinfo(curl_, CURLINFO_RESPONSE_CODE, &response_code_);
// 退化路径:调用方未传 callback 时,整块累积后一次性回调
if (!full_response.empty()) {
callback(full_response);
}
@@ -328,8 +358,20 @@ size_t HttpClient::header_callback(char* buffer, size_t size, size_t nitems, voi
size_t HttpClient::write_callback(char* ptr, size_t size, size_t nmemb, void* userdata) {
size_t total_size = size * nmemb;
std::string* response = static_cast<std::string*>(userdata);
response->append(ptr, total_size);
auto* ctx = static_cast<WriteCtx*>(userdata);
// 响应上限:超限返回 0,让 curl 以 CURLE_WRITE_ERROR 中止传输
if (ctx->max > 0 && ctx->received + total_size > ctx->max) {
ctx->overflow = true;
return 0;
}
ctx->received += total_size;
// 流式:callback 非空时每块即回调(不累积,避免二次内存)
if (ctx->callback && *ctx->callback) {
(*ctx->callback)(std::string(ptr, total_size));
} else if (ctx->full_response) {
// 退化:callback 为空时整块累积,供一次性回调
ctx->full_response->append(ptr, total_size);
}
return total_size;
}