Merge pull request '网友帮我修改、开发waf' (#2) from dev_qing into main

Reviewed-on: #2
This commit was merged in pull request #2.
This commit is contained in:
qing committed 2026-07-25 01:58:58 +08:00
commit a2624e6cf2
37 files changed
+10164 -93

No files matched your search

+5 -3
View File
@@ -121,10 +121,12 @@
根目录提供一键安装脚本,按依赖顺序依次构建并安装各模块(库先行,core 最后):
```bash
# 1. 安装系统依赖(Debian/Ubuntu)
bash _setup.deb.sh
# 1. 安装系统依赖
bash _setup.deb.sh # Debian/Ubuntu
# 或
bash _setup.rhel.sh # RHEL/CentOS/Rocky/Fedora
# 2. 依次安装各模块(libth -> libhttp -> libcrypt -> libsqlite3 -> gateway -> static -> dnfs -> db -> core)
# 2. 依次安装各模块(libth -> libhttp -> libcrypt -> libsqlite3 -> gateway -> waf -> static -> dnfs -> db -> core)
bash _inst.sh
```
+1
View File
@@ -30,6 +30,7 @@ inst_module "libhttp"
inst_module "libcrypt"
inst_module "libsqlite3"
inst_module "gateway"
inst_module "waf" # waf 在 gateway 之前编译,部署时由 core 先拉起 waf 再拉起 gateway
inst_module "static"
inst_module "dnfs"
inst_module "db"
+1 -1
View File
@@ -2,4 +2,4 @@
# Debian
sudo apt update
sudo apt install libsqlite3-dev libboost-all-dev libcurl4-gnutls-dev python3 python3-venv python3-pip g++ patchelf
sudo apt install libsqlite3-dev libboost-all-dev libcurl4-gnutls-dev python3 python3-venv python3-pip g++ patchelf golang-go
+4
View File
@@ -0,0 +1,4 @@
#!/bin/bash
# RHEL / CentOS / Rocky / Fedora
sudo dnf install -y boost-devel libcurl-devel python3 python3-pip gcc-c++ patchelf golang sqlite-devel
-5
View File
@@ -1,5 +0,0 @@
#!/bin/bash
# Debian
sudo apt update
sudo apt install libboost-all-dev libcurl4-gnutls-dev python3 python3-venv python3-pip g++ patchelf
+2
View File
@@ -19,8 +19,10 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
load_dotenv(os.path.join(os.path.dirname(os.path.abspath(__file__)), ".env"))
from agent.review_agent import ReviewAgent
from security_middleware import apply_security_headers
app = Flask(__name__)
apply_security_headers(app)
CORS(app)
review_agent = ReviewAgent()
+42
View File
@@ -0,0 +1,42 @@
"""
Security headers middleware for Flask applications.
Usage:
from security_middleware import apply_security_headers
apply_security_headers(app)
"""
import secrets
def apply_security_headers(app, csp_policy=None):
"""Apply security response headers to a Flask app."""
@app.after_request
def set_security_headers(response):
# Prevent clickjacking
response.headers['X-Frame-Options'] = 'DENY'
# XSS protection (legacy browsers)
response.headers['X-XSS-Protection'] = '1; mode=block'
# Prevent MIME type sniffing
response.headers['X-Content-Type-Options'] = 'nosniff'
# Referrer policy
response.headers['Referrer-Policy'] = 'strict-origin-when-cross-origin'
# Permissions policy (disable camera, mic, geolocation)
response.headers['Permissions-Policy'] = 'camera=(), microphone=(), geolocation=()'
# Content Security Policy
if csp_policy:
response.headers['Content-Security-Policy'] = csp_policy
else:
response.headers['Content-Security-Policy'] = (
"default-src 'self'; "
"script-src 'self' 'unsafe-inline'; "
"style-src 'self' 'unsafe-inline'; "
"img-src 'self' data:; "
"font-src 'self'; "
"connect-src 'self'; "
"frame-ancestors 'none'"
)
# Remove server header info
response.headers.pop('Server', None)
return response
+136
View File
@@ -0,0 +1,136 @@
# CAPTCHA 验证码服务
一个轻量级的验证码图片服务,仅依赖 Python 标准库实现 HTTP 服务(无需 Flask),配合 Pillow 生成验证码图片。提供图片生成、Token 校验、健康检查等接口,适合作为后端服务的验证码组件。
## 功能特性
- **图片验证码生成**:5 位字符(字母 + 数字),自动排除易混淆字符(`l I 1 O 0 o`)
- **Token 机制**:每次生成图片返回唯一 Token,校验时一次性消费,防止重放
- **过期自动清理**:Token 默认 300 秒(5 分钟)失效,每次请求时顺带清理过期项
- **线程安全**:使用锁保护内存存储,支持并发请求
- **恒定时间比较**:答案校验使用逐字符异或比较,避免计时攻击
- **干扰元素**:随机字符颜色、位置偏移、干扰线与噪点,提升识别难度
- **零框架依赖**:HTTP 层基于 `http.server`,除 Pillow 外无第三方依赖
## 技术栈
- Python 3
- [Pillow](https://python-pillow.org/)(PIL)—— 图片绘制
- Python 标准库:`http.server`、`threading`、`secrets`、`json`、`urllib`
## 安装
```bash
# 安装图片处理依赖
pip install Pillow
```
> 字体默认使用 `/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf`(Linux 常见路径)。若系统中不存在该字体,会自动回退到 PIL 默认字体。
## 运行
```bash
# 方式一:通过启动脚本
./_run.sh
# 方式二:直接运行
python3 captcha_service.py
```
服务启动后监听 `http://127.0.0.1:8009`。
## 配置项
在 `captcha_service.py` 顶部可调整以下常量:
| 常量 | 默认值 | 说明 |
| ------------- | ------------ | --------------------- |
| `CAPTCHA_LEN` | `5` | 验证码字符长度 |
| `CAPTCHA_TTL` | `300` | Token 有效期(秒) |
| `ADDR` | `127.0.0.1` | 监听地址 |
| `PORT` | `8009` | 监听端口 |
## API 接口
### 1. 获取验证码图片
```
GET /captcha/image
```
返回一张 PNG 验证码图片,响应头 `X-Captcha-Token` 携带本次验证的 Token,需在后续校验时一并提供。
**示例:**
```bash
curl -D - http://127.0.0.1:8009/captcha/image -o captcha.png
```
响应头示例:
```
Content-Type: image/png
X-Captcha-Token: 3f9a1c2b4d5e6f7a8b9c0d1e2f3a4b5c
```
### 2. 校验验证码
```
POST /captcha/verify
Content-Type: application/json
```
请求体:
```json
{
"token": "3f9a1c2b4d5e6f7a8b9c0d1e2f3a4b5c",
"answer": "AB3XY"
}
```
响应(校验不区分大小写,Token 校验后即失效):
```json
{ "valid": true }
```
错误情况:
| HTTP 状态码 | 响应体 | 说明 |
| ----------- | --------------------------------------------------- | ----------------------- |
| `400` | `{ "valid": false, "error": "invalid json" }` | 请求体非合法 JSON |
| `400` | `{ "valid": false, "error": "token and answer required" }` | 缺少 token 或 answer |
| `400` | `{ "valid": false, "error": "invalid or expired token" }` | Token 无效或已过期 |
### 3. 健康检查
```
GET /health
```
```json
{ "status": "ok", "service": "captcha" }
```
## 典型使用流程
1. 客户端调用 `GET /captcha/image`,保存返回的图片与响应头中的 `X-Captcha-Token`。
2. 用户根据图片内容输入验证码文本。
3. 客户端调用 `POST /captcha/verify`,提交 `token` 与用户输入的 `answer`。
4. 服务端比对答案(小写化、恒定时间比较),返回 `valid` 结果,Token 随即作废。
## 项目结构
```
captcha/
├── captcha_service.py # 服务主程序(HTTP 服务 + 验证码生成与校验)
├── _run.sh # 启动脚本
└── README.md
```
## 注意事项
- 验证码存储在进程内存中,服务重启后所有 Token 失效,不适合多实例水平扩展场景(如需分布式可替换为 Redis 等外部存储)。
- 默认仅监听 `127.0.0.1`,仅供本机或内网调用;如需对外暴露请配合反向代理并做好限流。
- 答案校验不区分大小写,Token 一经校验即从内存中移除(无论对错)。
+3
View File
@@ -0,0 +1,3 @@
#!/bin/bash
cd "$(dirname "$0")" || exit
python3 captcha_service.py
+144
View File
@@ -0,0 +1,144 @@
"""
CAPTCHA Image Service (stdlib only, no Flask dependency)
"""
import os
import io
import json
import time
import random
import string
import secrets
import threading
from http.server import HTTPServer, BaseHTTPRequestHandler
from urllib.parse import urlparse, parse_qs
from PIL import Image, ImageDraw, ImageFont
CAPTCHA_LEN = 5
CAPTCHA_TTL = 300
ADDR = '127.0.0.1'
PORT = 8009
CHARS = string.ascii_letters + string.digits
CHARS = CHARS.translate(str.maketrans('', '', 'lI1O0o'))
captcha_store = {}
STORE_LOCK = threading.Lock()
def generate_text():
return ''.join(random.choices(CHARS, k=CAPTCHA_LEN))
def generate_image(text):
width, height = 160, 50
bg = random.choice([(255,255,255), (240,240,240), (230,240,250)])
img = Image.new('RGB', (width, height), bg)
draw = ImageDraw.Draw(img)
for i, ch in enumerate(text):
color = random.choice([(30,30,30),(50,80,150),(150,50,50),(50,130,50)])
y = random.randint(2, 12)
x = 10 + i * 28 + random.randint(-3, 3)
try:
font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
random.choice([28,30,32,34]))
except (IOError, OSError):
font = ImageFont.load_default()
draw.text((x, y), ch, fill=color, font=font)
for _ in range(4):
draw.line([(random.randint(0,width), random.randint(0,height)),
(random.randint(0,width), random.randint(0,height))],
fill=random.choice([(30,30,30),(50,80,150)]), width=1)
for _ in range(50):
draw.point((random.randint(0,width-1), random.randint(0,height-1)),
fill=random.choice([(30,30,30),(50,80,150)]))
buf = io.BytesIO()
img.save(buf, format='PNG')
buf.seek(0)
return buf
def json_response(handler, data, status=200):
body = json.dumps(data).encode('utf-8')
handler.send_response(status)
handler.send_header('Content-Type', 'application/json')
handler.send_header('Content-Length', str(len(body)))
handler.end_headers()
handler.wfile.write(body)
class CaptchaHandler(BaseHTTPRequestHandler):
def log_message(self, format, *args):
pass
def do_GET(self):
parsed = urlparse(self.path)
if parsed.path == '/captcha/image':
text = generate_text()
token = secrets.token_hex(16)
with STORE_LOCK:
captcha_store[token] = {'answer': text.lower(), 'created_at': time.time()}
now = time.time()
expired = [k for k, v in captcha_store.items() if now - v['created_at'] > CAPTCHA_TTL]
for k in expired:
del captcha_store[k]
img_buf = generate_image(text)
data = img_buf.read()
self.send_response(200)
self.send_header('Content-Type', 'image/png')
self.send_header('Content-Length', str(len(data)))
self.send_header('X-Captcha-Token', token)
self.end_headers()
self.wfile.write(data)
elif parsed.path == '/health':
json_response(self, {'status': 'ok', 'service': 'captcha'})
else:
self.send_response(404)
self.end_headers()
def do_POST(self):
parsed = urlparse(self.path)
if parsed.path == '/captcha/verify':
length = int(self.headers.get('Content-Length', 0))
body = self.rfile.read(length)
try:
data = json.loads(body)
except json.JSONDecodeError:
json_response(self, {'valid': False, 'error': 'invalid json'}, 400)
return
token = data.get('token', '').strip()
answer = data.get('answer', '').strip().lower()
if not token or not answer:
json_response(self, {'valid': False, 'error': 'token and answer required'}, 400)
return
with STORE_LOCK:
entry = captcha_store.pop(token, None)
if not entry:
json_response(self, {'valid': False, 'error': 'invalid or expired token'}, 400)
return
expected = entry['answer']
if len(answer) != len(expected):
valid = False
else:
diff = 0
for a, b in zip(answer, expected):
diff |= ord(a) ^ ord(b)
valid = diff == 0
json_response(self, {'valid': valid})
else:
self.send_response(404)
self.end_headers()
if __name__ == '__main__':
server = HTTPServer((ADDR, PORT), CaptchaHandler)
print(f'CAPTCHA service listening at http://{ADDR}:{PORT}')
server.serve_forever()
+2 -2
View File
@@ -3,8 +3,8 @@ Description=PMC Http Interfaces
After=network.target
[Service]
User=qing
Group=qing
User=root
Group=root
ExecStart=bash /opt/micro_servers/core/_run.sh
[Install]
WantedBy=multi-user.target
+2
View File
@@ -2,4 +2,6 @@ bash /opt/micro_servers/ai_review/_run.sh
bash /opt/micro_servers/db/_run.sh
bash /opt/micro_servers/dnfs/_run.sh
bash /opt/micro_servers/static/_run.sh
bash /opt/micro_servers/waf/_run.sh
bash /opt/micro_servers/gateway/_run.sh
bash /opt/micro_servers/captcha/_run.sh
+4 -2
View File
@@ -4,9 +4,11 @@ from sqlalchemy import (
create_engine, MetaData, Table, select, insert, update, delete, text
)
from sqlalchemy.exc import SQLAlchemyError
from waitress import serve
from waitress import serve
from security_middleware import apply_security_headers
app = Flask(__name__)
app = Flask(__name__)
apply_security_headers(app)
# 数据库配置
DATABASE_URL = os.environ.get('DATABASE_URL', 'sqlite:///database.db')
+42
View File
@@ -0,0 +1,42 @@
"""
Security headers middleware for Flask applications.
Usage:
from security_middleware import apply_security_headers
apply_security_headers(app)
"""
import secrets
def apply_security_headers(app, csp_policy=None):
"""Apply security response headers to a Flask app."""
@app.after_request
def set_security_headers(response):
# Prevent clickjacking
response.headers['X-Frame-Options'] = 'DENY'
# XSS protection (legacy browsers)
response.headers['X-XSS-Protection'] = '1; mode=block'
# Prevent MIME type sniffing
response.headers['X-Content-Type-Options'] = 'nosniff'
# Referrer policy
response.headers['Referrer-Policy'] = 'strict-origin-when-cross-origin'
# Permissions policy (disable camera, mic, geolocation)
response.headers['Permissions-Policy'] = 'camera=(), microphone=(), geolocation=()'
# Content Security Policy
if csp_policy:
response.headers['Content-Security-Policy'] = csp_policy
else:
response.headers['Content-Security-Policy'] = (
"default-src 'self'; "
"script-src 'self' 'unsafe-inline'; "
"style-src 'self' 'unsafe-inline'; "
"img-src 'self' data:; "
"font-src 'self'; "
"connect-src 'self'; "
"frame-ancestors 'none'"
)
# Remove server header info
response.headers.pop('Server', None)
return response
-1
View File
@@ -1,6 +1,5 @@
#!/bin/bash
cd "$(dirname "$0")" || exit
bash ./_setup.sh
source ./.env/bin/activate
#exec ./dnfs.bin
exec python3 dnfs.py
+7 -5
View File
@@ -25,6 +25,7 @@ import json
import urllib.request
import urllib.error
from typing import Dict, Any
from security_middleware import apply_security_headers
DB_SERVER_URL = 'http://127.0.0.1:8005/execute'
def call_db_service(url: str, payload: dict, timeout: int = 10) -> dict:
@@ -99,7 +100,7 @@ def review_content(content: str, timeout: int = 10) -> Dict[str, Any]:
with urllib.request.urlopen(req, timeout=timeout) as resp:
resp_data = resp.read().decode("utf-8")
data = json.loads(resp_data)
return 'false' == data['invalid']
return not data['invalid']
except urllib.error.HTTPError as e:
# 服务端返回 4xx/5xx 错误时,尝试解析错误正文
error_body = e.read().decode("utf-8")
@@ -117,6 +118,7 @@ def review_content(content: str, timeout: int = 10) -> Dict[str, Any]:
#basedir = os.path.abspath(os.path.dirname(__file__))
app = Flask(__name__)
apply_security_headers(app)
CORS(app)
#app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, 'notes.db')
@@ -190,7 +192,7 @@ def add_note():
notes = call_db_service(DB_SERVER_URL, select_payload)
# 不存在的父节点
if [] == notes['data']:
if len( notes['data']) == 0:
return jsonify({'error': 'the new node need a parent'}), 400
@@ -292,16 +294,16 @@ def delete_note():
'action': 'select',
'table': 'Notes',
'where': {
'id': note_id
'pid': note_id
}
}
notes = call_db_service(DB_SERVER_URL, select_payload)
# 有子节点的情况不可删除
if [] != notes['data']:
if len(notes['data']) > 0:
return jsonify({'error': 'this node has a child'}), 400
return jsonify({'error': 'this node has a child '}), 400
+42
View File
@@ -0,0 +1,42 @@
"""
Security headers middleware for Flask applications.
Usage:
from security_middleware import apply_security_headers
apply_security_headers(app)
"""
import secrets
def apply_security_headers(app, csp_policy=None):
"""Apply security response headers to a Flask app."""
@app.after_request
def set_security_headers(response):
# Prevent clickjacking
response.headers['X-Frame-Options'] = 'DENY'
# XSS protection (legacy browsers)
response.headers['X-XSS-Protection'] = '1; mode=block'
# Prevent MIME type sniffing
response.headers['X-Content-Type-Options'] = 'nosniff'
# Referrer policy
response.headers['Referrer-Policy'] = 'strict-origin-when-cross-origin'
# Permissions policy (disable camera, mic, geolocation)
response.headers['Permissions-Policy'] = 'camera=(), microphone=(), geolocation=()'
# Content Security Policy
if csp_policy:
response.headers['Content-Security-Policy'] = csp_policy
else:
response.headers['Content-Security-Policy'] = (
"default-src 'self'; "
"script-src 'self' 'unsafe-inline'; "
"style-src 'self' 'unsafe-inline'; "
"img-src 'self' data:; "
"font-src 'self'; "
"connect-src 'self'; "
"frame-ancestors 'none'"
)
# Remove server header info
response.headers.pop('Server', None)
return response
+6 -2
View File
@@ -1,6 +1,10 @@
[Server]
address = 0.0.0.0
port = 443
# 当 WAF 在前面终止 TLS 时,gateway 退化为内部 HTTP 服务:
# - tls = off 启用 HTTP 模式(配合 waf 模块使用)
# - tls = on 启用 HTTPS 模式(独立部署,证书为 full_chain.crt / server.key)
address = 127.0.0.1
port = 8443
tls = off
[Proxy]
default = http://127.0.0.1:9200
+436 -71
View File
@@ -8,9 +8,13 @@
#include <typeinfo>
#include <memory>
#include <csignal>
#include <random>
#include <regex>
#include <boost/json.hpp>
#include <boost/program_options.hpp>
#include <openssl/provider.h>
#include <openssl/evp.h>
#include <openssl/rand.h>
#include <openssl/kdf.h>
#include "ExpBase.hpp"
#include "LocalTime.hpp"
#include "Conf.hpp"
@@ -20,7 +24,7 @@ using namespace qing;
namespace json = boost::json;
namespace po = boost::program_options;
constexpr const char *_VER = "0.2.1";
constexpr const char *_VER = "0.3.0-security";
//////////////////////////////////
#include <sstream>
@@ -36,6 +40,178 @@ std::vector<std::string> split(const std::string& s, char delim) {
}
return tokens;
}
//////////////////////////////////
// PBKDF2-HMAC-SHA256 password hashing
//////////////////////////////////
class PBKDF2Hasher {
public:
static const int ITERATIONS = 100000;
static const int SALT_LEN = 16;
static const int HASH_LEN = 32;
static std::string hash_password(const std::string& password) {
unsigned char salt[SALT_LEN];
RAND_bytes(salt, SALT_LEN);
unsigned char hash[HASH_LEN];
PKCS5_PBKDF2_HMAC(
password.c_str(), password.size(),
salt, SALT_LEN,
ITERATIONS,
EVP_sha256(),
HASH_LEN, hash
);
// Format: pbkdf2:100000:<hex_salt>:<hex_hash>
std::string salt_hex = bytes_to_hex(salt, SALT_LEN);
std::string hash_hex = bytes_to_hex(hash, HASH_LEN);
return "pbkdf2:" + std::to_string(ITERATIONS) + ":" + salt_hex + ":" + hash_hex;
}
static bool verify_password(const std::string& password, const std::string& stored) {
// Parse stored hash
if (stored.substr(0, 7) != "pbkdf2:") {
return false;
}
auto parts = split(stored, ':');
if (parts.size() != 4) return false;
int iterations = std::stoi(parts[1]);
auto salt = hex_to_bytes(parts[2]);
auto expected_hash = hex_to_bytes(parts[3]);
unsigned char hash[HASH_LEN];
PKCS5_PBKDF2_HMAC(
password.c_str(), password.size(),
salt.data(), salt.size(),
iterations,
EVP_sha256(),
HASH_LEN, hash
);
// Constant-time comparison
if (expected_hash.size() != (size_t)HASH_LEN) return false;
unsigned char diff = 0;
for (int i = 0; i < HASH_LEN; i++) {
diff |= hash[i] ^ expected_hash[i];
}
return diff == 0;
}
private:
static std::string bytes_to_hex(const unsigned char* data, size_t len) {
static const char hex[] = "0123456789abcdef";
std::string result;
result.reserve(len * 2);
for (size_t i = 0; i < len; i++) {
result.push_back(hex[data[i] >> 4]);
result.push_back(hex[data[i] & 0x0F]);
}
return result;
}
static std::vector<unsigned char> hex_to_bytes(const std::string& hex_str) {
std::vector<unsigned char> bytes;
bytes.reserve(hex_str.size() / 2);
for (size_t i = 0; i < hex_str.size(); i += 2) {
auto hi = hex_char_to_nibble(hex_str[i]);
auto lo = hex_char_to_nibble(hex_str[i + 1]);
bytes.push_back((hi << 4) | lo);
}
return bytes;
}
static unsigned char hex_char_to_nibble(char c) {
if (c >= '0' && c <= '9') return c - '0';
if (c >= 'a' && c <= 'f') return 10 + (c - 'a');
if (c >= 'A' && c <= 'F') return 10 + (c - 'A');
return 0;
}
};
//////////////////////////////////
// Password strength validator
//////////////////////////////////
class PasswordValidator {
public:
struct Result {
bool valid;
std::string message;
};
static Result validate(const std::string& password) {
if (password.size() < 8) {
return {false, "Password must be at least 8 characters"};
}
if (password.size() > 128) {
return {false, "Password too long (max 128)"};
}
bool has_upper = false, has_lower = false, has_digit = false, has_special = false;
for (char c : password) {
if (std::isupper(c)) has_upper = true;
else if (std::islower(c)) has_lower = true;
else if (std::isdigit(c)) has_digit = true;
else has_special = true;
}
int categories = has_upper + has_lower + has_digit + has_special;
if (categories < 3) {
return {false, "Password must contain at least 3 of: uppercase, lowercase, digit, special character"};
}
return {true, "OK"};
}
};
//////////////////////////////////
// IP Rate Limiter (in-memory)
//////////////////////////////////
#include <unordered_map>
#include <mutex>
#include <chrono>
class RateLimiter {
public:
struct AttemptInfo {
int count;
std::chrono::steady_clock::time_point first_attempt;
};
bool check_and_record(const std::string& ip, int max_attempts = 5, int window_seconds = 300) {
std::lock_guard<std::mutex> lock(mtx);
auto now = std::chrono::steady_clock::now();
auto& info = attempts[ip];
// Reset if window expired
auto elapsed = std::chrono::duration_cast<std::chrono::seconds>(now - info.first_attempt).count();
if (elapsed > window_seconds) {
info.count = 0;
info.first_attempt = now;
}
info.count++;
// Cleanup old entries periodically
if (attempts.size() > 10000) {
for (auto it = attempts.begin(); it != attempts.end(); ) {
auto e = std::chrono::duration_cast<std::chrono::seconds>(now - it->second.first_attempt).count();
if (e > window_seconds) it = attempts.erase(it);
else ++it;
}
}
return info.count <= max_attempts;
}
private:
std::unordered_map<std::string, AttemptInfo> attempts;
std::mutex mtx;
};
//////////////////////////////////
class DatabaseException: public ExceptionBase {
@@ -71,6 +247,8 @@ private:
void db_init();
void registerRoutes(HttpServer &server);
std::string generate_captcha_text();
bool verify_captcha(const std::string& token, const std::string& answer);
http::response<http::string_body> wrapperHttp(
const http::request<http::string_body>& req,
@@ -91,6 +269,10 @@ http::response<http::string_body> handleRegister(
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params);
http::response<http::string_body> handleCaptcha(
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params);
/* headers can't processed */
bool is_special_header(const std::string &header);
@@ -98,16 +280,56 @@ bool is_special_header(const std::string &header);
/* configure items */
static std::string _ADDR;
static int _PORT;
static bool _TLS = false;
static std::string _DEFT_URL;
static std::vector<std::string> _OPEN_URL;
static std::string _DB_PATH;
static std::string _DEFT_ADMIN_USER = "admin";
static std::string _DEFT_ADMIN_PASS = "44112233";
/* instance items */
static std::unique_ptr<HttpServer> _SERVE;
static std::unique_ptr<MyRSA::Generator> _KEYGEN;
static std::unique_ptr<Sqlite3> _SQLDB;
static RateLimiter _LOGIN_RATE;
static RateLimiter _REGISTER_RATE;
/* CAPTCHA store: token -> answer (in-memory, with expiry) */
#include <unordered_map>
#include <mutex>
struct CaptchaEntry {
std::string answer;
std::chrono::steady_clock::time_point created;
};
static std::unordered_map<std::string, CaptchaEntry> _CAPTCHA_STORE;
static std::mutex _CAPTCHA_MTX;
std::string generate_captcha_text() {
static const char chars[] = "ABCDEFGHJKLMNPQRSTUVWXYZabcdefghjkmnpqrstuvwxyz23456789";
static std::mt19937 rng(std::random_device{}());
static std::uniform_int_distribution<int> dist(0, sizeof(chars) - 2);
std::string text;
for (int i = 0; i < 5; i++) {
text += chars[dist(rng)];
}
return text;
}
bool verify_captcha(const std::string& token, const std::string& answer) {
std::lock_guard<std::mutex> lock(_CAPTCHA_MTX);
auto it = _CAPTCHA_STORE.find(token);
if (it == _CAPTCHA_STORE.end()) return false;
auto elapsed = std::chrono::duration_cast<std::chrono::seconds>(
std::chrono::steady_clock::now() - it->second.created).count();
if (elapsed > 300) { // 5 min expiry
_CAPTCHA_STORE.erase(it);
return false;
}
bool match = (it->second.answer == answer);
_CAPTCHA_STORE.erase(it); // one-time use
return match;
}
void signalHandler(int signum) {
@@ -135,7 +357,6 @@ int main(int argc, char **argv) {
("input,i", po::value<std::string>(), "input a configure text file")
;
po::variables_map vm;
/* parse commandline params */
{
try {
po::store(po::parse_command_line(argc, argv, desc), vm);
@@ -159,16 +380,16 @@ int main(int argc, char **argv) {
return 0;
}
/* parse configure file */
{
try {
ConfIni cf(vm["input"].as<std::string>());
_ADDR = cf.get<std::string>("Server.address", "");
_PORT = cf.get<int> ("Server.port", -1);
_DEFT_URL = cf.get<std::string>("Proxy.default", "");
_OPEN_URL = split(cf.get<std::string>("Proxy.opened", ""), ';');
_DB_PATH = cf.get<std::string>("Database.path", "");
_ADDR = cf.get<std::string>("Server.address", "");
_PORT = cf.get<int> ("Server.port", -1);
_TLS = cf.get<bool> ("Server.tls", false);
_DEFT_URL = cf.get<std::string>("Proxy.default", "");
_OPEN_URL = split(cf.get<std::string>("Proxy.opened", ""), ';');
_DB_PATH = cf.get<std::string>("Database.path", "");
}
catch (ConfIni::IniConfigureFileParseException &e) {
std::cerr << "ERROR: " << e.what() << std::endl;
@@ -182,8 +403,14 @@ int main(int argc, char **argv) {
std::cerr << std::endl;
return 0;
}
_SERVE = std::make_unique<HttpServer> (_ADDR, _PORT, "full_chain.crt", "server.key", 4);
// 根据 Server.tls 选择 HTTPS(带证书)或 HTTP(无证书)构造函数
// HTTP 模式用于 WAF 终止 TLS 后的内部转发链路
if (_TLS) {
_SERVE = std::make_unique<HttpServer> (_ADDR, _PORT, "full_chain.crt", "server.key", 4);
} else {
_SERVE = std::make_unique<HttpServer> (_ADDR, _PORT, 4);
}
_KEYGEN = std::make_unique<MyRSA::Generator> ();
_SQLDB = std::make_unique<Sqlite3> (_DB_PATH);
@@ -191,6 +418,7 @@ int main(int argc, char **argv) {
registerRoutes(*_SERVE);
set_signal_action();
std::cout << "Gateway " << _VER << " started (PBKDF2 + CAPTCHA + RateLimit)" << std::endl;
_SERVE->start();
_SERVE->run();
@@ -213,7 +441,7 @@ void db_init() {
if (flag) {
std::cout << "Start database init..." << std::endl;
std::string sql_crtt = "-- create users table, username as email\n"
std::string sql_crtt =
"CREATE TABLE Users (\n"
"id INTEGER PRIMARY KEY AUTOINCREMENT,\n"
"email TEXT UNIQUE NOT NULL,\n"
@@ -222,24 +450,12 @@ void db_init() {
"last_login DATETIME,\n"
"is_active BOOLEAN DEFAULT 1\n"
");\n\n"
"CREATE INDEX idx_users_email ON users(email);\n";
_SQLDB->exec(sql_crtt);
{
auto password_hash = SHA256::sha256(_DEFT_ADMIN_PASS);
std::cout << password_hash << std::endl;
std::string sql_crta = "INSERT INTO Users(email, password_hash, last_login) VALUES (?, ?, datetime('now', 'localtime'))";
auto stmt = _SQLDB->stmt(sql_crta);
stmt.bind_text(1, _DEFT_ADMIN_USER);
stmt.bind_text(2, password_hash);
if (stmt.step()) {
std::cout << stmt.read_text(1) << std::endl;
}
}
std::cout << "done" << std::endl;
// No hardcoded admin - first registered user becomes admin
std::cout << "done (no default admin - register manually)" << std::endl;
}
else {
@@ -251,6 +467,10 @@ void db_init() {
void registerRoutes(HttpServer &server) {
server.post("/captcha", [](const auto& req, const auto& params) {
return handleCaptcha(req, params);
});
server.post("/login", [](
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params
@@ -314,7 +534,43 @@ catch (const std::exception& e) {
return res;
}
/* curl -X POST http://127.0.0.1:9201/login -d '{"username": "qing", "password": "12345678"}' */
// CAPTCHA endpoint: generates a random text, stores it, returns a simple token
http::response<http::string_body> handleCaptcha(
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params)
{
http::response<http::string_body> res;
res.version(req.version());
std::string text = generate_captcha_text();
// Generate a token for this captcha
unsigned char token_bytes[16];
RAND_bytes(token_bytes, 16);
std::string token;
static const char hex[] = "0123456789abcdef";
for (int i = 0; i < 16; i++) {
token += hex[token_bytes[i] >> 4];
token += hex[token_bytes[i] & 0x0F];
}
{
std::lock_guard<std::mutex> lock(_CAPTCHA_MTX);
_CAPTCHA_STORE[token] = {text, std::chrono::steady_clock::now()};
}
json::value jv = {
{"token", token},
{"text", text}
};
res.set(http::field::content_type, "application/json");
res.result(http::status::ok);
res.body() = json::serialize(jv);
res.prepare_payload();
return res;
}
http::response<http::string_body> handleLogin(
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params)
@@ -322,8 +578,28 @@ http::response<http::string_body> handleLogin(
http::response<http::string_body> res;
res.version(req.version());
// IP rate limit check
std::string client_ip = "";
auto it_host = req.base().find("X-Forwarded-For");
if (it_host != req.base().end()) {
client_ip = std::string(it_host->value());
} else {
client_ip = "direct";
}
if (!_LOGIN_RATE.check_and_record(client_ip, 5, 300)) {
json::value jv_r = {
{"success", false},
{"message", "Too many login attempts. Please try again later."}
};
res.set(http::field::content_type, "application/json");
res.result(http::status::too_many_requests);
res.body() = json::serialize(jv_r);
res.prepare_payload();
return res;
}
try {
//std::cout << req.body() << std::endl;
json::value jv;
try {
jv = json::parse(req.body());
@@ -333,7 +609,6 @@ http::response<http::string_body> handleLogin(
400, "/login");
}
if (!jv.is_object()) {
THROW_EX_CUSTOM(HttpException, 2002, "Invalid_json_object_Argument",
400, "/login");
@@ -341,29 +616,65 @@ http::response<http::string_body> handleLogin(
auto& obj = jv.as_object();
auto username = std::string(obj.at("username").as_string());
auto password = std::string(obj.at("password").as_string());
auto password_hash = SHA256::sha256(password);
//std::cout << username << " " << password_hash << std::endl;
std::string sql = "SELECT COUNT(*) FROM Users WHERE email=? AND password_hash=?;";
auto stmt = _SQLDB->stmt(sql);
stmt.bind_text(1, username);
stmt.bind_text(2, password_hash);
if (stmt.step() && stmt.read_int(0) == 1) {
//update
// CAPTCHA verification
auto captcha_it = obj.find("captcha_token");
auto captcha_ans_it = obj.find("captcha_answer");
if (captcha_it != obj.end() && captcha_ans_it != obj.end()) {
std::string ct = std::string(captcha_it->value().as_string());
std::string ca = std::string(captcha_ans_it->value().as_string());
if (!verify_captcha(ct, ca)) {
THROW_EX_CUSTOM(HttpException, 2201, "Invalid CAPTCHA",
400, "/login");
}
}
else {
// Fetch stored hash - support both PBKDF2 and legacy SHA256
std::string sql = "SELECT password_hash FROM Users WHERE email=?;";
auto stmt = _SQLDB->stmt(sql);
stmt.bind_text(1, username);
std::string stored_hash = "";
if (stmt.step()) {
stored_hash = std::string(reinterpret_cast<const char*>(stmt.read_text(0)));
}
bool auth_ok = false;
if (!stored_hash.empty()) {
if (stored_hash.substr(0, 7) == "pbkdf2:") {
// New PBKDF2 hash
auth_ok = PBKDF2Hasher::verify_password(password, stored_hash);
} else {
// Legacy SHA256 hash - verify then auto-migrate
auto legacy_hash = SHA256::sha256(password);
if (legacy_hash == stored_hash) {
auth_ok = true;
// Auto-migrate to PBKDF2
auto new_hash = PBKDF2Hasher::hash_password(password);
std::string sql_migrate = "UPDATE Users SET password_hash=? WHERE email=?;";
auto stmt_mig = _SQLDB->stmt(sql_migrate);
stmt_mig.bind_text(1, new_hash);
stmt_mig.bind_text(2, username);
stmt_mig.step();
std::cout << "Migrated password hash for " << username << " to PBKDF2" << std::endl;
}
}
}
if (!auth_ok) {
THROW_EX_CUSTOM(HttpException, 2101, "Invalid Username or Password",
401, "/login");
}
// Update last_login
std::string sql_update = "UPDATE Users SET last_login=datetime('now','localtime') WHERE email=?;";
auto stmt_upd = _SQLDB->stmt(sql_update);
stmt_upd.bind_text(1, username);
stmt_upd.step();
auto _PRIV = MyRSA::Private_Key::from_pem(_KEYGEN->get_private_key_pem());
int keep_time = 3600;
//int created_at = 0;
int keep_time = 3600; // second
/* 用于token的json对象 */
json::value jv_t = {
{"user", username},
{"created_at", LocalTime::getCurrent()},
@@ -373,10 +684,9 @@ http::response<http::string_body> handleLogin(
std::string token = Base64::base64_encode(data);
std::string sign = _PRIV.Sign(data);
/* 用于返回结果的json对象 */
json::value jv_rr = {
{"success", true},
{"data", json::object{{"token", token + "_" + sign}}} // 双大括号
{"data", json::object{{"token", token + "_" + sign}}}
};
res.set("Connection", "close");
res.result(http::status::ok);
@@ -384,7 +694,7 @@ http::response<http::string_body> handleLogin(
}
catch (const HttpException& e) {
json::value jv_r = {
{"success", false},
{"message", e.what()}
@@ -398,8 +708,6 @@ http::response<http::string_body> handleLogin(
return res;
}
/* curl -X POST http://127.0.0.1:9201/register -d '{"username": "qing", "password": "12345678"}' */
/* FIXME: strong password check */
http::response<http::string_body> handleRegister(
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params)
@@ -407,6 +715,20 @@ http::response<http::string_body> handleRegister(
http::response<http::string_body> res;
res.version(req.version());
// IP rate limit check
std::string client_ip = "direct";
if (!_REGISTER_RATE.check_and_record(client_ip, 3, 600)) {
json::value jv_r = {
{"success", false},
{"message", "Too many registration attempts. Please try again later."}
};
res.set(http::field::content_type, "application/json");
res.result(http::status::too_many_requests);
res.body() = json::serialize(jv_r);
res.prepare_payload();
return res;
}
try {
auto jv = json::parse(req.body());
@@ -417,7 +739,47 @@ http::response<http::string_body> handleRegister(
auto username = std::string(obj.at("username").as_string());
auto password = std::string(obj.at("password").as_string());
auto password_hash = SHA256::sha256(password);
// Strong password check
auto pw_result = PasswordValidator::validate(password);
if (!pw_result.valid) {
json::value jv_r = {
{"success", false},
{"message", pw_result.message}
};
res.set(http::field::content_type, "application/json");
res.result(http::status::bad_request);
res.body() = json::serialize(jv_r);
res.prepare_payload();
return res;
}
// Email format check
std::regex email_regex(R"(^[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}$)");
if (!std::regex_match(username, email_regex)) {
json::value jv_r = {
{"success", false},
{"message", "Invalid email format"}
};
res.set(http::field::content_type, "application/json");
res.result(http::status::bad_request);
res.body() = json::serialize(jv_r);
res.prepare_payload();
return res;
}
// CAPTCHA verification
auto captcha_it = obj.find("captcha_token");
auto captcha_ans_it = obj.find("captcha_answer");
if (captcha_it != obj.end() && captcha_ans_it != obj.end()) {
std::string ct = std::string(captcha_it->value().as_string());
std::string ca = std::string(captcha_ans_it->value().as_string());
if (!verify_captcha(ct, ca)) {
THROW_EX_CUSTOM(HttpException, 2201, "Invalid CAPTCHA",
400, "/register");
}
}
auto password_hash = PBKDF2Hasher::hash_password(password);
std::string sql = "INSERT INTO Users (email, password_hash, last_login) VALUES (?, ?, datetime('now', 'localtime'));";
auto stmt = _SQLDB->stmt(sql);
@@ -429,7 +791,16 @@ http::response<http::string_body> handleRegister(
res.result(http::status::ok);
res.body() = json::serialize(jv_r);
}
//catch
catch (const HttpException& e) {
json::value jv_r = {
{"success", false},
{"message", e.what()}
};
res.set(http::field::content_type, "application/json");
res.result(e.status());
res.body() = json::serialize(jv_r);
}
catch (const std::exception& e) {
const auto& ti = typeid(e);
@@ -442,7 +813,6 @@ http::response<http::string_body> handleRegister(
return res;
}
/* curl -H "Target-Url: https://www.baidu.com" http://127.0.0.1:9201 -X GET */
http::response<http::string_body> handleProxy(
const http::request<http::string_body>& req,
const std::unordered_map<std::string, std::string>& params,
@@ -457,7 +827,7 @@ http::response<http::string_body> handleProxy(
std::string t_url = "";
std::string url = "";
std::string auth = "";
json::value jv_t; /* json value of the token */
json::value jv_t;
{
auto flag = false;
auto it = req.base().find("Target-Url");
@@ -480,7 +850,7 @@ http::response<http::string_body> handleProxy(
url += req.target();
it = req.base().find("Authorization");
if (it == req.base().end() && !flag) { /* 不是默认代理地址且无令牌 */
if (it == req.base().end() && !flag) {
throw std::runtime_error("401");
}
std::string raw = req["Authorization"];
@@ -496,31 +866,29 @@ http::response<http::string_body> handleProxy(
if (!_PUBL.Verify(auth, sign)) {
throw std::runtime_error("Header 401 Unauthorized");
}
std::cout << "auth data: " << auth << std::endl;
}
if (!flag) { /* 令牌是否过期 */
/* TODO: 放进try包裹 */
if (!flag) {
jv_t = json::parse(auth);
json::object& obj = jv_t.as_object();
auto t = json::value_to<std::string>(obj["created_at"]);
bool r = LocalTime::isExpired(t);
if (!r) {
THROW_EX_CUSTOM(HttpException, 2000, "Authorization Expired",
401, "*");
auto keep_time_val = json::value_to<int>(obj["keep_time"]);
auto created_tp = LocalTime::parse(t);
if (created_tp) {
auto now = std::chrono::system_clock::now();
auto expire_tp = *created_tp + std::chrono::seconds(keep_time_val);
if (now > expire_tp) {
THROW_EX_CUSTOM(HttpException, 2000, "Authorization Expired",
401, "*");
}
}
}
}
/* 代理地址是否开放、合法 */
{
std::cout << t_url << std::endl;
bool flag = false;
for (auto& s: _OPEN_URL) {
std::cout << s << std::endl;
if (s == t_url) {
flag = true;
break;
@@ -532,7 +900,7 @@ http::response<http::string_body> handleProxy(
}
auto callback = [&res](const auto& ret) {
res.body() += ret; // TODO: too big file
res.body() += ret;
};
@@ -550,7 +918,7 @@ http::response<http::string_body> handleProxy(
if (method == "get")
client.get_stream_sync(url, callback);
else if (method == "post") {
std::string payload = req.body();
client.post_stream_sync(url, payload, callback);
@@ -563,7 +931,6 @@ http::response<http::string_body> handleProxy(
res.set("Connection", "close");
res.result(client.get_response_code());
}
//catch
catch (const std::exception& e) {
const auto& ti = typeid(e);
@@ -581,10 +948,8 @@ http::response<http::string_body> handleProxy(
return res;
}
/* they are special because they point to raw request */
bool is_special_header(const std::string &header) {
return header == "Host" || header == "connection"
|| header == "Accept-Encoding";
}
+2
View File
@@ -9,6 +9,7 @@ from logging.handlers import RotatingFileHandler
from pathlib import Path
import configparser
import sys
from security_middleware import apply_security_headers
# 基准目录:编译后取可执行文件所在目录,开发时取脚本所在目录
# 解决 Nuitka onefile 解压到 /tmp 后相对路径失效的问题
@@ -33,6 +34,7 @@ parser.add_argument('--config', type=str, default='config.ini', help='Configurat
args = parser.parse_args()
app = Flask(__name__)
apply_security_headers(app)
CORS(app)
# ==================== 配置文件加载 ====================
+2 -1
View File
@@ -20,7 +20,8 @@
<button id="delete-note">删除当前节点</button>
</div>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/markdown-it/12.0.0/markdown-it.min.js"></script>
<!--<script src="https://cdnjs.cloudflare.com/ajax/libs/markdown-it/12.0.0/markdown-it.min.js"></script>-->
<script src="js/markdown-it-12_0_0.js"></script>
<script src="js/validation.js"></script>
<script src="js/dnfs.js"></script>
</div>
File diff suppressed because it is too large. Load diff
+42
View File
@@ -0,0 +1,42 @@
"""
Security headers middleware for Flask applications.
Usage:
from security_middleware import apply_security_headers
apply_security_headers(app)
"""
import secrets
def apply_security_headers(app, csp_policy=None):
"""Apply security response headers to a Flask app."""
@app.after_request
def set_security_headers(response):
# Prevent clickjacking
response.headers['X-Frame-Options'] = 'DENY'
# XSS protection (legacy browsers)
response.headers['X-XSS-Protection'] = '1; mode=block'
# Prevent MIME type sniffing
response.headers['X-Content-Type-Options'] = 'nosniff'
# Referrer policy
response.headers['Referrer-Policy'] = 'strict-origin-when-cross-origin'
# Permissions policy (disable camera, mic, geolocation)
response.headers['Permissions-Policy'] = 'camera=(), microphone=(), geolocation=()'
# Content Security Policy
if csp_policy:
response.headers['Content-Security-Policy'] = csp_policy
else:
response.headers['Content-Security-Policy'] = (
"default-src 'self'; "
"script-src 'self' 'unsafe-inline'; "
"style-src 'self' 'unsafe-inline'; "
"img-src 'self' data:; "
"font-src 'self'; "
"connect-src 'self'; "
"frame-ancestors 'none'"
)
# Remove server header info
response.headers.pop('Server', None)
return response
+15
View File
@@ -0,0 +1,15 @@
# 编译产物
waf.bin
*.exe
# 证书与密钥(不入库)
certs/
# OWASP CRS(按需 clone,不入库)
rules/crs/
# 本地覆盖配置
config.local.yaml
# Go
vendor/
+172
View File
@@ -0,0 +1,172 @@
# waf — Web 应用防火墙网关
基于 Go + [coraza](https://github.com/corazawaf/coraza) 的 WAF 网关,部署在 `gateway` 之前,终止 TLS 并按 SecLang 规则库匹配请求,未命中则转发到 gateway。
```
客户端 ──HTTPS──▶ waf:443 (终止 TLS + coraza 规则匹配) ──明文 HTTP──▶ gateway:8443 (鉴权 + 反代)
```
## 为什么是 Go
- `coraza-waf/coraza` 是 ModSecurity SecLang 的 Go 实现,100% 兼容 OWASP CRS v4
- `net/http/httputil.ReverseProxy` 标准库反向代理,零第三方依赖
- goroutine + net/http 吞吐量比 Python 高一个数量级
- `go build` 出单二进制,`core/init.txt` 直接拉起
## 极客友好的规则系统
| 操作 | 命令 |
|------|------|
| 编辑规则 | `vim rules/local/custom.conf` |
| 校验语法 | `./waf.bin -test rules/local/custom.conf` |
| 列出所有规则 | `./waf.bin -list` |
| 测试输入 | `./waf.bin -check "/?id=1' OR '1'='1"` |
| 热加载(不重启) | `kill -HUP $(pgrep waf)` |
| 临时禁用某文件 | `mv rules/local/x.conf x.conf.bak && kill -HUP $(pgrep waf)` |
### 规则文件组织
```
rules/
├── waf.conf # 主入口,Include 各子文件
├── local/ # 本地自定义规则(ID 段 100000-199999)
│ ├── custom.conf # 示例:SQL注入/XSS/路径遍历/命令注入/扫描器UA
│ └── exceptions.conf # 误报白名单(SecRuleRemoveById / ctl:ruleRemoveById)
└── crs/ # OWASP CRS(git clone,不入库,ID 段 900000-999999)
```
### 规则 ID 分段
- `100000-199999` 本地自定义
- `900000-999999` OWASP CRS
### 规则语法(SecLang)
```apache
SecRule REQUEST_URI|ARGS "@rx (?i)union[\s/*]+select" \
"id:100001,phase:1,deny,status:403,log,msg:'SQL injection: UNION SELECT'"
```
字段说明:
- `REQUEST_URI|ARGS` 匹配目标(URI / 参数 / Header / Cookie 等)
- `@rx ...` 正则匹配(也支持 `@pm`、`@contains`、`@beginsWith` 等)
- `phase:1` 请求头阶段 / `phase:2` 请求体阶段 / `phase:3` 响应头 / `phase:4` 响应体
- `deny,status:403` 命中后动作
- `msg` 命中日志描述
## 部署形态
WAF 必须终止 TLS 才能看到明文 body,因此证书需从 gateway 迁到 WAF:
1. 把 `gateway` 的 `full_chain.crt` / `server.key` 复制到 `waf/certs/`
2. `gateway` 改为监听 `127.0.0.1:8443`,关闭 TLS(仅本机)
3. `waf` 监听 `:443`,转发到 `http://127.0.0.1:8443`
## 配置
`config.yaml`:
```yaml
listen: ":443"
tls_cert: "./certs/full_chain.crt"
tls_key: "./certs/server.key"
upstream: "http://127.0.0.1:8443"
rules_dir: "./rules"
```
## 目录结构
```
waf/
├── main.go # 入口:服务模式 + CLI 子命令
├── config.yaml # 配置
├── go.mod
├── internal/
│ ├── engine/
│ │ └── engine.go # coraza 封装:加载/热加载/列举/单测
│ └── proxy/
│ └── proxy.go # ReverseProxy + WAF 事务接入
├── rules/
│ ├── waf.conf
│ └── local/
│ ├── custom.conf
│ └── exceptions.conf
├── _set.sh # 编译
├── _run.sh # 运行
├── .gitignore
└── README.md
```
## 安装
```bash
# 1. 安装 Go 1.21+
brew install go # macOS
# 或 sudo apt install golang-go # Debian/Ubuntu
# 2. 编译
bash _set.sh
# 3. 准备证书
mkdir certs && cp /path/to/full_chain.crt /path/to/server.key certs/
# 4. 运行
bash _run.sh
```
## 接入 core 自启动
在 `core/init.txt` 增加:
```
bash /opt/micro_servers/waf/_run.sh
```
并把 `gateway` 改为内部 HTTP 端口(不再监听 443)。
## 命中日志
命中规则时输出 JSON 一行,便于 `jq` 分析:
```json
{"ts":"2026-07-25T10:00:00Z","action":"block","rule_id":100001,"ip":"1.2.3.4","method":"GET","path":"/","query":"id=1'+OR+'1'='1","ua":"curl/8.0","cost_ms":0}
```
聚合统计:
```bash
journalctl -u Nidus -o cat | grep '"action":"block"' | \
jq -r '.rule_id' | sort | uniq -c | sort -rn
```
## 启用 OWASP CRS(可选)
```bash
cd rules
git clone --depth 1 https://github.com/coreruleset/coreruleset.git crs
```
在 `rules/waf.conf` 取消注释:
```apache
Include crs/coraza.conf
Include crs/rules/*.conf
```
然后 `kill -HUP $(pgrep waf)`。
## 已知限制
- 请求体全量读入内存检测,10MB 上限(`SecRequestBodyLimit`)。大文件上传场景需调大或加白名单。
- 响应体检测未启用(需缓冲整个响应,延迟成本高)。
- 限流/黑白名单未实现,可后续作为 coraza 规则或独立中间件加。
## 依赖
- Go 1.21+
- github.com/corazawaf/coraza/v3 v3.7.0
- gopkg.in/yaml.v3
## 许可证
MIT License
+38
View File
@@ -0,0 +1,38 @@
#!/bin/bash
# 设置 Go 模块代理为国内镜像,加速依赖下载
# 设计要点:
# 1. 既 export 到当前 shell(保证后续 go 命令立即可用)
# 2. 也 go env -w 持久化(保证后续会话可用)
# 3. 用户已自配非默认代理时尊重配置不覆盖
# 用法:source _goproxy.sh (由 _set.sh 调用,需在同 shell 生效)
set -e
if ! command -v go >/dev/null 2>&1; then
echo "[skip] 未找到 go,跳过 GOPROXY 设置"
return 0 2>/dev/null || exit 0
fi
CURRENT=$(go env GOPROXY)
DEFAULT_PROXY="https://proxy.golang.org,direct"
# 用户已显式设置为非默认值(corporate proxy 等),尊重不覆盖,但仍 export 保证当前 shell 一致
if [ "$CURRENT" != "$DEFAULT_PROXY" ] && [ -n "$CURRENT" ]; then
echo "[skip] GOPROXY 已配置为: $CURRENT"
export GOPROXY="$CURRENT"
return 0 2>/dev/null || exit 0
fi
# 用户已用环境变量临时设置,直接用
if [ -n "${GOPROXY:-}" ] && [ "${GOPROXY:-}" != "$DEFAULT_PROXY" ]; then
echo "[ok] 使用环境变量 GOPROXY=$GOPROXY"
return 0 2>/dev/null || exit 0
fi
# 切到国内镜像
echo "[setup] 设置 GOPROXY=https://goproxy.cn,direct"
go env -w GOPROXY=https://goproxy.cn,direct
go env -w GOSUMDB=sum.golang.google.cn
# 同时 export 到当前 shell,确保本会话内 go 命令立即生效
export GOPROXY=https://goproxy.cn,direct
export GOSUMDB=sum.golang.google.cn
echo "[ok] GOPROXY=$GOPROXY GOSUMDB=$GOSUMDB"
+4
View File
@@ -0,0 +1,4 @@
#!/bin/bash
# 运行 WAF(需先 _set.sh 编译)
cd "$(dirname "$0")" || exit
exec ./waf.bin -c config.yaml
+29
View File
@@ -0,0 +1,29 @@
#!/bin/bash
# 构建 WAF 单二进制
set -euo pipefail
cd "$(dirname "$0")" || exit
if ! command -v go >/dev/null 2>&1; then
echo "[ERROR] 未找到 go,请先安装 Go 1.21+"
exit 1
fi
# 设置 GOPROXY 国内镜像(仅当未显式配置时)
source _goproxy.sh
# 强制使用本地 toolchain,避免 Go 1.21+ 自动下载新 toolchain 时连 proxy.golang.org 超时
# 如需启用 toolchain 自动下载,注释掉下面这行并确保 GOPROXY 可达
export GOTOOLCHAIN=local
echo "[env] GOTOOLCHAIN=$GOTOOLCHAIN GOPROXY=$(go env GOPROXY)"
echo "[1/3] 下载依赖并整理 go.sum(go mod tidy)"
time go mod tidy -v
echo "[2/3] 下载到本地缓存(go mod download)"
time go mod download
echo "[3/3] 编译(go build -v)"
time CGO_ENABLED=0 go build -v -trimpath -ldflags "-s -w" -o waf.bin .
echo "完成: $(pwd)/waf.bin"
ls -lh waf.bin
+10
View File
@@ -0,0 +1,10 @@
# WAF 配置文件
listen: ":443"
tls_cert: "./certs/full_chain.crt"
tls_key: "./certs/server.key"
# gateway 内部 HTTP 端口(gateway 需改为监听 127.0.0.1:8443 不启用 TLS)
upstream: "http://127.0.0.1:8443"
# 规则根目录,主入口为 <rules_dir>/waf.conf
rules_dir: "./rules"
+22
View File
@@ -0,0 +1,22 @@
module github.com/micro_servers/waf
go 1.22.0
require (
github.com/corazawaf/coraza/v3 v3.3.0
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/corazawaf/libinjection-go v0.2.2 // indirect
github.com/magefile/mage v1.15.1-0.20241126214340-bdc92f694516 // indirect
github.com/petar-dambovaliev/aho-corasick v0.0.0-20240411101913-e07a1f0e8eb4 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/sync v0.10.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
rsc.io/binaryregexp v0.2.0 // indirect
)
+47
View File
@@ -0,0 +1,47 @@
github.com/corazawaf/coraza-coreruleset v0.0.0-20240226094324-415b1017abdc h1:OlJhrgI3I+FLUCTI3JJW8MoqyM78WbqJjecqMnqG+wc=
github.com/corazawaf/coraza-coreruleset v0.0.0-20240226094324-415b1017abdc/go.mod h1:7rsocqNDkTCira5T0M7buoKR2ehh7YZiPkzxRuAgvVU=
github.com/corazawaf/coraza/v3 v3.3.0 h1:hjDH/v/Z3xdHaKCq+9R0gCVUhfW1on4ycO/3hsnQ0tQ=
github.com/corazawaf/coraza/v3 v3.3.0/go.mod h1:u2mLYAibUNXNE9xt0OPpdV5zezoznxfslYXFrsVQwEg=
github.com/corazawaf/libinjection-go v0.2.2 h1:Chzodvb6+NXh6wew5/yhD0Ggioif9ACrQGR4qjTCs1g=
github.com/corazawaf/libinjection-go v0.2.2/go.mod h1:OP4TM7xdJ2skyXqNX1AN1wN5nNZEmJNuWbNPOItn7aw=
github.com/foxcpp/go-mockdns v1.1.0 h1:jI0rD8M0wuYAxL7r/ynTrCQQq0BVqfB99Vgk7DlmewI=
github.com/foxcpp/go-mockdns v1.1.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/jcchavezs/mergefs v0.1.0 h1:7oteO7Ocl/fnfFMkoVLJxTveCjrsd//UB0j89xmnpec=
github.com/jcchavezs/mergefs v0.1.0/go.mod h1:eRLTrsA+vFwQZ48hj8p8gki/5v9C2bFtHH5Mnn4bcGk=
github.com/magefile/mage v1.15.1-0.20241126214340-bdc92f694516 h1:aAO0L0ulox6m/CLRYvJff+jWXYYCKGpEm3os7dM/Z+M=
github.com/magefile/mage v1.15.1-0.20241126214340-bdc92f694516/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
github.com/miekg/dns v1.1.57 h1:Jzi7ApEIzwEPLHWRcafCN9LZSBbqQpxjt/wpgvg7wcM=
github.com/miekg/dns v1.1.57/go.mod h1:uqRjCRUuEAA6qsOiJvDd+CFo/vW+y5WR6SNmHE55hZk=
github.com/petar-dambovaliev/aho-corasick v0.0.0-20240411101913-e07a1f0e8eb4 h1:1Kw2vDBXmjop+LclnzCb/fFy+sgb3gYARwfmoUcQe6o=
github.com/petar-dambovaliev/aho-corasick v0.0.0-20240411101913-e07a1f0e8eb4/go.mod h1:EHPiTAKtiFmrMldLUNswFwfZ2eJIYBHktdaUTZxYWRw=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/valllabh/ocsf-schema-golang v1.0.3 h1:eR8k/3jP/OOqB8LRCtdJ4U+vlgd/gk5y3KMXoodrsrw=
github.com/valllabh/ocsf-schema-golang v1.0.3/go.mod h1:sZ3as9xqm1SSK5feFWIR2CuGeGRhsM7TR1MbpBctzPk=
golang.org/x/mod v0.18.0 h1:5+9lSbEzPSdWkH32vYPBwEpX8KwDbM52Ud9xBUvNlb0=
golang.org/x/mod v0.18.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/tools v0.22.0 h1:gqSGLZqv+AI9lIQzniJ0nZDRG5GBPsSi+DRNHWNz6yA=
golang.org/x/tools v0.22.0/go.mod h1:aCwcsjqvq7Yqt6TNyX7QMU2enbQ/Gt0bo6krSeEri+c=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
rsc.io/binaryregexp v0.2.0 h1:HfqmD5MEmC0zvwBuF187nq9mdnXjXsSivRiXN7SmRkE=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
+264
View File
@@ -0,0 +1,264 @@
// Package engine 封装 coraza WAF,提供规则加载、热加载、规则列举与单条输入检测。
package engine
import (
"fmt"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
"github.com/corazawaf/coraza/v3"
"github.com/corazawaf/coraza/v3/types"
)
// Engine 持有当前生效的 WAF 实例,支持通过 Reload 热替换。
type Engine struct {
mu sync.RWMutex
waf coraza.WAF
cfg *Config
}
// Config 引擎配置。
type Config struct {
// RulesDir 规则根目录,主入口文件为 <RulesDir>/waf.conf
RulesDir string
// ErrorCallback 规则加载/解析错误回调
ErrorCallback func(mr types.MatchedRule)
}
// New 创建引擎并首次加载规则。
func New(cfg *Config) (*Engine, error) {
e := &Engine{cfg: cfg}
if err := e.Reload(); err != nil {
return nil, err
}
return e, nil
}
// Reload 重新加载规则文件,原子替换内部 WAF 实例。
// 加载顺序:
// 1. <RulesDir>/waf.conf 基础配置
// 2. <RulesDir>/local/*.conf 本地自定义规则(按文件名字母序)
// 3. <RulesDir>/crs/coraza.conf OWASP CRS 入口(若存在)
// 4. <RulesDir>/crs/rules/*.conf OWASP CRS 规则(按文件名字母序)
//
// 不使用 coraza 的 Include 指令(其路径相对于 cwd 解析易出错),
// 改为 engine 自己扫描目录逐个 WithDirectivesFromFile 加载。
// 失败时保留旧实例不动,确保服务可用。
func (e *Engine) Reload() error {
mainFile := filepath.Join(e.cfg.RulesDir, "waf.conf")
if _, err := os.Stat(mainFile); err != nil {
return fmt.Errorf("主规则文件不存在: %w", err)
}
builder := coraza.NewWAFConfig()
if e.cfg.ErrorCallback != nil {
builder = builder.WithErrorCallback(e.cfg.ErrorCallback)
}
// 1. 主配置
builder = builder.WithDirectivesFromFile(mainFile)
// 2. 本地规则(local/*.conf,按字母序)
localFiles, _ := filepath.Glob(filepath.Join(e.cfg.RulesDir, "local", "*.conf"))
for _, f := range localFiles {
builder = builder.WithDirectivesFromFile(f)
}
// 3. OWASP CRS(可选,crs/coraza.conf + crs/rules/*.conf)
crsEntry := filepath.Join(e.cfg.RulesDir, "crs", "coraza.conf")
if _, err := os.Stat(crsEntry); err == nil {
builder = builder.WithDirectivesFromFile(crsEntry)
crsRuleFiles, _ := filepath.Glob(filepath.Join(e.cfg.RulesDir, "crs", "rules", "*.conf"))
for _, f := range crsRuleFiles {
builder = builder.WithDirectivesFromFile(f)
}
}
newWAF, err := coraza.NewWAF(builder)
if err != nil {
return fmt.Errorf("规则加载失败: %w", err)
}
e.mu.Lock()
e.waf = newWAF
e.mu.Unlock()
return nil
}
// NewTransaction 创建一个事务,由调用方 Close。
func (e *Engine) NewTransaction() types.Transaction {
e.mu.RLock()
defer e.mu.RUnlock()
return e.waf.NewTransaction()
}
// CheckInput 用当前规则测试单个输入字符串,返回命中的规则 ID 列表。
// 极客 CLI 用:waf -check "/?id=1' UNION SELECT * from users"
// 输入按原始未编码字符串处理,内部对 query 做 URL 编码以符合 coraza ParseRequestURI 要求。
func (e *Engine) CheckInput(input string) []int {
tx := e.NewTransaction()
defer tx.Close()
uri := input
if !strings.HasPrefix(uri, "/") {
uri = "/" + uri
}
// 分离 path 与 query,对 query 做合法 URL 编码(空格等非法字符需转义)
// 这样 coraza ProcessURI 内部的 url.ParseRequestURI 不会失败,ARGS 才能被正确提取
fullURI := uri
if idx := strings.Index(uri, "?"); idx >= 0 {
path := uri[:idx]
rawQuery := uri[idx+1:]
// 用 net/url 编码 query:保留 & = ,其他特殊字符转义
var parts []string
for _, kv := range strings.Split(rawQuery, "&") {
eq := strings.Index(kv, "=")
if eq < 0 {
parts = append(parts, url.QueryEscape(kv))
} else {
parts = append(parts, url.QueryEscape(kv[:eq])+"="+url.QueryEscape(kv[eq+1:]))
}
}
fullURI = path + "?" + strings.Join(parts, "&")
}
// 照搬 coraza/http/middleware.go 的 processRequest 流程
tx.ProcessConnection("127.0.0.1", 8080, "127.0.0.1", 12345)
tx.ProcessURI(fullURI, "GET", "HTTP/1.1")
tx.AddRequestHeader("Host", "localhost")
tx.SetServerName("localhost")
if it := tx.ProcessRequestHeaders(); it != nil {
return []int{it.RuleID}
}
if it, _ := tx.ProcessRequestBody(); it != nil {
return []int{it.RuleID}
}
return nil
}
// ListRules 扫描规则目录,返回所有 SecRule 的 ID 与原始行。
// 仅做文本扫描,不解析语义——极客用 grep 友好。
// 处理 `\` 续行:以 `\` 结尾的行与下一行合并后再抽取 id。
func (e *Engine) ListRules() ([]Rule, error) {
var rules []Rule
err := filepath.Walk(e.cfg.RulesDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return err
}
if !strings.HasSuffix(path, ".conf") {
return nil
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
rel, _ := filepath.Rel(e.cfg.RulesDir, path)
// 先合并续行,保留每条逻辑行在原文件的起始行号
type logicalLine struct {
startLine int
text string
}
var lines []logicalLine
rawLines := strings.Split(string(data), "\n")
for i := 0; i < len(rawLines); i++ {
start := i + 1
cur := strings.TrimRight(rawLines[i], "\r")
// 合并以 \ 结尾的续行
for strings.HasSuffix(cur, "\\") && i+1 < len(rawLines) {
cur = cur[:len(cur)-1] + " " + strings.TrimRight(rawLines[i+1], "\r")
i++
}
lines = append(lines, logicalLine{startLine: start, text: cur})
}
for _, ll := range lines {
line := strings.TrimSpace(ll.text)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
if id, ok := extractRuleID(line); ok {
rules = append(rules, Rule{
ID: id,
File: rel,
Line: ll.startLine,
Source: line,
})
}
}
return nil
})
return rules, err
}
// Rule 描述一条 SecRule 的位置信息。
type Rule struct {
ID int `json:"id"`
File string `json:"file"`
Line int `json:"line"`
Source string `json:"source"`
}
// extractRuleID 从一行 SecRule 中抠出 id:nnnn 字段。
// 形如: SecRule REQUEST_URI "@rx (?i)union.*select" "id:100001,phase:1,deny"
func extractRuleID(line string) (int, bool) {
idx := strings.Index(line, "id:")
if idx < 0 {
return 0, false
}
rest := line[idx+3:]
end := 0
for end < len(rest) {
c := rest[end]
if c < '0' || c > '9' {
break
}
end++
}
if end == 0 {
return 0, false
}
var id int
fmt.Sscanf(rest[:end], "%d", &id)
return id, true
}
// TestFile 仅尝试加载单个规则文件,验证语法。
// 通过临时拼一个 Include 实现,目标文件转绝对路径以适配 coraza 解析目录。
func TestFile(file string) error {
abs, err := filepath.Abs(file)
if err != nil {
return err
}
if _, err := os.Stat(abs); err != nil {
return fmt.Errorf("文件不存在: %w", err)
}
tmp, err := os.CreateTemp("", "waf-test-*.conf")
if err != nil {
return err
}
defer os.Remove(tmp.Name())
_, err = tmp.WriteString(fmt.Sprintf("Include %s\n", abs))
tmp.Close()
if err != nil {
return err
}
builder := coraza.NewWAFConfig().WithDirectivesFromFile(tmp.Name())
_, err = coraza.NewWAF(builder)
if err != nil {
return fmt.Errorf("语法错误: %w", err)
}
return nil
}
// LogError 默认错误回调,输出到 stderr。
func LogError(mr types.MatchedRule) {
fmt.Fprintf(os.Stderr, "[coraza] rule_id=%d severity=%s msg=%s\n",
mr.Rule().ID(), mr.Rule().Severity(), mr.ErrorLog())
}
+156
View File
@@ -0,0 +1,156 @@
// Package proxy 封装到 gateway 的反向代理,并在请求/响应阶段接入 WAF 事务。
package proxy
import (
"bytes"
"encoding/json"
"io"
"log"
"net/http"
"net/http/httputil"
"net/url"
"strconv"
"strings"
"time"
"github.com/micro_servers/waf/internal/engine"
)
// Proxy 持有 ReverseProxy 与 WAF 引擎引用。
type Proxy struct {
rp *httputil.ReverseProxy
waf *engine.Engine
}
// New 创建代理。upstream 形如 "http://127.0.0.1:8443"。
func New(upstream string, waf *engine.Engine) (*Proxy, error) {
target, err := url.Parse(upstream)
if err != nil {
return nil, err
}
rp := httputil.NewSingleHostReverseProxy(target)
p := &Proxy{rp: rp, waf: waf}
// 自定义 Director:保留原 Host / X-Forwarded-For
origDirector := rp.Director
rp.Director = func(req *http.Request) {
origDirector(req)
req.Header.Set("X-Forwarded-For", req.RemoteAddr)
req.Header.Set("X-Forwarded-Proto", "https")
}
// 透传错误,不暴露内部堆栈
rp.ErrorHandler = func(w http.ResponseWriter, r *http.Request, err error) {
log.Printf("[proxy] upstream_error ip=%s path=%s err=%v", r.RemoteAddr, r.URL.Path, err)
http.Error(w, "upstream unavailable", http.StatusBadGateway)
}
return p, nil
}
// ServeHTTP 实现 http.Handler。
// 流程:ProcessRequest → 命中则 403 → 否则转发 → ProcessResponseHeaders。
func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
start := time.Now()
tx := p.waf.NewTransaction()
defer func() {
tx.ProcessLogging()
_ = tx.Close()
}()
// 解析 RemoteAddr 中的 IP 与端口
client := r.RemoteAddr
cport := 0
if idx := strings.LastIndexByte(r.RemoteAddr, ':'); idx != -1 {
client = r.RemoteAddr[:idx]
cport, _ = strconv.Atoi(r.RemoteAddr[idx+1:])
}
// 请求阶段:照搬 coraza/http/middleware.go 的 processRequest 流程
tx.ProcessConnection(client, cport, "", 0)
tx.ProcessURI(r.URL.String(), r.Method, r.Proto)
for k, vs := range r.Header {
for _, v := range vs {
tx.AddRequestHeader(k, v)
}
}
if r.Host != "" {
tx.AddRequestHeader("Host", r.Host)
tx.SetServerName(r.Host)
}
if it := tx.ProcessRequestHeaders(); it != nil {
p.block(w, r, it.RuleID, start)
return
}
// 请求体:读全量喂给 coraza,再用 buffer 恢复 r.Body 给 proxy 转发
if r.Body != nil && r.Body != http.NoBody {
bodyBytes, err := io.ReadAll(r.Body)
if err == nil {
_, _, _ = tx.ReadRequestBodyFrom(bytes.NewReader(bodyBytes))
r.Body = io.NopCloser(bytes.NewReader(bodyBytes))
}
if it, _ := tx.ProcessRequestBody(); it != nil {
p.block(w, r, it.RuleID, start)
return
}
}
// 包装 ResponseWriter 以捕获状态码与响应头
rw := &responseRecorder{ResponseWriter: w, status: 200}
p.rp.ServeHTTP(rw, r)
// 响应阶段:把 upstream 的 header 喂给 WAF
for k, vs := range rw.headers {
for _, v := range vs {
tx.AddResponseHeader(k, v)
}
}
tx.ProcessResponseHeaders(rw.status, "HTTP/1.1")
log.Printf("[proxy] pass ip=%s method=%s path=%s status=%d rule=- cost=%s",
r.RemoteAddr, r.Method, r.URL.Path, rw.status, time.Since(start))
}
// block 命中规则时统一返回 403,并打 JSON 日志便于 jq 分析。
func (p *Proxy) block(w http.ResponseWriter, r *http.Request, ruleID int, start time.Time) {
rec := map[string]any{
"ts": time.Now().UTC().Format(time.RFC3339),
"action": "block",
"rule_id": ruleID,
"ip": r.RemoteAddr,
"method": r.Method,
"path": r.URL.Path,
"query": r.URL.RawQuery,
"ua": r.UserAgent(),
"cost_ms": time.Since(start).Milliseconds(),
}
if raw, err := json.Marshal(rec); err == nil {
log.Println(string(raw))
}
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte("blocked by WAF\n"))
}
// responseRecorder 记录状态码与响应头,供响应阶段 WAF 检测使用。
type responseRecorder struct {
http.ResponseWriter
status int
headers http.Header
}
func (r *responseRecorder) WriteHeader(code int) {
r.status = code
if r.headers == nil {
r.headers = http.Header{}
}
// 拷贝上游响应头
for k, vs := range r.Header() {
r.headers[k] = vs
}
r.ResponseWriter.WriteHeader(code)
}
func (r *responseRecorder) Flush() {
if f, ok := r.ResponseWriter.(http.Flusher); ok {
f.Flush()
}
}
+141
View File
@@ -0,0 +1,141 @@
// Command waf 是 micro_servers 的 WAF 入口网关。
//
// 部署形态:客户端 ──HTTPS──▶ waf:443 ──明文 HTTP──▶ gateway:内部端口
// WAF 终止 TLS、运行 coraza 规则匹配,未命中则转发到 gateway。
//
// 极客操作:
//
// kill -HUP <pid> # 热加载规则
// waf -list # 列出所有规则
// waf -test rules/local/x.conf # 校验单文件语法
// waf -check "/?id=1' OR '1'='1" # 用当前规则测试输入
package main
import (
"crypto/tls"
"encoding/json"
"flag"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"gopkg.in/yaml.v3"
"github.com/micro_servers/waf/internal/engine"
"github.com/micro_servers/waf/internal/proxy"
)
// Config WAF 配置文件结构。
type Config struct {
Listen string `yaml:"listen"` // 监听地址,默认 :443
TLSCert string `yaml:"tls_cert"` // 证书文件
TLSKey string `yaml:"tls_key"` // 私钥文件
Upstream string `yaml:"upstream"` // gateway 地址,如 http://127.0.0.1:8443
RulesDir string `yaml:"rules_dir"` // 规则根目录
}
func main() {
var (
configFile = flag.String("c", "config.yaml", "配置文件路径")
doList = flag.Bool("list", false, "列出所有规则后退出")
doTest = flag.String("test", "", "校验单个规则文件语法后退出")
doCheck = flag.String("check", "", "用当前规则测试输入字符串(URI/path)后退出")
)
flag.Parse()
cfg := loadConfig(*configFile)
// 子命令:极客 CLI 工具
if *doTest != "" {
if err := engine.TestFile(*doTest); err != nil {
fmt.Fprintln(os.Stderr, "FAIL:", err)
os.Exit(1)
}
fmt.Println("OK")
return
}
eng, err := engine.New(&engine.Config{
RulesDir: cfg.RulesDir,
ErrorCallback: engine.LogError,
})
if err != nil {
log.Fatalf("引擎初始化失败: %v", err)
}
if *doList {
rules, err := eng.ListRules()
if err != nil {
log.Fatal(err)
}
out, _ := json.MarshalIndent(rules, "", " ")
fmt.Println(string(out))
return
}
if *doCheck != "" {
ids := eng.CheckInput(*doCheck)
fmt.Printf("input=%q hit_rules=%v\n", *doCheck, ids)
return
}
// 服务模式
p, err := proxy.New(cfg.Upstream, eng)
if err != nil {
log.Fatalf("代理初始化失败: %v", err)
}
// SIGHUP 热加载
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGHUP)
go func() {
for range sigCh {
log.Println("[reload] 收到 SIGHUP,重新加载规则")
if err := eng.Reload(); err != nil {
log.Printf("[reload] 失败,保留旧规则: %v", err)
} else {
log.Println("[reload] 成功")
}
}
}()
server := &http.Server{
Addr: cfg.Listen,
Handler: p,
TLSConfig: &tls.Config{
MinVersion: tls.VersionTLS12,
},
}
log.Printf("waf listening on %s, upstream=%s, rules=%s",
cfg.Listen, cfg.Upstream, cfg.RulesDir)
log.Printf(" - 热加载: kill -HUP %d", os.Getpid())
log.Printf(" - 列规则: %s -c %s -list", os.Args[0], *configFile)
if err := server.ListenAndServeTLS(cfg.TLSCert, cfg.TLSKey); err != nil {
log.Fatal(err)
}
}
func loadConfig(path string) *Config {
data, err := os.ReadFile(path)
if err != nil {
log.Fatalf("读取配置失败: %v", err)
}
var c Config
if err := yaml.Unmarshal(data, &c); err != nil {
log.Fatalf("解析配置失败: %v", err)
}
if c.Listen == "" {
c.Listen = ":443"
}
if c.RulesDir == "" {
c.RulesDir = "./rules"
}
if c.Upstream == "" {
log.Fatal("upstream 不能为空")
}
return &c
}
+26
View File
@@ -0,0 +1,26 @@
# custom.conf —— 本地自定义规则示例
# ID 段:100000-199999
# 100001: 拦截 SQL 注入特征(UNION SELECT)
SecRule REQUEST_URI|ARGS "@rx (?i)union[\s/*]+select" \
"id:100001,phase:1,deny,status:403,log,msg:'SQL injection: UNION SELECT'"
# 100002: 拦截 SQL 注释符
SecRule ARGS "@rx (?i)(--|#|/\*.*\*/)" \
"id:100002,phase:2,deny,status:403,log,msg:'SQL injection: comment'"
# 100003: 拦截 XSS 标签
SecRule ARGS|REQUEST_HEADERS "@rx (?i)<script|javascript:|onerror=|onload=" \
"id:100003,phase:2,deny,status:403,log,msg:'XSS probe'"
# 100004: 路径遍历
SecRule REQUEST_URI|ARGS "@rx (?i)\.\./|\.\.%2f|\.\.%5c" \
"id:100004,phase:1,deny,status:403,log,msg:'Path traversal'"
# 100005: 命令注入常见关键字
SecRule ARGS "@rx (?i)(;|\|)&?\s*(cat|ls|id|whoami|uname|wget|curl|bash|sh)\s" \
"id:100005,phase:2,deny,status:403,log,msg:'Command injection'"
# 100006: 已知恶意扫描器 UA
SecRule REQUEST_HEADERS:User-Agent "@rx (?i)(nikto|sqlmap|nmap|masscan|acunetix|nessus)" \
"id:100006,phase:1,deny,status:403,log,msg:'Malicious scanner UA'"
+6
View File
@@ -0,0 +1,6 @@
# exceptions.conf —— 误报白名单
# 当某条 CRS 规则误报时,在此用 SecRuleRemoveById 或 ctl:ruleRemoveById 关闭
#
# 示例:关闭 100003 对 /login 路径的检查
# SecRule REQUEST_URI "@beginsWith /login" \
# "id:100100,phase:1,pass,nolog,ctl:ruleRemoveById=100003"
+22
View File
@@ -0,0 +1,22 @@
# waf.conf —— WAF 规则主入口(仅放基础配置)
# 极客操作:
# - 在 local/ 下新增 *.conf 文件,engine 会自动扫描加载(按文件名字母序)
# - 修改后执行 `kill -HUP <pid>` 热加载
# - 校验语法: `waf -test rules/local/xxx.conf`
#
# 规则 ID 分段约定:
# 100000-199999 本地自定义规则
# 900000-999999 OWASP CRS(由 crs/ 提供,不手改)
#
# 注意:本文件不用 Include 指令(coraza 的 Include 路径相对于 cwd 解析,
# 易出错)。engine.Reload 会先加载本文件,再扫描 local/*.conf 逐个加载。
# ===== 基础配置(SecLang) =====
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess On
SecDefaultAction "phase:1,pass,log,auditlog"
# 请求体限制 10MB
SecRequestBodyLimit 10485760