RpcClient.call() returns the data dict, not a raw string.
configparser.read_string() expects str, hence TypeError.
Now uses .get("content") to unwrap the ini text.
Pure Python stdlib http.server + ThreadingMixIn, zero web framework deps.
- GET/HEAD → 301, POST/PUT/DELETE/PATCH → 308
- Reads Host header, strips port, redirects to https://{host}{path}
- Config via confs (retry_on_refused, hard dependency) + env var override
- Logs via log_client → db Logs table (fail-safe async batch)
- Port 80 privilege via setcap cap_net_bind_service in _run.sh
Follow the same convention as the static module: remove local config.ini
fallback and --config/-i command-line argument entirely. All configuration
is now managed exclusively through confs service.
- Delete gateway/example.ini (moved to confs/configs/gateway.ini).
- Remove -i/--input option from boost::program_options; help (-h) and
version (-v) are the only CLI flags remaining.
- Simplify main.cpp config loading: remove local-file branch, always fetch
gateway.ini from confs with retry_on_refused=true waiting for confs ready.
- Update _run.sh: no longer passes any arguments; config comes from confs.
- Update README: remove example.ini references, -i flag docs, local file
mode description, and example.ini from directory structure tree.
- Clarify changelog entry for v0.7.0.
Migrate gateway from local -i ini file to the unified confs configuration
service, following the same two-layer pattern as Python services (dnfs/db/static):
- Add confs/configs/gateway.ini with all existing settings plus new
[Downstream] section for db/captcha/smtp host:port (previously hardcoded).
- Conf.hpp: add fromString() to parse ini from text (confs response),
env()/env_int() static helpers for environment variables, public default
constructor, and empty-filename guard in file constructor.
- DbC.hpp: add init(host, port) static method and static _host/_port members,
replacing hardcoded "127.0.0.1":8005. Defaults preserved for backward compat.
- main.cpp:
* Rewrite config loading: bootstrap CONFS_HOST/CONFS_PORT from env
(default 127.0.0.1:8010), fetch gateway.ini via pymsg from confs
(retry_on_refused=true), parse with ConfIni::fromString().
* -i flag is now optional; when provided, reads local ini (dev/debug mode).
* Add downstream host/port globals (_DB_HOST/_DB_PORT, _CAPTCHA_HOST/_CAPTCHA_PORT,
_SMTP_HOST/_SMTP_PORT, _CONFS_HOST/_CONFS_PORT), populated from [Downstream].
* Environment variable overrides (highest priority): LISTEN_ADDR/LISTEN_PORT,
DB_HOST/DB_PORT, CAPTCHA_HOST/CAPTCHA_PORT, SMTP_HOST/SMTP_PORT.
* Config integrity validation: required fields (Server.address/port,
Proxy.default, Downstream.db_host/db_port) cause startup failure if missing.
* Initialize DbC::init() and logs().set_endpoint() after config load.
* Replace all hardcoded "127.0.0.1":8005/8008/8009 in captcha/smtp/db calls
with configurable globals.
* Add downstream service info to startup log output.
* Fix: -v/--version flag now actually prints version and exits.
* Bump version to 0.7.0.
- _run.sh: remove -i example.ini default; now starts without args (confs mode),
passes "$@" through for optional -i override.
- Update README with new config loading docs, Downstream config section,
updated directory structure, and v0.7.0 changelog.
- Replace unsafe base64-only decode_token() with HTTP POST to gateway /whoami,
which performs RSA signature verification and expiry checks internally
- Fail-closed on gateway unreachable: return 503 instead of allowing access
- Add [gateway] config section (url/timeout) with GATEWAY_URL env override
- Add missing @_handle_exp decorator to delete_note and update_note views
- Fix Flask endpoint name collision using @functools.wraps on _handle_exp
- Remove unused base64 import
- Update dnfs.ini with gateway config (http://127.0.0.1:8443/whoami, timeout=5)
- Update docker-compose.yml with GATEWAY_URL and host.docker.internal for containers
Replace the insecure local base64 decode (with FIXME comment about
missing signature verification) with proper remote RSA signature
verification through gateway's /whoami endpoint:
- Add [gateway] config section (url, timeout) to dnfs.ini with defaults.
- Support GATEWAY_URL environment variable override for Docker.
- decode_token() now POSTs to gateway /whoami with Authorization: Bearer
<token>; gateway verifies RSA signature and expiry, returns login/user/
is_admin. Network/gateway errors fail-closed with 503.
- Add @_handle_exp decorators to delete_note() and update_note() which
were missing exception handling wrappers (consistent with other routes).
- docker-compose.yml: add GATEWAY_URL, CONFS_HOST, CONFS_PORT env vars
plus host.docker.internal extra_hosts so dnfs container can reach
host-side gateway and confs services.
- Add MsgConnectionRefusedException to distinguish ECONNREFUSED from other
connection errors; openConnection() now throws it when connect() returns
ECONNREFUSED.
- MsgClient::call() adds retry_on_refused/retry_interval parameters (defaults
false/1.0s), matching Python pymsg.MsgClient.call(retry_on_refused=True).
- Retry loop: only retries on ECONNREFUSED; other exceptions (timeout,
connection reset, business errors) propagate immediately. Throttled logging:
first failure prints immediately, subsequent logs every 5 seconds.
- DbC::call_msg() enables retry_on_refused=true (db is a hard dependency;
gateway blocks waiting for db on startup instead of crashing).
- LogC::_send() explicitly passes retry_on_refused=false (fail-safe; never
block the background log thread when db is unavailable).
- captcha/smtp auxiliary services keep default retry_on_refused=false
(fail-fast; blocking in request threads would exhaust the thread pool).
- Update README with pymsg protocol documentation, downstream retry strategy
table, and v0.6.1 changelog entry.
- index.html: lang 改为 zh-CN，新增"关于本站"介绍段落，按钮改为有 id 的登录/注册/进入/退出四按钮（默认隐藏进入/退出）
- 新增 index.js: POST /whoami 检测登录态（HttpOnly Cookie 自动携带），已登录则显示用户名+进入笔记/退出按钮，未登录显示登录/注册
- 退出按钮调用 POST /logout 后刷新页面
Merge WebSocket client functionality into HttpClient using Boost.Beast,
enabling a single client instance to handle both HTTP (libcurl) and
WebSocket (ws:// and wss://) protocols. Backward-compatible type aliases
(WebSocketClient, WebSocketClientBase) preserve existing code.
Changes:
- Add synchronous WebSocket client: ws_connect/ws_send/ws_recv/ws_close
- Support ws:// (plain TCP) and wss:// (TLS with SNI) connections
- Layered stream architecture with beast::tcp_stream for timeout support
- Binary message support via ws_send_binary()
- Update Makefile to link libcurl (-lcurl)
- Add README.md documenting unified HttpServer + HttpClient API
- Add _ai_prefab flag to track pre-created empty AI widget
- start_reasoning() now reorders widgets: remove empty AI widget, mount reasoning first, then remount AI widget
- This ensures reasoning always appears BEFORE the actual answer
- Filter out 'Conversation started'/'对话开始' info notifications (implicit event, no need to show)
- Increase reasoning display limit from 2000 to 3000 chars
Merge WebSocketServer functionality into HttpServer so a single port can
serve both HTTP(S) and WebSocket(S) endpoints while preserving full backward
compatibility for existing HTTP interfaces.
Key changes:
- Add WebSocket route registration: ws(), onWsConnect(), onWsDisconnect()
- Add client management: sendToWsClient(), broadcastWs(), disconnectWsClient()
- Rewrite Session class using beast::tcp_stream for unified HTTP/WS handling
- Support automatic ws:// and wss:// upgrades on HTTP/HTTPS ports
- Thread-safe message queue with asio::post for cross-thread sends
- Update Makefile to link OpenSSL and pthread for TLS/WebSocket support
- Add test_server.cpp demonstrating HTTP + WebSocket mixed usage
- Implement http.Hijacker on responseRecorder to enable WebSocket upgrade
(delegates Hijack() to underlying ResponseWriter, marks hijacked state)
- After WebSocket upgrade (101 Switching Protocols), skip HTTP response-phase
WAF checks since connection becomes a binary frame stream
- Log WebSocket upgrades at Info level for observability
- Fix P2-3: use r.Proto instead of hardcoded "HTTP/1.1" in ProcessResponseHeaders
for correct HTTP/2 protocol version reporting
- Add Write() method to responseRecorder to ensure WriteHeader is called
before first Write (complies with http.ResponseWriter contract)
- Add bufio/net imports required by Hijacker interface
- Update README with WebSocket support section and v1.2 changelog
- Update SECURITY_AUDIT to mark P2-3 and P2-4 as fixed
- Add dnfs.ini to confs/configs/ with server, database, ai_review, confs sections
- Refactor dnfs.py to use three-layer fail-safe config loading (defaults → confs → env vars)
- Unify timeouts for DB RPC and AI review calls via configuration
- Add _merge_ini() whitelist merging to prevent unexpected config injection
- Environment variables have highest priority for Docker networking (db/ai_review hostnames)
- Document config loading mechanism in both confs/README.md and dnfs/README.md
Follows the same pattern as static service integration; confs failure falls back
gracefully to built-in defaults + environment variables.
- P0: add 10MB hard request body size limit with io.LimitReader,
return 413 Request Entity Too Large when exceeded, prevents OOM
- P0: configure http.Server timeouts: ReadHeaderTimeout=10s (Slowloris),
ReadTimeout=15s, WriteTimeout=30s, IdleTimeout=120s, MaxHeaderBytes=1MB
- P1: properly handle body read errors, log and return on client disconnect
- P2: drain and close r.Body in defer to ensure TCP connection reuse
- Log errors from ReadRequestBodyFrom and ProcessRequestBody
- Inject security headers on 413 responses
- Add security audit report (docs/SECURITY_AUDIT_20260801.md)
- Update README with timeout/body-limit docs and changelog
- ReadFileTool: replace readlines() with on-demand range reading; add reverse block reading for large file tail; limit single line to 10KB; hard-cap max_lines to 500
- SubprocessCommandExecutor: replace capture_output with line-by-line Popen reading; enforce 1MB output limit with immediate process termination; merge stdout/stderr counting for both sync and streaming modes
- Raise TOOL_MAX_RESULT_LENGTH default from 10KB to 1MB as final safety net
- Add SECURITY_AUDIT.md documenting vulnerabilities and remediation roadmap
- Update README with memory safety features and v2.3 changelog
P1 fixes:
- P1-1 _handle_reload: empty string/0/false/falsy name no longer clears all
caches; only explicit null/omitted name triggers full cache clear; non-string
name raises ConfigError instead of being silently str()-converted
- P1-2 ConfigStore.get(): return dict(entry) shallow copy instead of direct
cache reference to prevent callers from mutating the internal cache
- P1-3 list(): resolve symlinks via os.path.realpath() and skip entries whose
real path escapes configs/ root; prevents leaking external file metadata
(size, mtime) through directory-traversing symlinks
- P1-4 _handle_get key: validate key is a string (isinstance check); reject
non-string/empty key with clear ConfigError instead of misleading str()
conversion leading to "key not found" errors
P2 fixes:
- P2-1 ConfigParser: use interpolation=None to disable %(key)s interpolation,
preventing InterpolationError on config values containing bare % characters
- P2-2 argparse: update description from "pymsg/TCP" to "pymsg RpcServer/TCP"
to match module docstring
Also update SECURITY_AUDIT_20260801.md to mark issues 9-14 as fixed and
refresh the security mechanisms summary table; all P0/P1 issues are now resolved.
- Split _resolve() into _validate_name() (pure string checks, lock-free
fast-fail) and _resolve_locked() (filesystem checks, must be called
under lock)
- Perform all filesystem operations (realpath → isfile → stat → open)
inside self._lock in get(), eliminating the window between path
validation and file open where a symlink swap could escape the
configs directory
- Apply same fix to reload(): validate name outside lock, resolve path
inside lock to prevent TOCTOU during targeted reload
- Hold lock during list() directory traversal so the `cached` flag
(which reads self._cache) is consistent with directory state; wrap
os.stat() in try/except to tolerate files disappearing mid-traversal
- Update docstring and README:
- Protocol description corrected to pymsg RpcServer/TCP
- Python client examples updated from legacy MsgClient to RpcClient
- Security section now documents TOCTOU mitigation and Slowloris
protection (added by earlier pymsg fix)
- Replace legacy MsgClient (low-level TCP frames) with RpcClient (high-level
RPC) to match dbv2's RpcServer interface; the old MsgClient-based version
was incompatible with the current RPC routing protocol
- Apply all hardening fixes from the service copies:
- Outer try/except in _flush_loop to prevent any exception from killing
the background thread
- _safe_json() helper for unserializable extra fields (fallback to str/repr)
- Timestamp generated at push() (enqueue time) instead of _send() time
- Event-based flush() synchronization instead of unreliable sleep(0.6)
- Queue maxsize=10000 with drop-oldest policy to prevent unbounded memory
- put_nowait with Full handling ensures push() never blocks business threads
- RpcClient construction wrapped in try/except
- Add explicit service_name parameter example for all 4 services in comments
- This is the template file referenced by db/README.md and copied by new
services; keeping it in sync prevents propagating old bugs
- Wrap _flush_loop main body in outer try/except to prevent any
unexpected exception (network errors, serialization errors, etc.)
from killing the background thread (which would cause unbounded
queue growth and eventual OOM)
- Wrap _send() call in inner try/except as final safety net
- Wrap RpcClient() construction in try/except (connection failures
during client creation were previously unhandled)
- Add _safe_json() helper: unserializable extra fields fall back to
repr()/str() instead of raising TypeError that could kill thread
- Generate timestamp at push() (enqueue time) instead of _send() time,
so timestamps reflect actual event time even when queue is backed up
- Replace unreliable time.sleep(0.6) in flush() with _flush_done Event
for proper synchronization between caller and background thread
- Add _QUEUE_MAXSIZE=10000 with drop-oldest policy to prevent unbounded
memory growth even in pathological cases
- Use put_nowait with Full handling to ensure push() never blocks
business threads
- Sync fix to all 4 copies: confs/static/smtp/dnfs
Prompt system bugs and improvements:
1. **SKILL_PREFIX bug fixed**: SKILL_PREFIX = "1_" only matched files starting
with "1_", causing 2_ through 10_ skill prompt files to be misclassified
as "other" files and sorted in wrong order (e.g., 10_task_handling.md
sorted before 2_memory_save.md due to lexicographic ordering). Replaced
with regex _SKILL_FILE_RE = re.compile(r'^(\d+)_') and _skill_sort_key()
that extracts the numeric prefix for proper numeric sorting.
2. **_init_system_prompts used unsorted list**: get_prompts_list() returned
prompts in os.listdir() lexicographic order, which put 100_STATUS.md
before 1_memory_search.md and 10_task_handling.md before 1_memory_search.md.
Now uses get_sorted_prompts_list() for correct ordering:
identity → skill prompts (1-10 by number) → soul → status.
3. **Verbose pre_prompt injected every turn (~30 lines)**: The pre_prompt
contained detailed workflow instructions (search_skills first, memory
search, ask_user, etc.) that duplicated what system skill prompts
already say, wasting ~300+ tokens per user message. Worse, the same
verbose prompt was injected on recursive think_and_respond calls after
tool execution, confusing the model with repeated "第一步搜索技能"
instructions when it was already mid-conversation with tool results.
- Reduced pre_prompt to 5 concise bullet points as a reminder
- Added is_continuation parameter: recursive calls after tool execution
skip pre_prompt entirely and just pass "请基于以上工具执行结果继续回答。"
- Removed post_prompt (save_memory/update_soul instructions already in
2_memory_save.md and 8_identity_management.md)
4. **Unreasonable large-file thresholds**: 3_file_operations.md default
said >50KB or >200 lines is "too large" — too conservative for code
files. Raised to 500KB/2000 lines; added tail=true and binary file
guidance.
5. **Overly rigid memory search instruction**: 1_memory_search.md said
"每次都要做！" (must do every time), causing unnecessary memory searches
for simple greetings/acknowledgments. Softened to "根据话题相关性判断",
with explicit examples of when to skip (greetings, confirmations).
New/changed methods:
- PromptManager._skill_sort_key(cls, filename) - extract numeric prefix
- PromptManager._prompt_sort_key(self, p) - tuple-based sort key
- PromptManager.get_sorted_prompts_list() - returns correctly ordered list
- PromptManager.get_skill_prompts() - uses regex instead of startswith("1_")
- DeepSeekChat.think_and_respond() - added is_continuation parameter
- Add MAX_CONFIG_SIZE constant (1 MiB) to limit single config file size
- Check file size via os.stat() before reading into memory
- Oversized files raise ConfigError with descriptive message instead of
being fully read (which could cause OOM with multi-GB files)
- Use single os.stat() call to get both mtime and size atomically
- Update README: document size limit in security mechanisms and notes
- Existing small configs (ini/json/yaml, typically <10KB) are unaffected
- Add configurable recv_timeout (default 10s) to MsgServer
- _handle_client now uses recv(s, timeout=...) instead of infinite blocking recv(s)
- Catch TimeoutError and close the slow client connection
- This prevents a single stalled connection from blocking the entire main select loop
- Update README with recv_timeout parameter and Slowloris mitigation docs
- Backward compatible: keyword-only arg with sensible default
- Fix all references from main.py to aibox.py (actual entry file)
- Update installation to use pip install -r requirements.txt
- Expand WebSocket message types table: add line/task_error/task_added/
notification/connection_ack/interrupt_ack, correct chunk/complete/error semantics
- Fix message flow example: tool calls are reported via 'line' messages,
not a non-existent 'tool_call' type
- Document _sanitize_tool_result() and error recovery (_rollback_messages)
in development guide
- Add FAQ entry for tool-call-induced API failures
- Add v2.1 changelog covering all robustness fixes (reasoning_content
filtering, tool_call id fallback, control character sanitization,
message rollback on API failure)
Root causes of API request failures after tool calls:
1. reasoning_content (an API response field) was echoed back in assistant
messages, which could cause API format errors
2. tool_call id could be empty string when model didn't return one in a
streaming chunk, producing invalid tool_calls that caused 400 errors
3. Tool execute() could return None or non-string types; direct string
operations (len(), + "\n", [:200]) raised TypeError and produced
non-string content in tool messages, breaking JSON serialization
4. Tool output (e.g. from run_command) could contain control characters
(NULL/BEL/BS) that corrupted JSON payloads
5. When API request failed (network/4xx/5xx/parse error), messages added
in that turn (user msg, assistant msg with tool_calls, tool responses)
remained in self.messages, permanently poisoning conversation history
so all subsequent requests would also fail
Fixes:
- Stop sending reasoning_content back to API in message construction;
normalize assistant message content: null when tool_calls present, ""
when neither content nor tool_calls exist, str otherwise
- Sanitize tool messages: ensure tool_call_id is non-empty (fallback id)
and content is always a string
- Generate fallback tool_call id when model omits it; handle streaming id
delivery where id arrives in a later chunk
- Add _sanitize_tool_result(): coerce None/non-string to str, strip
control characters (preserve \n \t \r) to prevent JSON/API corruption
- Wrap execute_tool call in try/except in _handle_tool_calls for double
safety; ensure history logging never crashes
- Truncate oversized results with an explicit note instead of replacing
with a generic error that lost all useful output
- Add _rollback_messages(to_length) and record msgs_before_turn at start
of think_and_respond; on any exception (RequestException, JSONDecodeError,
or unexpected Exception), rollback self.messages to the pre-turn state
so the conversation is not left in a broken state
- In _handle_tool_calls, wrap execute_tool in try/except and always
produce a safe string result
Root cause: aibox.py think_and_respond recursively calls itself after tool
execution, and each recursive call's complete callback carries only the
final-round full_response. The old finish_ai logic treated complete as a
full-text alignment signal — when full_text != self._ai_text (which always
happened after recursive tool calls because _ai_text accumulated all chunks
while complete only held the last round), it triggered remove+remount,
causing visible duplicate/flash of AI responses. A secondary issue was
dialogue_ended arriving before complete, which made a second finish_ai call
remount the message a second time.
Changes:
- Remove the full-text replacement branch in finish_ai; chunk streaming is
now the authoritative content source, complete is a finalization signal only
- Add _ai_finalized flag: start_ai resets it, finish_ai sets it; repeated
finish_ai calls (dialogue_ended + complete) return early to prevent remount
- append_ai drops late chunks after finalization to avoid dirty appends
- Empty pre-built widget without chunks still gets filled via w.update(full_text)
- Full lost-frame fallback (no widget + full_text) is preserved
- Update BotOps/README.md protocol section to reflect new complete semantics
qing