- ReadFileTool: replace readlines() with on-demand range reading; add reverse block reading for large file tail; limit single line to 10KB; hard-cap max_lines to 500
- SubprocessCommandExecutor: replace capture_output with line-by-line Popen reading; enforce 1MB output limit with immediate process termination; merge stdout/stderr counting for both sync and streaming modes
- Raise TOOL_MAX_RESULT_LENGTH default from 10KB to 1MB as final safety net
- Add SECURITY_AUDIT.md documenting vulnerabilities and remediation roadmap
- Update README with memory safety features and v2.3 changelog
Prompt system bugs and improvements:
1. **SKILL_PREFIX bug fixed**: SKILL_PREFIX = "1_" only matched files starting
with "1_", causing 2_ through 10_ skill prompt files to be misclassified
as "other" files and sorted in wrong order (e.g., 10_task_handling.md
sorted before 2_memory_save.md due to lexicographic ordering). Replaced
with regex _SKILL_FILE_RE = re.compile(r'^(\d+)_') and _skill_sort_key()
that extracts the numeric prefix for proper numeric sorting.
2. **_init_system_prompts used unsorted list**: get_prompts_list() returned
prompts in os.listdir() lexicographic order, which put 100_STATUS.md
before 1_memory_search.md and 10_task_handling.md before 1_memory_search.md.
Now uses get_sorted_prompts_list() for correct ordering:
identity → skill prompts (1-10 by number) → soul → status.
3. **Verbose pre_prompt injected every turn (~30 lines)**: The pre_prompt
contained detailed workflow instructions (search_skills first, memory
search, ask_user, etc.) that duplicated what system skill prompts
already say, wasting ~300+ tokens per user message. Worse, the same
verbose prompt was injected on recursive think_and_respond calls after
tool execution, confusing the model with repeated "第一步搜索技能"
instructions when it was already mid-conversation with tool results.
- Reduced pre_prompt to 5 concise bullet points as a reminder
- Added is_continuation parameter: recursive calls after tool execution
skip pre_prompt entirely and just pass "请基于以上工具执行结果继续回答。"
- Removed post_prompt (save_memory/update_soul instructions already in
2_memory_save.md and 8_identity_management.md)
4. **Unreasonable large-file thresholds**: 3_file_operations.md default
said >50KB or >200 lines is "too large" — too conservative for code
files. Raised to 500KB/2000 lines; added tail=true and binary file
guidance.
5. **Overly rigid memory search instruction**: 1_memory_search.md said
"每次都要做！" (must do every time), causing unnecessary memory searches
for simple greetings/acknowledgments. Softened to "根据话题相关性判断",
with explicit examples of when to skip (greetings, confirmations).
New/changed methods:
- PromptManager._skill_sort_key(cls, filename) - extract numeric prefix
- PromptManager._prompt_sort_key(self, p) - tuple-based sort key
- PromptManager.get_sorted_prompts_list() - returns correctly ordered list
- PromptManager.get_skill_prompts() - uses regex instead of startswith("1_")
- DeepSeekChat.think_and_respond() - added is_continuation parameter
Root causes of API request failures after tool calls:
1. reasoning_content (an API response field) was echoed back in assistant
messages, which could cause API format errors
2. tool_call id could be empty string when model didn't return one in a
streaming chunk, producing invalid tool_calls that caused 400 errors
3. Tool execute() could return None or non-string types; direct string
operations (len(), + "\n", [:200]) raised TypeError and produced
non-string content in tool messages, breaking JSON serialization
4. Tool output (e.g. from run_command) could contain control characters
(NULL/BEL/BS) that corrupted JSON payloads
5. When API request failed (network/4xx/5xx/parse error), messages added
in that turn (user msg, assistant msg with tool_calls, tool responses)
remained in self.messages, permanently poisoning conversation history
so all subsequent requests would also fail
Fixes:
- Stop sending reasoning_content back to API in message construction;
normalize assistant message content: null when tool_calls present, ""
when neither content nor tool_calls exist, str otherwise
- Sanitize tool messages: ensure tool_call_id is non-empty (fallback id)
and content is always a string
- Generate fallback tool_call id when model omits it; handle streaming id
delivery where id arrives in a later chunk
- Add _sanitize_tool_result(): coerce None/non-string to str, strip
control characters (preserve \n \t \r) to prevent JSON/API corruption
- Wrap execute_tool call in try/except in _handle_tool_calls for double
safety; ensure history logging never crashes
- Truncate oversized results with an explicit note instead of replacing
with a generic error that lost all useful output
- Add _rollback_messages(to_length) and record msgs_before_turn at start
of think_and_respond; on any exception (RequestException, JSONDecodeError,
or unexpected Exception), rollback self.messages to the pre-turn state
so the conversation is not left in a broken state
- In _handle_tool_calls, wrap execute_tool in try/except and always
produce a safe string result
qing