fix(aibox): Batch N 波次3 平台兼容(T3/T11/T25/T26/T27)

- T3 basic_tools 平台感知引用与 Windows 指引:_quote_arg 按 os.name
  分支(Windows 双引号包裹,拒绝双引号/换行/%变量名% 注入形态);
  ReadFile/WriteFile/ListFiles/Web 四工具 Windows 走 PowerShell/
  curl.exe 等价命令(WriteFile 用 Base64 规避 cmd 换行/转义限制)
- T11 executors 与 process_manager 的 _terminate_tree Windows 分支
  改 taskkill /T /F 递归收割进程树,兜底单进程 kill
- T25 MemoryDatabase 统一 _connect 工厂(busy_timeout=10000),
  7 处直连替换,对齐 MemoDatabase J9
- T26 SkillManager._load_skills 原子替换 self.skills(对齐
  PromptManager M6),reload 期间不再读到空/半截技能表
- T27 atomic_write_text 对 PermissionError 做 3 次退避重试

测试:新增 tests/test_n_fixes.py 30 项;全量 566 passed / 1 failed
(Windows 向 ping 用例,环境项),零回归。审计报告补记 §24.5/§24.6。
This commit is contained in:
qing committed 2026-08-21 13:56:33 +00:00
1 parent 5894e64287
commit 2dc79f0637
8 files changed
+617 -34

No files matched your search

+18 -1
View File
@@ -10,6 +10,7 @@ mkstemp 随机文件名避免多进程并发写同路径互踩(conversation_hi
import os
import tempfile
import time
def atomic_write_text(path: str, content: str) -> None:
@@ -23,10 +24,26 @@ def atomic_write_text(path: str, content: str) -> None:
f.write(content)
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, path)
# T27(审计 2026-08-21):Windows 下目标被其他进程打开(杀毒扫描/
# 编辑器/并发读者)时 os.replace 抛 PermissionError。POSIX 无条件
# 成功不受影响;Windows 做短退避重试,高频写路径(每条消息都原子
# 写 current_conversation.json)一次冲突不再直接丢该条持久化。
last_exc = None
for attempt in range(3):
try:
os.replace(tmp_path, path)
last_exc = None
break
except PermissionError as e:
last_exc = e
if attempt < 2:
time.sleep(0.05 * (attempt + 1))
if last_exc is not None:
raise last_exc
except BaseException:
try:
os.unlink(tmp_path)
except OSError:
pass
raise
+16 -7
View File
@@ -33,9 +33,18 @@ class MemoryDatabase:
self.logger = Logger("MemoryDB")
self._init_database()
def _connect(self) -> sqlite3.Connection:
"""统一连接工厂(T25,审计 2026-08-21):对齐 MemoDatabase J9 修复——
busy_timeout 覆盖全部路径。两类默认共用同一 memories.db,此前
MemoryDatabase 直连只靠 sqlite3 默认 5s 等待,CLI 与服务进程并发
写时锁冲突即抛 database is locked(记忆保存/检索失败)。"""
conn = sqlite3.connect(self.db_path)
conn.execute("PRAGMA busy_timeout = 10000")
return conn
def _init_database(self):
"""初始化数据库表"""
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
# WAL 模式:读写并发不互斥,缓解多 CLI 进程并发写锁等待
conn.execute("PRAGMA journal_mode=WAL")
cursor = conn.cursor()
@@ -118,7 +127,7 @@ class MemoryDatabase:
tags_str = ",".join(tags) if tags else ""
created_at = datetime.now().strftime(constants.DATETIME_FORMAT)
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
cursor = conn.cursor()
cursor.execute('''
INSERT INTO memories (topic, summary, content, start_time, end_time, created_at, tags, importance)
@@ -139,7 +148,7 @@ class MemoryDatabase:
def delete_memory(self, memory_id: int) -> bool:
"""删除记忆"""
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
cursor = conn.cursor()
# J3(审计 2026-08-19):必须先删 fts 再删内容表——外部内容表
# (content=memories)删除索引条目需从内容表回读旧值,内容行
@@ -166,7 +175,7 @@ class MemoryDatabase:
if limit is None:
limit = constants.MEMORY_MAX_SEARCH_RESULTS
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
@@ -240,7 +249,7 @@ class MemoryDatabase:
的 None→默认上限语义对齐)。"""
if limit is None:
limit = constants.MEMORY_MAX_SEARCH_RESULTS
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
cursor.execute('SELECT * FROM memories ORDER BY end_time DESC LIMIT ?', (limit,))
@@ -256,7 +265,7 @@ class MemoryDatabase:
def get_memory_by_id(self, memory_id: int) -> Optional[Dict]:
"""根据ID获取记忆"""
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
cursor.execute('SELECT * FROM memories WHERE id = ?', (memory_id,))
@@ -274,7 +283,7 @@ class MemoryDatabase:
if not memory_ids:
return 0
with sqlite3.connect(self.db_path) as conn:
with self._connect() as conn:
cursor = conn.cursor()
placeholders = ','.join(['?'] * len(memory_ids))
# J3:同 delete_memory——先删 fts 再删内容表,防索引条目残留
+19 -4
View File
@@ -115,7 +115,12 @@ class SubprocessCommandExecutor(CommandExecutor):
ProcessLookupError,届时无法再从进程反查组。
T5:pgid 由调用方显式传入(注册槽位时捕获),不再读共享
_current_pgid——并发执行时那是别人的进程组。
Windows 无进程组概念,退回单进程 terminate/kill。"""
T11(审计 2026-08-21):Windows 无进程组概念,单进程 terminate/
kill 收割不到 shell=True 生成的 cmd.exe /c 之下 spawn 的孙进程
(超时/取消后孙进程成孤儿,持续占用 CPU/端口/文件锁)。用
taskkill /T 递归终止整棵进程树;Windows 无 SIGTERM 等价物,
taskkill 不带 /F 仅对有消息窗口的 GUI 进程有效,控制台进程
杀不掉,故统一 /F 强制,force 参数在 Windows 上不区分。"""
try:
if os.name == "posix":
sig = signal.SIGKILL if force else signal.SIGTERM
@@ -128,10 +133,20 @@ class SubprocessCommandExecutor(CommandExecutor):
os.killpg(pgid, sig)
else:
proc.kill() if force else proc.terminate()
elif force:
proc.kill()
else:
proc.terminate()
# T11:Windows 树收割——taskkill /T /F 递归终止子孙进程,
# 失败(进程已退出/无权限)时兜底单进程 kill
try:
subprocess.run(
["taskkill", "/PID", str(proc.pid), "/T", "/F"],
capture_output=True, timeout=10)
except Exception:
pass
try:
if proc.poll() is None:
proc.kill()
except Exception:
pass
except Exception:
try:
proc.kill()
+15 -4
View File
@@ -602,7 +602,7 @@ class ProcessManager:
self.logger.info(f"后台进程 #{entry.handle} TTL 到期已终止")
def _terminate_tree(self, entry: _ProcEntry, force: bool = False):
"""按进程组终止整棵进程树(对齐 executors L9 语义与 Windows 回退)"""
"""按进程组终止整棵进程树(对齐 executors L9/T11 语义与 Windows 回退)"""
proc = entry.popen
try:
if os.name == "posix":
@@ -611,10 +611,21 @@ class ProcessManager:
if pgid is None:
pgid = os.getpgid(proc.pid)
os.killpg(pgid, sig)
elif force:
proc.kill()
else:
proc.terminate()
# T11(审计 2026-08-21):Windows 无进程组,单进程 kill 收割
# 不到 spawn 的孙进程(dev server/训练任务成孤儿继续占用端口
# /文件锁)。taskkill /T /F 递归终止整棵树,失败兜底单进程 kill
try:
subprocess.run(
["taskkill", "/PID", str(proc.pid), "/T", "/F"],
capture_output=True, timeout=10)
except Exception:
pass
try:
if proc.poll() is None:
proc.kill()
except Exception:
pass
except Exception:
try:
proc.kill()
+13 -6
View File
@@ -123,12 +123,18 @@ class SkillManager:
# ------------------------------------------------------------------
def _load_skills(self):
"""扫描技能目录,加载所有技能文件(可执行文件或 Markdown 文件)。"""
"""扫描技能目录,加载所有技能文件(可执行文件或 Markdown 文件)。
T26(审计 2026-08-21):本地构建 new_skills 完成后一次性原子替换,
对齐 PromptManager M6——此前先 self.skills = {} 清空再逐个重建,
reload 期间(每个可执行技能 --help 最多 10s)并发读者会看到空/半截
技能表(技能"消失"再"逐渐出现")。"""
skills_dir = constants.SKILLS_DIR
self.skills = {}
new_skills = {}
if not os.path.exists(skills_dir):
self.logger.info(f"技能目录不存在: {skills_dir}")
self.skills = new_skills
return
skill_files = [
@@ -153,7 +159,7 @@ class SkillManager:
is_markdown = self._is_markdown_file(skill_file)
if is_markdown:
if skill_name in self.skills:
if skill_name in new_skills:
self.logger.warning(
f"Markdown 技能 '{skill_file}' 与同名可执行技能冲突,已跳过"
f"(可执行技能优先)")
@@ -161,7 +167,7 @@ class SkillManager:
# Markdown 格式技能:直接读取内容
summary, content = self._read_markdown_skill(skill_path)
if content:
self.skills[skill_name] = {
new_skills[skill_name] = {
"path": skill_path,
"file": skill_file,
"summary": summary,
@@ -182,7 +188,7 @@ class SkillManager:
help_output = self._run_help(skill_path)
summary = self._first_nonempty_line(help_output) or f"扩展技能: {skill_name}"
self.skills[skill_name] = {
new_skills[skill_name] = {
"path": skill_path,
"file": skill_file,
"summary": summary,
@@ -192,7 +198,8 @@ class SkillManager:
}
self.logger.info(f"已加载可执行技能: {skill_name} — {summary}")
self.logger.info(f"共加载 {len(self.skills)} 个技能({len([s for s in self.skills.values() if s['is_markdown']])} 个 Markdown 技能)")
self.skills = new_skills
self.logger.info(f"共加载 {len(new_skills)} 个技能({len([s for s in new_skills.values() if s['is_markdown']])} 个 Markdown 技能)")
def reload(self) -> int:
"""重新扫描技能目录,返回加载的技能数量。"""
+334
View File
@@ -0,0 +1,334 @@
"""tests.test_n_fixes - Batch N 修复单元测试
波次 3 平台兼容(2026-08-21,审计第八轮 T 系列):
- T3 basic_tools 平台感知引用 + Windows PowerShell/curl.exe 指引
- T11 Windows 进程树收割(taskkill /T /F,executors + process_manager)
- T25 MemoryDatabase 统一连接工厂(busy_timeout)
- T26 SkillManager 原子 reload(中途不清空 self.skills)
- T27 atomic_write_text 对 Windows PermissionError 短退避重试
Windows 分支在 Linux 上以 mock.patch("os.name", "nt") 模拟验证命令生成
逻辑(不实际执行生成的命令)。
"""
import os
import sqlite3
from unittest import mock
import pytest
from common.atomic import atomic_write_text
from subsystems import MemoryDatabase, SkillManager
from subsystems.executors import SubprocessCommandExecutor
from tools.basic_tools import (
ListFilesTool,
ReadFileTool,
SimpleWebCrawlerTool,
WriteFileTool,
_quote_arg,
)
# ==================== T3:平台感知引用 ====================
class TestQuoteArg:
def test_posix_uses_shlex(self):
quoted, err = _quote_arg("/tmp/my file.txt")
assert err is None
assert quoted == "'/tmp/my file.txt'"
@pytest.mark.skipif(os.name != "posix", reason="POSIX 引用语义")
def test_posix_specials_escaped(self):
quoted, err = _quote_arg("a`b$c$(d)\\e")
assert err is None
assert quoted == "'a`b$c$(d)\\e'"
def test_win_wraps_double_quotes(self):
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg(r"C:\Program Files\my app\doc.txt")
assert err is None
assert quoted == r'"C:\Program Files\my app\doc.txt"'
def test_win_rejects_double_quote(self):
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg('evil" & del /f /s /q C:\\')
assert quoted == ""
assert err is not None and "双引号" in err
def test_win_rejects_cmd_var_expansion(self):
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg("https://evil.com/?x=%PATH%")
assert quoted == ""
assert err is not None and "注入" in err
def test_win_allows_url_percent_encoding(self):
"""URL 编码(%20/%E4 等)非 %变量名% 形态,不拒绝。"""
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg("https://example.com/a%20b?x=1&y=2")
assert err is None
assert quoted == '"https://example.com/a%20b?x=1&y=2"'
# ==================== T3:Windows 指引命令生成 ====================
class TestReadFileWindows:
def test_head_powershell(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
with mock.patch("os.name", "nt"):
r = ReadFileTool().execute(filepath=str(f), head=True, max_lines=10)
assert "powershell" in r and "Get-Content" in r
assert "run_command" in r
assert "sed -n" not in r and "head -n" not in r
def test_tail_powershell(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
with mock.patch("os.name", "nt"):
r = ReadFileTool().execute(filepath=str(f), tail=True, max_lines=5)
assert "Get-Content" in r and "-Tail 5" in r
def test_range_powershell(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
with mock.patch("os.name", "nt"):
r = ReadFileTool().execute(filepath=str(f), start_line=5, end_line=15)
assert "Select-Object -Skip 4 -First 11" in r
def test_posix_unchanged(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
r = ReadFileTool().execute(filepath=str(f), start_line=1, end_line=5)
assert "sed -n '1,5p'" in r
class TestWriteFileWindows:
def test_base64_set_content(self, tmp_path):
target = tmp_path / "sub" / "out.txt"
content = "line1\nline2\n含中文 \"quoted\" & special"
with mock.patch("os.name", "nt"):
r = WriteFileTool().execute(filepath=str(target), content=content, mode="w")
assert "Set-Content" in r
assert "FromBase64String" in r
# 明文内容(含元字符)不得出现在命令行中
assert "quoted" not in r and "special" not in r
def test_append_uses_add_content(self, tmp_path):
with mock.patch("os.name", "nt"):
r = WriteFileTool().execute(
filepath=str(tmp_path / "a.txt"), content="more", mode="a")
assert "Add-Content" in r
def test_oversize_rejected(self, tmp_path):
with mock.patch("os.name", "nt"):
r = WriteFileTool().execute(
filepath=str(tmp_path / "big.txt"), content="x" * 10000, mode="w")
assert "过大" in r and "追加" in r
class TestListFilesWindows:
def test_get_child_item(self, tmp_path):
with mock.patch("os.name", "nt"):
r = ListFilesTool().execute(directory=str(tmp_path), show_hidden=True)
assert "Get-ChildItem" in r and "-Force" in r
assert "ls -l" not in r
def test_posix_unchanged(self, tmp_path):
r = ListFilesTool().execute(directory=str(tmp_path))
assert "ls -l " in r
class TestWebWindows:
def test_get_curl_exe_amp_url_quoted(self):
tool = SimpleWebCrawlerTool()
url = "https://example.com/?a=1&b=2"
with mock.patch("os.name", "nt"):
r = tool.execute(action="get", url=url)
assert "curl.exe" in r
# & 必须在双引号内(cmd 双引号内为字面量)
assert '"https://example.com/?a=1&b=2"' in r
def test_cmd_var_url_rejected(self):
tool = SimpleWebCrawlerTool()
with mock.patch("os.name", "nt"):
r = tool.execute(action="get", url="https://evil.com/?x=%PATH%")
assert "拒绝" in r
def test_text_action_powershell_pipe(self):
tool = SimpleWebCrawlerTool()
with mock.patch("os.name", "nt"):
r = tool.execute(action="text", url="https://example.com/")
assert "curl.exe" in r and "powershell" in r
def test_posix_unchanged(self):
tool = SimpleWebCrawlerTool()
r = tool.execute(action="get", url="https://example.com/")
assert "curl -sL" in r
# ==================== T11:Windows 进程树收割 ====================
class TestTerminateTreeWindows:
def test_executors_taskkill_tree(self):
ex = SubprocessCommandExecutor()
proc = mock.Mock()
proc.pid = 12345
proc.poll.return_value = None
with mock.patch("os.name", "nt"), \
mock.patch("subprocess.run") as run:
ex._terminate_tree(proc, force=True)
run.assert_called_once()
assert run.call_args[0][0] == ["taskkill", "/PID", "12345", "/T", "/F"]
proc.kill.assert_called_once() # 兜底单进程 kill
def test_executors_taskkill_process_gone(self):
"""taskkill 目标已退出:兜底 kill 不执行(poll 非 None)。"""
ex = SubprocessCommandExecutor()
proc = mock.Mock()
proc.pid = 12346
proc.poll.return_value = 0
with mock.patch("os.name", "nt"), \
mock.patch("subprocess.run") as run:
ex._terminate_tree(proc, force=False)
run.assert_called_once()
proc.kill.assert_not_called()
def test_process_manager_taskkill_tree(self):
from subsystems.process_manager import ProcessManager
pm = ProcessManager()
entry = mock.Mock()
entry.popen.pid = 999
entry.popen.poll.return_value = None
with mock.patch("os.name", "nt"), \
mock.patch("subprocess.run") as run:
pm._terminate_tree(entry, force=True)
assert run.call_args[0][0] == ["taskkill", "/PID", "999", "/T", "/F"]
@pytest.mark.skipif(os.name != "posix", reason="POSIX 进程组语义")
def test_posix_no_taskkill(self):
"""POSIX 分支不调用 taskkill(走 killpg)。"""
import subprocess as _sp
ex = SubprocessCommandExecutor()
proc = _sp.Popen(["sleep", "30"], start_new_session=True)
try:
with mock.patch("subprocess.run") as run:
ex._terminate_tree(proc, force=False)
run.assert_not_called()
finally:
try:
import signal as _sig
os.killpg(os.getpgid(proc.pid), _sig.SIGKILL)
except Exception:
pass
# ==================== T25:MemoryDatabase busy_timeout ====================
class TestMemoryDbBusyTimeout:
def test_connect_sets_busy_timeout(self, tmp_path):
db = MemoryDatabase(db_path=str(tmp_path / "mem.db"))
conn = db._connect()
try:
value = conn.execute("PRAGMA busy_timeout").fetchone()[0]
finally:
conn.close()
assert value == 10000
def test_all_paths_via_factory(self, tmp_path):
"""写/读路径不再直连 sqlite3.connect(统一走 _connect 工厂)。"""
db = MemoryDatabase(db_path=str(tmp_path / "mem.db"))
mid = db.add_memory(topic="t", summary="s", content="c",
start_time="2026-01-01 00:00:00",
end_time="2026-01-01 00:01:00")
assert mid is not None
rows = db.get_recent_memories(limit=5)
assert any(r["id"] == mid for r in rows)
# ==================== T26:SkillManager 原子 reload ====================
@pytest.fixture
def isolated_skill_dir(tmp_path, monkeypatch):
from common import constants
skills_dir = tmp_path / "skills"
skills_dir.mkdir()
monkeypatch.setattr(type(constants), "SKILLS_DIR",
property(lambda self: str(skills_dir)))
return skills_dir
class TestSkillReloadAtomic:
def test_skills_visible_during_reload(self, isolated_skill_dir):
"""reload 过程中 self.skills 保持旧表(T26:不再先清空)。"""
(isolated_skill_dir / "a.md").write_text("# A\n内容A", encoding="utf-8")
sm = SkillManager()
assert sm.count() == 1
(isolated_skill_dir / "b.md").write_text("# B\n内容B", encoding="utf-8")
original = sm._read_markdown_skill
observed = []
calls = []
def spy(path):
calls.append(path)
if len(calls) == 2:
observed.append(dict(sm.skills))
return original(path)
sm._read_markdown_skill = spy
sm.reload()
# 加载第 2 个技能时,旧表完整可见(未被清空/半截)
assert len(observed) == 1
assert set(observed[0].keys()) == {"a"}
# reload 完成后一次性替换为新表
assert set(sm.skills.keys()) == {"a", "b"}
def test_reload_missing_dir_keeps_old(self, isolated_skill_dir, monkeypatch):
"""目录消失时清空(旧行为保留),正常 reload 不经过空表中间态。"""
from common import constants
(isolated_skill_dir / "a.md").write_text("# A\n内容A", encoding="utf-8")
sm = SkillManager()
assert sm.count() == 1
monkeypatch.setattr(type(constants), "SKILLS_DIR",
property(lambda self: str(isolated_skill_dir / "nope")))
sm.reload()
assert sm.count() == 0
# ==================== T27:atomic_write_text 重试 ====================
class TestAtomicWriteRetry:
def test_retries_then_succeeds(self, tmp_path):
target = tmp_path / "cfg.json"
real_replace = os.replace
calls = []
def flaky_replace(src, dst):
calls.append((src, dst))
if len(calls) < 3:
raise PermissionError(5, "Access is denied")
return real_replace(src, dst)
with mock.patch("os.replace", side_effect=flaky_replace):
atomic_write_text(str(target), '{"k": 1}')
assert len(calls) == 3
assert target.read_text(encoding="utf-8") == '{"k": 1}'
def test_raises_after_exhausted(self, tmp_path):
target = tmp_path / "cfg.json"
with mock.patch("os.replace",
side_effect=PermissionError(5, "Access is denied")):
with pytest.raises(PermissionError):
atomic_write_text(str(target), "x")
assert not target.exists()
# 临时文件已清理
leftovers = [p for p in tmp_path.iterdir() if p.name.startswith(".aibox_tmp_")]
assert leftovers == []
def test_normal_write_no_retry(self, tmp_path):
target = tmp_path / "ok.txt"
with mock.patch("os.replace", wraps=os.replace) as spy:
atomic_write_text(str(target), "hello")
spy.assert_called_once()
assert target.read_text(encoding="utf-8") == "hello"
+180 -8
View File
@@ -15,8 +15,10 @@ tools.basic_tools - 基础工具集(A 批抽离)
依赖方向:common <- tools.basic_tools,不反向依赖 aibox 任何符号。
"""
import base64
import os
import queue
import re
import shlex
from datetime import datetime
from typing import Dict, Optional, Tuple
@@ -25,6 +27,40 @@ from common import ConfigManager, constants
from .base import Tool
# T3(审计 2026-08-21):cmd.exe 变量展开形态——%VAR% 匹配已定义环境变量时
# 展开为变量值,值中的 & | 会成为命令分隔符(经典 cmd 注入)。URL 编码
# (%20/%E4 等)与 PowerShell 的 %{http_code} 均不匹配该模式,不受影响。
_WIN_VAR_PATTERN = re.compile(r"%[A-Za-z_][A-Za-z0-9_]*%")
def _quote_arg(value: str) -> Tuple[str, Optional[str]]:
"""平台感知的 shell 参数引用,返回 (quoted, error)。
POSIX:shlex.quote(sh 单引号语义)。
Windows(run_command 经 cmd.exe / shell=True 执行):
- 双引号包裹(& | < > ( ) ^ 在双引号内为字面量)
- 含 " → 拒绝(闭合外层双引号,其后内容暴露给 cmd 解析)
- 匹配 %VAR% 形态 → 拒绝(cmd 变量展开注入面)
- 含换行/控制字符 → 拒绝(cmd 命令行不可含换行)
"""
value = str(value)
if os.name != "nt":
return shlex.quote(value), None
if '"' in value:
return "", f"❌ 参数含双引号,无法安全生成 Windows 命令: {value[:100]}"
if "\n" in value or "\r" in value:
return "", f"❌ 参数含换行符,无法生成命令: {value[:100]}"
if _WIN_VAR_PATTERN.search(value):
return "", (f"❌ 参数疑似含 cmd 变量引用(%变量名%),已拒绝以防命令注入: "
f"{value[:100]}")
return f'"{value}"', None
def _ps_quote(value: str) -> str:
"""PowerShell 单引号字面量(内部单引号翻倍),用于 -Command 内层参数。"""
return "'" + str(value).replace("'", "''") + "'"
# ==================== 文件操作工具基类 ====================
class FileToolBase(Tool):
@@ -106,9 +142,54 @@ class ReadFileTool(FileToolBase):
return "❌ 参数错误: start_line/end_line 必须为整数"
start_line = max(1, start_line)
end_line = max(0, min(end_line, 10_000_000))
quoted = shlex.quote(abs_path)
# 构造等价的命令行读取命令
# T3(审计 2026-08-21):shlex.quote 是 POSIX 单引号语义,Windows 上
# run_command 经 cmd.exe 执行——单引号不被识别,& | % < > 仍是元字符
# (URL 含 & 即命令被截断;%VAR% 变量展开可注入)。按平台生成指引:
# POSIX 保持 head/tail/sed,Windows 走 PowerShell 等价命令。
quoted, qerr = _quote_arg(abs_path)
if qerr:
return qerr
if os.name == "nt":
# PowerShell 等价命令(经 cmd 调 powershell,-Command 整体双引号
# 包裹,内层路径用 _ps_quote 单引号字面量)
if head:
cmd = (f'powershell -NoProfile -Command "Get-Content -Path '
f'{_ps_quote(abs_path)} -TotalCount {max_lines}"')
wc_cmd = (f'powershell -NoProfile -Command "(Get-Content '
f'{_ps_quote(abs_path)} | Measure-Object -Line).Lines"')
elif tail:
cmd = (f'powershell -NoProfile -Command "Get-Content -Path '
f'{_ps_quote(abs_path)} -Tail {max_lines}"')
wc_cmd = (f'powershell -NoProfile -Command "(Get-Content '
f'{_ps_quote(abs_path)} | Measure-Object -Line).Lines"')
else:
if end_line > 0:
first = start_line - 1
take = end_line - start_line + 1
elif max_lines > 0:
first = start_line - 1
take = max_lines
else:
first = start_line - 1
take = 10_000_000
cmd = (f'powershell -NoProfile -Command "Get-Content -Path '
f'{_ps_quote(abs_path)} | Select-Object -Skip {first} '
f'-First {take}"')
wc_cmd = (f'powershell -NoProfile -Command "(Get-Content '
f'{_ps_quote(abs_path)} | Measure-Object -Line).Lines"')
prompt = f"""请使用 `run_command` 工具执行以下命令来读取文件内容,不要直接返回文件内容:
```
{cmd}
```
- 若需统计总行数:`{wc_cmd}`
- 文件大小上限 {constants.MAX_FILE_SIZE} 字节,单次最多读取 500 行"""
return prompt
# 构造等价的命令行读取命令(POSIX)
if head:
cmd = f"head -n {max_lines} {quoted}"
elif tail:
@@ -180,7 +261,40 @@ class WriteFileTool(FileToolBase):
if not is_safe:
return error
quoted = shlex.quote(abs_path)
quoted, qerr = _quote_arg(abs_path)
if qerr:
return qerr
if os.name == "nt":
# T3:Windows 下 heredoc 不可用(cmd 命令行不能含换行,且引号
# 转义跨 cmd/PowerShell 两层极易出错)。content 整体 Base64 编码
# 后经 PowerShell 解码写入——Base64 字符集 [A-Za-z0-9+/=] 不含
# 任何 cmd/PowerShell 元字符,多行/引号/特殊字符内容全覆盖。
b64 = base64.b64encode(content.encode("utf-8")).decode("ascii")
if len(b64) > 6000:
return ("❌ 写入内容过大:Windows 命令行长度上限约 8191 字符,"
"Base64 编码后超出。请将内容分成多次调用(每次 ≤4KB),"
"首次 mode=w、后续 mode=a 追加写入")
writer = "Add-Content" if mode == "a" else "Set-Content"
write_cmd = (f'powershell -NoProfile -Command "{writer} -Path '
f'{_ps_quote(abs_path)} -Value ([Text.Encoding]::UTF8.'
f'GetString([Convert]::FromBase64String(\'{b64}\')))"')
directory = os.path.dirname(abs_path)
if directory:
write_cmd = (f'powershell -NoProfile -Command "New-Item '
f'-ItemType Directory -Force -Path '
f'{_ps_quote(directory)}" & {write_cmd}')
prompt = f"""请使用 `run_command` 工具执行以下命令来写入文件,不要在此工具中直接写入:
```
{write_cmd}
```
- 写入内容已编码为 Base64(避免换行/引号/特殊字符的跨 cmd+PowerShell 转义问题),命令会自动解码并以 UTF-8 写入
- 若需检查文件大小:`powershell -NoProfile -Command "(Get-Item {_ps_quote(abs_path)}).Length"`
- 文件大小上限 {constants.MAX_FILE_SIZE} 字节"""
return prompt
directory = os.path.dirname(abs_path)
dir_cmd = f"mkdir -p {shlex.quote(directory)}" if directory else ""
@@ -254,7 +368,26 @@ class ListFilesTool(FileToolBase):
if not os.path.isdir(abs_path):
return f"❌ 路径不是目录: {abs_path}"
quoted = shlex.quote(abs_path)
quoted, qerr = _quote_arg(abs_path)
if qerr:
return qerr
if os.name == "nt":
# T3:Windows 走 PowerShell 等价命令(ls/du/find/file 无原生等价物)
force_flag = " -Force" if show_hidden else ""
cmd = (f'powershell -NoProfile -Command "Get-ChildItem{force_flag} '
f'{_ps_quote(abs_path)} | Select-Object '
f'Mode,LastWriteTime,Length,Name"')
prompt = f"""请使用 `run_command` 工具执行以下命令来列出目录内容,不要直接在此工具中列出:
```
{cmd}
```
- 若需统计目录总大小:`powershell -NoProfile -Command "(Get-ChildItem -Recurse {_ps_quote(abs_path)} | Measure-Object Length -Sum).Sum"`
- 若需递归列出:`powershell -NoProfile -Command "Get-ChildItem -Recurse -Depth 2 {_ps_quote(abs_path)} | Select-Object FullName"`"""
return prompt
flag = "-la" if show_hidden else "-l"
# 返回 markdown 提示词,指导智能体用命令行工具列目录
@@ -324,7 +457,9 @@ class SimpleWebCrawlerTool(Tool):
host_port = proxy_url.split("://", 1)[-1]
if "@" in host_port:
host_port = host_port.split("@", 1)[-1]
return f"--socks5 {shlex.quote(host_port)}"
# T3:平台感知引用(原 shlex.quote 在 Windows cmd.exe 下无效)
quoted, _err = _quote_arg(host_port)
return f"--socks5 {quoted}" if quoted else ""
def execute(self, action: str, url: str, **kwargs) -> str:
if action not in ("get", "text", "title", "links", "search"):
@@ -336,10 +471,44 @@ class SimpleWebCrawlerTool(Tool):
if not url or not str(url).strip().lower().startswith(("http://", "https://")):
return "❌ 仅支持 http/https 协议地址(已拦截 file:// 等非网络协议)"
quoted_url = shlex.quote(url)
quoted_url, qerr = _quote_arg(url)
if qerr:
return qerr
timeout = constants.WEB_TIMEOUT
user_agent = shlex.quote(constants.WEB_USER_AGENT)
user_agent, ua_err = _quote_arg(constants.WEB_USER_AGENT)
if ua_err:
return ua_err
proxy = self._proxy_curl_args()
if os.name == "nt":
# T3:Windows 用系统内置 curl.exe(Win10 1803+);sed/grep 等
# POSIX 管道无原生等价,分析类动作(text/title/links/search)
# 指引先取原始 HTML 再由 AI 自行解析。URL 含 & 时靠双引号
# 保护(cmd 双引号内为字面量),%VAR% 形态已在 _quote_arg 拒绝。
base = f"curl.exe -sL --max-time {timeout} -A {user_agent} {proxy}".strip()
if action == "get":
cmd = f'{base} -w "\\nHTTP_CODE:%{{http_code}}" {quoted_url}'
extra = "如需查看响应头:`curl.exe -sIL ...`;如需保存到文件:`curl.exe -sL ... -o 文件名`"
elif action == "text":
cmd = (f'{base} {quoted_url} | powershell -NoProfile -Command '
f'"$input -replace \'<[^>]+>\',\'\'"')
extra = "去除 HTML 标签提取纯文本"
else:
cmd = f"{base} {quoted_url}"
extra = ("Windows 下无 grep/sed 管道,请取回原始 HTML 后自行分析"
"(提取标题/链接/检索关键词)")
prompt = f"""请使用 `run_command` 工具执行以下命令来完成网页操作,不要直接返回网页内容:
```
{cmd}
```
{extra}
- 下载大小限制 {constants.WEB_MAX_DOWNLOAD_SIZE} 字节,可在命令后追加 `| powershell -NoProfile -Command "$input | Select-Object -First 100"` 限制
- 当前已按配置自动附带代理:{proxy or '无(如需代理请先调用 set_proxy 工具)'}"""
return prompt
# POSIX 分支:_quote_arg 在 POSIX 下即 shlex.quote,直接沿用
base = f"curl -sL --max-time {timeout} -A {user_agent} {proxy}".strip()
if action == "get":
@@ -358,7 +527,10 @@ class SimpleWebCrawlerTool(Tool):
keyword = kwargs.get("keyword")
if not keyword:
return "❌ 需要提供 keyword 参数"
cmd = f"{base} {quoted_url} | grep -c {shlex.quote(keyword)}"
quoted_kw, kw_err = _quote_arg(keyword)
if kw_err:
return kw_err
cmd = f"{base} {quoted_url} | grep -c {quoted_kw}"
extra = f"返回匹配行数,大于 0 即表示找到关键词 '{keyword}'"
# 返回 markdown 提示词,指导智能体用 curl 完成网页访问
+22 -4
View File
@@ -1283,9 +1283,9 @@ FTS5 `MATCH` 左操作数必须是 FROM 子句中的表名或别名。`memories_
八轮累计发现 **166+ 项**(高危 18 / 中危 54 / 低危与信息 94+);前七轮 118 项全部修复落地(Batch E/F/G/H/I/K/L/M),第八轮 T 系列待落地(Batch N 建议见 §23.7)。
## 二十四、Batch N 修复记录(波次 1~2,2026-08-21)
## 二十四、Batch N 修复记录(波次 1~3,2026-08-21)
范围:§23.7 波次 1(速赢 7 项)+ 波次 2(并发正确性 6 项),共 13 项。
范围:§23.7 波次 1(速赢 7 项)+ 波次 2(并发正确性 6 项)+ 波次 3(平台兼容 5 项),共 18 项。
### 24.1 波次 1 落地(速赢批)
@@ -1314,6 +1314,24 @@ FTS5 `MATCH` 左操作数必须是 FROM 子句中的表名或别名。`memories_
波次 1 后:**532 passed / 4 failed / 1 skipped**(43.8s);波次 2 后:**532 passed / 4 failed / 1 skipped**(41.5s)。4 失败均为 §23.8 所列同一批 Windows 平台环境项,零回归。
### 24.4 待落地(波次 3~5)
### 24.4 待落地(波次 4~5)
波次 3 平台兼容(T3/T11/T25/T26/T27)、波次 4 LLM 健壮性(T10/T14/T19/T20/T21/T23,其中 T14 需先写 FTS5 复现用例)、波次 5 权限隔离(T1/T2/T22,含产品决策项)。
波次 4 LLM 健壮性(T10/T14/T19/T20/T21/T23,其中 T14 需先写 FTS5 复现用例)、波次 5 权限隔离(T1/T2/T22,含产品决策项)。
### 24.5 波次 3 落地(平台兼容批,2026-08-21)
范围:T3/T11/T25/T26/T27,共 5 项。
| # | 修复内容 | 文件/位置 |
|---|---|---|
| T3 | `basic_tools.py` 新增平台感知引用 `_quote_arg()`(POSIX=shlex.quote;Windows=双引号包裹 + 拒绝 `"`/换行/`%变量名%` 形态——cmd 变量展开是经典注入向量,URL 编码 %20 与 PowerShell `%{http_code}` 不匹配该模式不受影响)与 `_ps_quote()`(PowerShell 单引号字面量);四个工具按 `os.name` 分支生成指引:ReadFile 走 Get-Content/Select-Object(head/tail/行范围/wc 等价),WriteFile 走 Base64+Set-Content/Add-Content(heredoc 在 cmd 下不可用:命令行不能含换行;Base64 字符集不含任何 cmd/PowerShell 元字符,多行/引号/特殊字符全覆盖;超 6000 字符提示分块追加),ListFiles 走 Get-ChildItem,Web 走系统内置 curl.exe(分析类动作 text 给 PowerShell 管道等价,title/links/search 指引取原始 HTML 自行分析);`_proxy_curl_args`/search keyword 引用同步切换 | `tools/basic_tools.py:33-78,150-237,297-343,389-414,474-533` |
| T11 | executors 与 process_manager 两处 `_terminate_tree` Windows 分支改 `taskkill /PID {pid} /T /F` 递归终止整棵进程树(无 /F 仅对 GUI 进程有效,控制台进程杀不掉,故统一强制;force 参数在 Windows 不区分),taskkill 失败(进程已退出/无权限)兜底单进程 kill;POSIX 分支行为不变 | `subsystems/executors.py:109-157`;`subsystems/process_manager.py:604-634` |
| T25 | MemoryDatabase 新增 `_connect()` 统一连接工厂(`PRAGMA busy_timeout = 10000`),7 处直连 `sqlite3.connect` 全部换走工厂——与 MemoDatabase J9 修复对齐,两类共库时锁等待策略一致,CLI `--memo-add` 与服务进程并发写不再抛 `database is locked` | `subsystems/db.py:36-45`(工厂)+ 7 处调用点 |
| T26 | `SkillManager._load_skills` 本地构建 `new_skills` 完成后一次性原子替换 `self.skills`(对齐 PromptManager M6)——reload 期间(每个可执行技能 `--help` 最多 10s)并发读者不再看到空/半截技能表;目录不存在分支同样保持显式赋值语义 | `subsystems/skill_manager.py:125-213` |
| T27 | `atomic_write_text` 的 `os.replace` 捕获 `PermissionError` 做 3 次短退避重试(50/100ms)——Windows 下目标被杀毒扫描/编辑器/并发读者占用时一次冲突不再直接丢该条持久化(conversation_history 每条消息都走此路径);重试耗尽上抛原异常,临时文件清理逻辑不变;POSIX 下 `os.replace` 无条件成功不受影响 | `common/atomic.py:15-52` |
**Windows 分支验证方式注记**:Linux 环境以 `mock.patch("os.name", "nt")` 模拟验证命令生成逻辑(引用规则/拒绝路径/命令形态),生成的 PowerShell/curl.exe 命令未在真实 Windows 上执行验证,Windows 开发机建议抽测指引命令的实际可用性。
### 24.6 测试基线(Linux,`aibox/venv`)
波次 3 后:**566 passed / 1 failed**(30.1s)。新增 `tests/test_n_fixes.py` 30 项(T3 引用规则 6 + Windows 指引生成 10 + T11 taskkill 4 + T25 busy_timeout 2 + T26 原子 reload 2 + T27 重试 3 + POSIX 锚定 3);1 失败为 §11.3 所列 Windows 向用例(`test_execute_timeout`,本机无 ping 命令),与代码无关,零回归。