fix(aibox): Batch N 波次3 平台兼容(T3/T11/T25/T26/T27)

- T3 basic_tools 平台感知引用与 Windows 指引:_quote_arg 按 os.name
  分支(Windows 双引号包裹,拒绝双引号/换行/%变量名% 注入形态);
  ReadFile/WriteFile/ListFiles/Web 四工具 Windows 走 PowerShell/
  curl.exe 等价命令(WriteFile 用 Base64 规避 cmd 换行/转义限制)
- T11 executors 与 process_manager 的 _terminate_tree Windows 分支
  改 taskkill /T /F 递归收割进程树,兜底单进程 kill
- T25 MemoryDatabase 统一 _connect 工厂(busy_timeout=10000),
  7 处直连替换,对齐 MemoDatabase J9
- T26 SkillManager._load_skills 原子替换 self.skills(对齐
  PromptManager M6),reload 期间不再读到空/半截技能表
- T27 atomic_write_text 对 PermissionError 做 3 次退避重试

测试:新增 tests/test_n_fixes.py 30 项;全量 566 passed / 1 failed
(Windows 向 ping 用例,环境项),零回归。审计报告补记 §24.5/§24.6。
This commit is contained in:
qing committed 2026-08-21 13:56:33 +00:00
1 parent 5894e64287
commit 2dc79f0637
8 files changed
+617 -34

No files matched your search

+18 -1
View File
@@ -10,6 +10,7 @@ mkstemp 随机文件名避免多进程并发写同路径互踩(conversation_hi
import os import os
import tempfile import tempfile
import time
def atomic_write_text(path: str, content: str) -> None: def atomic_write_text(path: str, content: str) -> None:
@@ -23,10 +24,26 @@ def atomic_write_text(path: str, content: str) -> None:
f.write(content) f.write(content)
f.flush() f.flush()
os.fsync(f.fileno()) os.fsync(f.fileno())
os.replace(tmp_path, path) # T27(审计 2026-08-21):Windows 下目标被其他进程打开(杀毒扫描/
# 编辑器/并发读者)时 os.replace 抛 PermissionError。POSIX 无条件
# 成功不受影响;Windows 做短退避重试,高频写路径(每条消息都原子
# 写 current_conversation.json)一次冲突不再直接丢该条持久化。
last_exc = None
for attempt in range(3):
try:
os.replace(tmp_path, path)
last_exc = None
break
except PermissionError as e:
last_exc = e
if attempt < 2:
time.sleep(0.05 * (attempt + 1))
if last_exc is not None:
raise last_exc
except BaseException: except BaseException:
try: try:
os.unlink(tmp_path) os.unlink(tmp_path)
except OSError: except OSError:
pass pass
raise raise
+16 -7
View File
@@ -33,9 +33,18 @@ class MemoryDatabase:
self.logger = Logger("MemoryDB") self.logger = Logger("MemoryDB")
self._init_database() self._init_database()
def _connect(self) -> sqlite3.Connection:
"""统一连接工厂(T25,审计 2026-08-21):对齐 MemoDatabase J9 修复——
busy_timeout 覆盖全部路径。两类默认共用同一 memories.db,此前
MemoryDatabase 直连只靠 sqlite3 默认 5s 等待,CLI 与服务进程并发
写时锁冲突即抛 database is locked(记忆保存/检索失败)。"""
conn = sqlite3.connect(self.db_path)
conn.execute("PRAGMA busy_timeout = 10000")
return conn
def _init_database(self): def _init_database(self):
"""初始化数据库表""" """初始化数据库表"""
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
# WAL 模式:读写并发不互斥,缓解多 CLI 进程并发写锁等待 # WAL 模式:读写并发不互斥,缓解多 CLI 进程并发写锁等待
conn.execute("PRAGMA journal_mode=WAL") conn.execute("PRAGMA journal_mode=WAL")
cursor = conn.cursor() cursor = conn.cursor()
@@ -118,7 +127,7 @@ class MemoryDatabase:
tags_str = ",".join(tags) if tags else "" tags_str = ",".join(tags) if tags else ""
created_at = datetime.now().strftime(constants.DATETIME_FORMAT) created_at = datetime.now().strftime(constants.DATETIME_FORMAT)
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
cursor = conn.cursor() cursor = conn.cursor()
cursor.execute(''' cursor.execute('''
INSERT INTO memories (topic, summary, content, start_time, end_time, created_at, tags, importance) INSERT INTO memories (topic, summary, content, start_time, end_time, created_at, tags, importance)
@@ -139,7 +148,7 @@ class MemoryDatabase:
def delete_memory(self, memory_id: int) -> bool: def delete_memory(self, memory_id: int) -> bool:
"""删除记忆""" """删除记忆"""
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
cursor = conn.cursor() cursor = conn.cursor()
# J3(审计 2026-08-19):必须先删 fts 再删内容表——外部内容表 # J3(审计 2026-08-19):必须先删 fts 再删内容表——外部内容表
# (content=memories)删除索引条目需从内容表回读旧值,内容行 # (content=memories)删除索引条目需从内容表回读旧值,内容行
@@ -166,7 +175,7 @@ class MemoryDatabase:
if limit is None: if limit is None:
limit = constants.MEMORY_MAX_SEARCH_RESULTS limit = constants.MEMORY_MAX_SEARCH_RESULTS
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
conn.row_factory = sqlite3.Row conn.row_factory = sqlite3.Row
cursor = conn.cursor() cursor = conn.cursor()
@@ -240,7 +249,7 @@ class MemoryDatabase:
的 None→默认上限语义对齐)。""" 的 None→默认上限语义对齐)。"""
if limit is None: if limit is None:
limit = constants.MEMORY_MAX_SEARCH_RESULTS limit = constants.MEMORY_MAX_SEARCH_RESULTS
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
conn.row_factory = sqlite3.Row conn.row_factory = sqlite3.Row
cursor = conn.cursor() cursor = conn.cursor()
cursor.execute('SELECT * FROM memories ORDER BY end_time DESC LIMIT ?', (limit,)) cursor.execute('SELECT * FROM memories ORDER BY end_time DESC LIMIT ?', (limit,))
@@ -256,7 +265,7 @@ class MemoryDatabase:
def get_memory_by_id(self, memory_id: int) -> Optional[Dict]: def get_memory_by_id(self, memory_id: int) -> Optional[Dict]:
"""根据ID获取记忆""" """根据ID获取记忆"""
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
conn.row_factory = sqlite3.Row conn.row_factory = sqlite3.Row
cursor = conn.cursor() cursor = conn.cursor()
cursor.execute('SELECT * FROM memories WHERE id = ?', (memory_id,)) cursor.execute('SELECT * FROM memories WHERE id = ?', (memory_id,))
@@ -274,7 +283,7 @@ class MemoryDatabase:
if not memory_ids: if not memory_ids:
return 0 return 0
with sqlite3.connect(self.db_path) as conn: with self._connect() as conn:
cursor = conn.cursor() cursor = conn.cursor()
placeholders = ','.join(['?'] * len(memory_ids)) placeholders = ','.join(['?'] * len(memory_ids))
# J3:同 delete_memory——先删 fts 再删内容表,防索引条目残留 # J3:同 delete_memory——先删 fts 再删内容表,防索引条目残留
+19 -4
View File
@@ -115,7 +115,12 @@ class SubprocessCommandExecutor(CommandExecutor):
ProcessLookupError,届时无法再从进程反查组。 ProcessLookupError,届时无法再从进程反查组。
T5:pgid 由调用方显式传入(注册槽位时捕获),不再读共享 T5:pgid 由调用方显式传入(注册槽位时捕获),不再读共享
_current_pgid——并发执行时那是别人的进程组。 _current_pgid——并发执行时那是别人的进程组。
Windows 无进程组概念,退回单进程 terminate/kill。""" T11(审计 2026-08-21):Windows 无进程组概念,单进程 terminate/
kill 收割不到 shell=True 生成的 cmd.exe /c 之下 spawn 的孙进程
(超时/取消后孙进程成孤儿,持续占用 CPU/端口/文件锁)。用
taskkill /T 递归终止整棵进程树;Windows 无 SIGTERM 等价物,
taskkill 不带 /F 仅对有消息窗口的 GUI 进程有效,控制台进程
杀不掉,故统一 /F 强制,force 参数在 Windows 上不区分。"""
try: try:
if os.name == "posix": if os.name == "posix":
sig = signal.SIGKILL if force else signal.SIGTERM sig = signal.SIGKILL if force else signal.SIGTERM
@@ -128,10 +133,20 @@ class SubprocessCommandExecutor(CommandExecutor):
os.killpg(pgid, sig) os.killpg(pgid, sig)
else: else:
proc.kill() if force else proc.terminate() proc.kill() if force else proc.terminate()
elif force:
proc.kill()
else: else:
proc.terminate() # T11:Windows 树收割——taskkill /T /F 递归终止子孙进程,
# 失败(进程已退出/无权限)时兜底单进程 kill
try:
subprocess.run(
["taskkill", "/PID", str(proc.pid), "/T", "/F"],
capture_output=True, timeout=10)
except Exception:
pass
try:
if proc.poll() is None:
proc.kill()
except Exception:
pass
except Exception: except Exception:
try: try:
proc.kill() proc.kill()
+15 -4
View File
@@ -602,7 +602,7 @@ class ProcessManager:
self.logger.info(f"后台进程 #{entry.handle} TTL 到期已终止") self.logger.info(f"后台进程 #{entry.handle} TTL 到期已终止")
def _terminate_tree(self, entry: _ProcEntry, force: bool = False): def _terminate_tree(self, entry: _ProcEntry, force: bool = False):
"""按进程组终止整棵进程树(对齐 executors L9 语义与 Windows 回退)""" """按进程组终止整棵进程树(对齐 executors L9/T11 语义与 Windows 回退)"""
proc = entry.popen proc = entry.popen
try: try:
if os.name == "posix": if os.name == "posix":
@@ -611,10 +611,21 @@ class ProcessManager:
if pgid is None: if pgid is None:
pgid = os.getpgid(proc.pid) pgid = os.getpgid(proc.pid)
os.killpg(pgid, sig) os.killpg(pgid, sig)
elif force:
proc.kill()
else: else:
proc.terminate() # T11(审计 2026-08-21):Windows 无进程组,单进程 kill 收割
# 不到 spawn 的孙进程(dev server/训练任务成孤儿继续占用端口
# /文件锁)。taskkill /T /F 递归终止整棵树,失败兜底单进程 kill
try:
subprocess.run(
["taskkill", "/PID", str(proc.pid), "/T", "/F"],
capture_output=True, timeout=10)
except Exception:
pass
try:
if proc.poll() is None:
proc.kill()
except Exception:
pass
except Exception: except Exception:
try: try:
proc.kill() proc.kill()
+13 -6
View File
@@ -123,12 +123,18 @@ class SkillManager:
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _load_skills(self): def _load_skills(self):
"""扫描技能目录,加载所有技能文件(可执行文件或 Markdown 文件)。""" """扫描技能目录,加载所有技能文件(可执行文件或 Markdown 文件)。
T26(审计 2026-08-21):本地构建 new_skills 完成后一次性原子替换,
对齐 PromptManager M6——此前先 self.skills = {} 清空再逐个重建,
reload 期间(每个可执行技能 --help 最多 10s)并发读者会看到空/半截
技能表(技能"消失"再"逐渐出现")。"""
skills_dir = constants.SKILLS_DIR skills_dir = constants.SKILLS_DIR
self.skills = {} new_skills = {}
if not os.path.exists(skills_dir): if not os.path.exists(skills_dir):
self.logger.info(f"技能目录不存在: {skills_dir}") self.logger.info(f"技能目录不存在: {skills_dir}")
self.skills = new_skills
return return
skill_files = [ skill_files = [
@@ -153,7 +159,7 @@ class SkillManager:
is_markdown = self._is_markdown_file(skill_file) is_markdown = self._is_markdown_file(skill_file)
if is_markdown: if is_markdown:
if skill_name in self.skills: if skill_name in new_skills:
self.logger.warning( self.logger.warning(
f"Markdown 技能 '{skill_file}' 与同名可执行技能冲突,已跳过" f"Markdown 技能 '{skill_file}' 与同名可执行技能冲突,已跳过"
f"(可执行技能优先)") f"(可执行技能优先)")
@@ -161,7 +167,7 @@ class SkillManager:
# Markdown 格式技能:直接读取内容 # Markdown 格式技能:直接读取内容
summary, content = self._read_markdown_skill(skill_path) summary, content = self._read_markdown_skill(skill_path)
if content: if content:
self.skills[skill_name] = { new_skills[skill_name] = {
"path": skill_path, "path": skill_path,
"file": skill_file, "file": skill_file,
"summary": summary, "summary": summary,
@@ -182,7 +188,7 @@ class SkillManager:
help_output = self._run_help(skill_path) help_output = self._run_help(skill_path)
summary = self._first_nonempty_line(help_output) or f"扩展技能: {skill_name}" summary = self._first_nonempty_line(help_output) or f"扩展技能: {skill_name}"
self.skills[skill_name] = { new_skills[skill_name] = {
"path": skill_path, "path": skill_path,
"file": skill_file, "file": skill_file,
"summary": summary, "summary": summary,
@@ -192,7 +198,8 @@ class SkillManager:
} }
self.logger.info(f"已加载可执行技能: {skill_name} — {summary}") self.logger.info(f"已加载可执行技能: {skill_name} — {summary}")
self.logger.info(f"共加载 {len(self.skills)} 个技能({len([s for s in self.skills.values() if s['is_markdown']])} 个 Markdown 技能)") self.skills = new_skills
self.logger.info(f"共加载 {len(new_skills)} 个技能({len([s for s in new_skills.values() if s['is_markdown']])} 个 Markdown 技能)")
def reload(self) -> int: def reload(self) -> int:
"""重新扫描技能目录,返回加载的技能数量。""" """重新扫描技能目录,返回加载的技能数量。"""
+334
View File
@@ -0,0 +1,334 @@
"""tests.test_n_fixes - Batch N 修复单元测试
波次 3 平台兼容(2026-08-21,审计第八轮 T 系列):
- T3 basic_tools 平台感知引用 + Windows PowerShell/curl.exe 指引
- T11 Windows 进程树收割(taskkill /T /F,executors + process_manager)
- T25 MemoryDatabase 统一连接工厂(busy_timeout)
- T26 SkillManager 原子 reload(中途不清空 self.skills)
- T27 atomic_write_text 对 Windows PermissionError 短退避重试
Windows 分支在 Linux 上以 mock.patch("os.name", "nt") 模拟验证命令生成
逻辑(不实际执行生成的命令)。
"""
import os
import sqlite3
from unittest import mock
import pytest
from common.atomic import atomic_write_text
from subsystems import MemoryDatabase, SkillManager
from subsystems.executors import SubprocessCommandExecutor
from tools.basic_tools import (
ListFilesTool,
ReadFileTool,
SimpleWebCrawlerTool,
WriteFileTool,
_quote_arg,
)
# ==================== T3:平台感知引用 ====================
class TestQuoteArg:
def test_posix_uses_shlex(self):
quoted, err = _quote_arg("/tmp/my file.txt")
assert err is None
assert quoted == "'/tmp/my file.txt'"
@pytest.mark.skipif(os.name != "posix", reason="POSIX 引用语义")
def test_posix_specials_escaped(self):
quoted, err = _quote_arg("a`b$c$(d)\\e")
assert err is None
assert quoted == "'a`b$c$(d)\\e'"
def test_win_wraps_double_quotes(self):
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg(r"C:\Program Files\my app\doc.txt")
assert err is None
assert quoted == r'"C:\Program Files\my app\doc.txt"'
def test_win_rejects_double_quote(self):
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg('evil" & del /f /s /q C:\\')
assert quoted == ""
assert err is not None and "双引号" in err
def test_win_rejects_cmd_var_expansion(self):
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg("https://evil.com/?x=%PATH%")
assert quoted == ""
assert err is not None and "注入" in err
def test_win_allows_url_percent_encoding(self):
"""URL 编码(%20/%E4 等)非 %变量名% 形态,不拒绝。"""
with mock.patch("os.name", "nt"):
quoted, err = _quote_arg("https://example.com/a%20b?x=1&y=2")
assert err is None
assert quoted == '"https://example.com/a%20b?x=1&y=2"'
# ==================== T3:Windows 指引命令生成 ====================
class TestReadFileWindows:
def test_head_powershell(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
with mock.patch("os.name", "nt"):
r = ReadFileTool().execute(filepath=str(f), head=True, max_lines=10)
assert "powershell" in r and "Get-Content" in r
assert "run_command" in r
assert "sed -n" not in r and "head -n" not in r
def test_tail_powershell(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
with mock.patch("os.name", "nt"):
r = ReadFileTool().execute(filepath=str(f), tail=True, max_lines=5)
assert "Get-Content" in r and "-Tail 5" in r
def test_range_powershell(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
with mock.patch("os.name", "nt"):
r = ReadFileTool().execute(filepath=str(f), start_line=5, end_line=15)
assert "Select-Object -Skip 4 -First 11" in r
def test_posix_unchanged(self, tmp_path):
f = tmp_path / "data.txt"
f.write_text("hello\n" * 30, encoding="utf-8")
r = ReadFileTool().execute(filepath=str(f), start_line=1, end_line=5)
assert "sed -n '1,5p'" in r
class TestWriteFileWindows:
def test_base64_set_content(self, tmp_path):
target = tmp_path / "sub" / "out.txt"
content = "line1\nline2\n含中文 \"quoted\" & special"
with mock.patch("os.name", "nt"):
r = WriteFileTool().execute(filepath=str(target), content=content, mode="w")
assert "Set-Content" in r
assert "FromBase64String" in r
# 明文内容(含元字符)不得出现在命令行中
assert "quoted" not in r and "special" not in r
def test_append_uses_add_content(self, tmp_path):
with mock.patch("os.name", "nt"):
r = WriteFileTool().execute(
filepath=str(tmp_path / "a.txt"), content="more", mode="a")
assert "Add-Content" in r
def test_oversize_rejected(self, tmp_path):
with mock.patch("os.name", "nt"):
r = WriteFileTool().execute(
filepath=str(tmp_path / "big.txt"), content="x" * 10000, mode="w")
assert "过大" in r and "追加" in r
class TestListFilesWindows:
def test_get_child_item(self, tmp_path):
with mock.patch("os.name", "nt"):
r = ListFilesTool().execute(directory=str(tmp_path), show_hidden=True)
assert "Get-ChildItem" in r and "-Force" in r
assert "ls -l" not in r
def test_posix_unchanged(self, tmp_path):
r = ListFilesTool().execute(directory=str(tmp_path))
assert "ls -l " in r
class TestWebWindows:
def test_get_curl_exe_amp_url_quoted(self):
tool = SimpleWebCrawlerTool()
url = "https://example.com/?a=1&b=2"
with mock.patch("os.name", "nt"):
r = tool.execute(action="get", url=url)
assert "curl.exe" in r
# & 必须在双引号内(cmd 双引号内为字面量)
assert '"https://example.com/?a=1&b=2"' in r
def test_cmd_var_url_rejected(self):
tool = SimpleWebCrawlerTool()
with mock.patch("os.name", "nt"):
r = tool.execute(action="get", url="https://evil.com/?x=%PATH%")
assert "拒绝" in r
def test_text_action_powershell_pipe(self):
tool = SimpleWebCrawlerTool()
with mock.patch("os.name", "nt"):
r = tool.execute(action="text", url="https://example.com/")
assert "curl.exe" in r and "powershell" in r
def test_posix_unchanged(self):
tool = SimpleWebCrawlerTool()
r = tool.execute(action="get", url="https://example.com/")
assert "curl -sL" in r
# ==================== T11:Windows 进程树收割 ====================
class TestTerminateTreeWindows:
def test_executors_taskkill_tree(self):
ex = SubprocessCommandExecutor()
proc = mock.Mock()
proc.pid = 12345
proc.poll.return_value = None
with mock.patch("os.name", "nt"), \
mock.patch("subprocess.run") as run:
ex._terminate_tree(proc, force=True)
run.assert_called_once()
assert run.call_args[0][0] == ["taskkill", "/PID", "12345", "/T", "/F"]
proc.kill.assert_called_once() # 兜底单进程 kill
def test_executors_taskkill_process_gone(self):
"""taskkill 目标已退出:兜底 kill 不执行(poll 非 None)。"""
ex = SubprocessCommandExecutor()
proc = mock.Mock()
proc.pid = 12346
proc.poll.return_value = 0
with mock.patch("os.name", "nt"), \
mock.patch("subprocess.run") as run:
ex._terminate_tree(proc, force=False)
run.assert_called_once()
proc.kill.assert_not_called()
def test_process_manager_taskkill_tree(self):
from subsystems.process_manager import ProcessManager
pm = ProcessManager()
entry = mock.Mock()
entry.popen.pid = 999
entry.popen.poll.return_value = None
with mock.patch("os.name", "nt"), \
mock.patch("subprocess.run") as run:
pm._terminate_tree(entry, force=True)
assert run.call_args[0][0] == ["taskkill", "/PID", "999", "/T", "/F"]
@pytest.mark.skipif(os.name != "posix", reason="POSIX 进程组语义")
def test_posix_no_taskkill(self):
"""POSIX 分支不调用 taskkill(走 killpg)。"""
import subprocess as _sp
ex = SubprocessCommandExecutor()
proc = _sp.Popen(["sleep", "30"], start_new_session=True)
try:
with mock.patch("subprocess.run") as run:
ex._terminate_tree(proc, force=False)
run.assert_not_called()
finally:
try:
import signal as _sig
os.killpg(os.getpgid(proc.pid), _sig.SIGKILL)
except Exception:
pass
# ==================== T25:MemoryDatabase busy_timeout ====================
class TestMemoryDbBusyTimeout:
def test_connect_sets_busy_timeout(self, tmp_path):
db = MemoryDatabase(db_path=str(tmp_path / "mem.db"))
conn = db._connect()
try:
value = conn.execute("PRAGMA busy_timeout").fetchone()[0]
finally:
conn.close()
assert value == 10000
def test_all_paths_via_factory(self, tmp_path):
"""写/读路径不再直连 sqlite3.connect(统一走 _connect 工厂)。"""
db = MemoryDatabase(db_path=str(tmp_path / "mem.db"))
mid = db.add_memory(topic="t", summary="s", content="c",
start_time="2026-01-01 00:00:00",
end_time="2026-01-01 00:01:00")
assert mid is not None
rows = db.get_recent_memories(limit=5)
assert any(r["id"] == mid for r in rows)
# ==================== T26:SkillManager 原子 reload ====================
@pytest.fixture
def isolated_skill_dir(tmp_path, monkeypatch):
from common import constants
skills_dir = tmp_path / "skills"
skills_dir.mkdir()
monkeypatch.setattr(type(constants), "SKILLS_DIR",
property(lambda self: str(skills_dir)))
return skills_dir
class TestSkillReloadAtomic:
def test_skills_visible_during_reload(self, isolated_skill_dir):
"""reload 过程中 self.skills 保持旧表(T26:不再先清空)。"""
(isolated_skill_dir / "a.md").write_text("# A\n内容A", encoding="utf-8")
sm = SkillManager()
assert sm.count() == 1
(isolated_skill_dir / "b.md").write_text("# B\n内容B", encoding="utf-8")
original = sm._read_markdown_skill
observed = []
calls = []
def spy(path):
calls.append(path)
if len(calls) == 2:
observed.append(dict(sm.skills))
return original(path)
sm._read_markdown_skill = spy
sm.reload()
# 加载第 2 个技能时,旧表完整可见(未被清空/半截)
assert len(observed) == 1
assert set(observed[0].keys()) == {"a"}
# reload 完成后一次性替换为新表
assert set(sm.skills.keys()) == {"a", "b"}
def test_reload_missing_dir_keeps_old(self, isolated_skill_dir, monkeypatch):
"""目录消失时清空(旧行为保留),正常 reload 不经过空表中间态。"""
from common import constants
(isolated_skill_dir / "a.md").write_text("# A\n内容A", encoding="utf-8")
sm = SkillManager()
assert sm.count() == 1
monkeypatch.setattr(type(constants), "SKILLS_DIR",
property(lambda self: str(isolated_skill_dir / "nope")))
sm.reload()
assert sm.count() == 0
# ==================== T27:atomic_write_text 重试 ====================
class TestAtomicWriteRetry:
def test_retries_then_succeeds(self, tmp_path):
target = tmp_path / "cfg.json"
real_replace = os.replace
calls = []
def flaky_replace(src, dst):
calls.append((src, dst))
if len(calls) < 3:
raise PermissionError(5, "Access is denied")
return real_replace(src, dst)
with mock.patch("os.replace", side_effect=flaky_replace):
atomic_write_text(str(target), '{"k": 1}')
assert len(calls) == 3
assert target.read_text(encoding="utf-8") == '{"k": 1}'
def test_raises_after_exhausted(self, tmp_path):
target = tmp_path / "cfg.json"
with mock.patch("os.replace",
side_effect=PermissionError(5, "Access is denied")):
with pytest.raises(PermissionError):
atomic_write_text(str(target), "x")
assert not target.exists()
# 临时文件已清理
leftovers = [p for p in tmp_path.iterdir() if p.name.startswith(".aibox_tmp_")]
assert leftovers == []
def test_normal_write_no_retry(self, tmp_path):
target = tmp_path / "ok.txt"
with mock.patch("os.replace", wraps=os.replace) as spy:
atomic_write_text(str(target), "hello")
spy.assert_called_once()
assert target.read_text(encoding="utf-8") == "hello"
+180 -8
View File
@@ -15,8 +15,10 @@ tools.basic_tools - 基础工具集(A 批抽离)
依赖方向:common <- tools.basic_tools,不反向依赖 aibox 任何符号。 依赖方向:common <- tools.basic_tools,不反向依赖 aibox 任何符号。
""" """
import base64
import os import os
import queue import queue
import re
import shlex import shlex
from datetime import datetime from datetime import datetime
from typing import Dict, Optional, Tuple from typing import Dict, Optional, Tuple
@@ -25,6 +27,40 @@ from common import ConfigManager, constants
from .base import Tool from .base import Tool
# T3(审计 2026-08-21):cmd.exe 变量展开形态——%VAR% 匹配已定义环境变量时
# 展开为变量值,值中的 & | 会成为命令分隔符(经典 cmd 注入)。URL 编码
# (%20/%E4 等)与 PowerShell 的 %{http_code} 均不匹配该模式,不受影响。
_WIN_VAR_PATTERN = re.compile(r"%[A-Za-z_][A-Za-z0-9_]*%")
def _quote_arg(value: str) -> Tuple[str, Optional[str]]:
"""平台感知的 shell 参数引用,返回 (quoted, error)。
POSIX:shlex.quote(sh 单引号语义)。
Windows(run_command 经 cmd.exe / shell=True 执行):
- 双引号包裹(& | < > ( ) ^ 在双引号内为字面量)
- 含 " → 拒绝(闭合外层双引号,其后内容暴露给 cmd 解析)
- 匹配 %VAR% 形态 → 拒绝(cmd 变量展开注入面)
- 含换行/控制字符 → 拒绝(cmd 命令行不可含换行)
"""
value = str(value)
if os.name != "nt":
return shlex.quote(value), None
if '"' in value:
return "", f"❌ 参数含双引号,无法安全生成 Windows 命令: {value[:100]}"
if "\n" in value or "\r" in value:
return "", f"❌ 参数含换行符,无法生成命令: {value[:100]}"
if _WIN_VAR_PATTERN.search(value):
return "", (f"❌ 参数疑似含 cmd 变量引用(%变量名%),已拒绝以防命令注入: "
f"{value[:100]}")
return f'"{value}"', None
def _ps_quote(value: str) -> str:
"""PowerShell 单引号字面量(内部单引号翻倍),用于 -Command 内层参数。"""
return "'" + str(value).replace("'", "''") + "'"
# ==================== 文件操作工具基类 ==================== # ==================== 文件操作工具基类 ====================
class FileToolBase(Tool): class FileToolBase(Tool):
@@ -106,9 +142,54 @@ class ReadFileTool(FileToolBase):
return "❌ 参数错误: start_line/end_line 必须为整数" return "❌ 参数错误: start_line/end_line 必须为整数"
start_line = max(1, start_line) start_line = max(1, start_line)
end_line = max(0, min(end_line, 10_000_000)) end_line = max(0, min(end_line, 10_000_000))
quoted = shlex.quote(abs_path)
# 构造等价的命令行读取命令 # T3(审计 2026-08-21):shlex.quote 是 POSIX 单引号语义,Windows 上
# run_command 经 cmd.exe 执行——单引号不被识别,& | % < > 仍是元字符
# (URL 含 & 即命令被截断;%VAR% 变量展开可注入)。按平台生成指引:
# POSIX 保持 head/tail/sed,Windows 走 PowerShell 等价命令。
quoted, qerr = _quote_arg(abs_path)
if qerr:
return qerr
if os.name == "nt":
# PowerShell 等价命令(经 cmd 调 powershell,-Command 整体双引号
# 包裹,内层路径用 _ps_quote 单引号字面量)
if head:
cmd = (f'powershell -NoProfile -Command "Get-Content -Path '
f'{_ps_quote(abs_path)} -TotalCount {max_lines}"')
wc_cmd = (f'powershell -NoProfile -Command "(Get-Content '
f'{_ps_quote(abs_path)} | Measure-Object -Line).Lines"')
elif tail:
cmd = (f'powershell -NoProfile -Command "Get-Content -Path '
f'{_ps_quote(abs_path)} -Tail {max_lines}"')
wc_cmd = (f'powershell -NoProfile -Command "(Get-Content '
f'{_ps_quote(abs_path)} | Measure-Object -Line).Lines"')
else:
if end_line > 0:
first = start_line - 1
take = end_line - start_line + 1
elif max_lines > 0:
first = start_line - 1
take = max_lines
else:
first = start_line - 1
take = 10_000_000
cmd = (f'powershell -NoProfile -Command "Get-Content -Path '
f'{_ps_quote(abs_path)} | Select-Object -Skip {first} '
f'-First {take}"')
wc_cmd = (f'powershell -NoProfile -Command "(Get-Content '
f'{_ps_quote(abs_path)} | Measure-Object -Line).Lines"')
prompt = f"""请使用 `run_command` 工具执行以下命令来读取文件内容,不要直接返回文件内容:
```
{cmd}
```
- 若需统计总行数:`{wc_cmd}`
- 文件大小上限 {constants.MAX_FILE_SIZE} 字节,单次最多读取 500 行"""
return prompt
# 构造等价的命令行读取命令(POSIX)
if head: if head:
cmd = f"head -n {max_lines} {quoted}" cmd = f"head -n {max_lines} {quoted}"
elif tail: elif tail:
@@ -180,7 +261,40 @@ class WriteFileTool(FileToolBase):
if not is_safe: if not is_safe:
return error return error
quoted = shlex.quote(abs_path) quoted, qerr = _quote_arg(abs_path)
if qerr:
return qerr
if os.name == "nt":
# T3:Windows 下 heredoc 不可用(cmd 命令行不能含换行,且引号
# 转义跨 cmd/PowerShell 两层极易出错)。content 整体 Base64 编码
# 后经 PowerShell 解码写入——Base64 字符集 [A-Za-z0-9+/=] 不含
# 任何 cmd/PowerShell 元字符,多行/引号/特殊字符内容全覆盖。
b64 = base64.b64encode(content.encode("utf-8")).decode("ascii")
if len(b64) > 6000:
return ("❌ 写入内容过大:Windows 命令行长度上限约 8191 字符,"
"Base64 编码后超出。请将内容分成多次调用(每次 ≤4KB),"
"首次 mode=w、后续 mode=a 追加写入")
writer = "Add-Content" if mode == "a" else "Set-Content"
write_cmd = (f'powershell -NoProfile -Command "{writer} -Path '
f'{_ps_quote(abs_path)} -Value ([Text.Encoding]::UTF8.'
f'GetString([Convert]::FromBase64String(\'{b64}\')))"')
directory = os.path.dirname(abs_path)
if directory:
write_cmd = (f'powershell -NoProfile -Command "New-Item '
f'-ItemType Directory -Force -Path '
f'{_ps_quote(directory)}" & {write_cmd}')
prompt = f"""请使用 `run_command` 工具执行以下命令来写入文件,不要在此工具中直接写入:
```
{write_cmd}
```
- 写入内容已编码为 Base64(避免换行/引号/特殊字符的跨 cmd+PowerShell 转义问题),命令会自动解码并以 UTF-8 写入
- 若需检查文件大小:`powershell -NoProfile -Command "(Get-Item {_ps_quote(abs_path)}).Length"`
- 文件大小上限 {constants.MAX_FILE_SIZE} 字节"""
return prompt
directory = os.path.dirname(abs_path) directory = os.path.dirname(abs_path)
dir_cmd = f"mkdir -p {shlex.quote(directory)}" if directory else "" dir_cmd = f"mkdir -p {shlex.quote(directory)}" if directory else ""
@@ -254,7 +368,26 @@ class ListFilesTool(FileToolBase):
if not os.path.isdir(abs_path): if not os.path.isdir(abs_path):
return f"❌ 路径不是目录: {abs_path}" return f"❌ 路径不是目录: {abs_path}"
quoted = shlex.quote(abs_path) quoted, qerr = _quote_arg(abs_path)
if qerr:
return qerr
if os.name == "nt":
# T3:Windows 走 PowerShell 等价命令(ls/du/find/file 无原生等价物)
force_flag = " -Force" if show_hidden else ""
cmd = (f'powershell -NoProfile -Command "Get-ChildItem{force_flag} '
f'{_ps_quote(abs_path)} | Select-Object '
f'Mode,LastWriteTime,Length,Name"')
prompt = f"""请使用 `run_command` 工具执行以下命令来列出目录内容,不要直接在此工具中列出:
```
{cmd}
```
- 若需统计目录总大小:`powershell -NoProfile -Command "(Get-ChildItem -Recurse {_ps_quote(abs_path)} | Measure-Object Length -Sum).Sum"`
- 若需递归列出:`powershell -NoProfile -Command "Get-ChildItem -Recurse -Depth 2 {_ps_quote(abs_path)} | Select-Object FullName"`"""
return prompt
flag = "-la" if show_hidden else "-l" flag = "-la" if show_hidden else "-l"
# 返回 markdown 提示词,指导智能体用命令行工具列目录 # 返回 markdown 提示词,指导智能体用命令行工具列目录
@@ -324,7 +457,9 @@ class SimpleWebCrawlerTool(Tool):
host_port = proxy_url.split("://", 1)[-1] host_port = proxy_url.split("://", 1)[-1]
if "@" in host_port: if "@" in host_port:
host_port = host_port.split("@", 1)[-1] host_port = host_port.split("@", 1)[-1]
return f"--socks5 {shlex.quote(host_port)}" # T3:平台感知引用(原 shlex.quote 在 Windows cmd.exe 下无效)
quoted, _err = _quote_arg(host_port)
return f"--socks5 {quoted}" if quoted else ""
def execute(self, action: str, url: str, **kwargs) -> str: def execute(self, action: str, url: str, **kwargs) -> str:
if action not in ("get", "text", "title", "links", "search"): if action not in ("get", "text", "title", "links", "search"):
@@ -336,10 +471,44 @@ class SimpleWebCrawlerTool(Tool):
if not url or not str(url).strip().lower().startswith(("http://", "https://")): if not url or not str(url).strip().lower().startswith(("http://", "https://")):
return "❌ 仅支持 http/https 协议地址(已拦截 file:// 等非网络协议)" return "❌ 仅支持 http/https 协议地址(已拦截 file:// 等非网络协议)"
quoted_url = shlex.quote(url) quoted_url, qerr = _quote_arg(url)
if qerr:
return qerr
timeout = constants.WEB_TIMEOUT timeout = constants.WEB_TIMEOUT
user_agent = shlex.quote(constants.WEB_USER_AGENT) user_agent, ua_err = _quote_arg(constants.WEB_USER_AGENT)
if ua_err:
return ua_err
proxy = self._proxy_curl_args() proxy = self._proxy_curl_args()
if os.name == "nt":
# T3:Windows 用系统内置 curl.exe(Win10 1803+);sed/grep 等
# POSIX 管道无原生等价,分析类动作(text/title/links/search)
# 指引先取原始 HTML 再由 AI 自行解析。URL 含 & 时靠双引号
# 保护(cmd 双引号内为字面量),%VAR% 形态已在 _quote_arg 拒绝。
base = f"curl.exe -sL --max-time {timeout} -A {user_agent} {proxy}".strip()
if action == "get":
cmd = f'{base} -w "\\nHTTP_CODE:%{{http_code}}" {quoted_url}'
extra = "如需查看响应头:`curl.exe -sIL ...`;如需保存到文件:`curl.exe -sL ... -o 文件名`"
elif action == "text":
cmd = (f'{base} {quoted_url} | powershell -NoProfile -Command '
f'"$input -replace \'<[^>]+>\',\'\'"')
extra = "去除 HTML 标签提取纯文本"
else:
cmd = f"{base} {quoted_url}"
extra = ("Windows 下无 grep/sed 管道,请取回原始 HTML 后自行分析"
"(提取标题/链接/检索关键词)")
prompt = f"""请使用 `run_command` 工具执行以下命令来完成网页操作,不要直接返回网页内容:
```
{cmd}
```
{extra}
- 下载大小限制 {constants.WEB_MAX_DOWNLOAD_SIZE} 字节,可在命令后追加 `| powershell -NoProfile -Command "$input | Select-Object -First 100"` 限制
- 当前已按配置自动附带代理:{proxy or '无(如需代理请先调用 set_proxy 工具)'}"""
return prompt
# POSIX 分支:_quote_arg 在 POSIX 下即 shlex.quote,直接沿用
base = f"curl -sL --max-time {timeout} -A {user_agent} {proxy}".strip() base = f"curl -sL --max-time {timeout} -A {user_agent} {proxy}".strip()
if action == "get": if action == "get":
@@ -358,7 +527,10 @@ class SimpleWebCrawlerTool(Tool):
keyword = kwargs.get("keyword") keyword = kwargs.get("keyword")
if not keyword: if not keyword:
return "❌ 需要提供 keyword 参数" return "❌ 需要提供 keyword 参数"
cmd = f"{base} {quoted_url} | grep -c {shlex.quote(keyword)}" quoted_kw, kw_err = _quote_arg(keyword)
if kw_err:
return kw_err
cmd = f"{base} {quoted_url} | grep -c {quoted_kw}"
extra = f"返回匹配行数,大于 0 即表示找到关键词 '{keyword}'" extra = f"返回匹配行数,大于 0 即表示找到关键词 '{keyword}'"
# 返回 markdown 提示词,指导智能体用 curl 完成网页访问 # 返回 markdown 提示词,指导智能体用 curl 完成网页访问
+22 -4
View File
@@ -1283,9 +1283,9 @@ FTS5 `MATCH` 左操作数必须是 FROM 子句中的表名或别名。`memories_
八轮累计发现 **166+ 项**(高危 18 / 中危 54 / 低危与信息 94+);前七轮 118 项全部修复落地(Batch E/F/G/H/I/K/L/M),第八轮 T 系列待落地(Batch N 建议见 §23.7)。 八轮累计发现 **166+ 项**(高危 18 / 中危 54 / 低危与信息 94+);前七轮 118 项全部修复落地(Batch E/F/G/H/I/K/L/M),第八轮 T 系列待落地(Batch N 建议见 §23.7)。
## 二十四、Batch N 修复记录(波次 1~2,2026-08-21) ## 二十四、Batch N 修复记录(波次 1~3,2026-08-21)
范围:§23.7 波次 1(速赢 7 项)+ 波次 2(并发正确性 6 项),共 13 项。 范围:§23.7 波次 1(速赢 7 项)+ 波次 2(并发正确性 6 项)+ 波次 3(平台兼容 5 项),共 18 项。
### 24.1 波次 1 落地(速赢批) ### 24.1 波次 1 落地(速赢批)
@@ -1314,6 +1314,24 @@ FTS5 `MATCH` 左操作数必须是 FROM 子句中的表名或别名。`memories_
波次 1 后:**532 passed / 4 failed / 1 skipped**(43.8s);波次 2 后:**532 passed / 4 failed / 1 skipped**(41.5s)。4 失败均为 §23.8 所列同一批 Windows 平台环境项,零回归。 波次 1 后:**532 passed / 4 failed / 1 skipped**(43.8s);波次 2 后:**532 passed / 4 failed / 1 skipped**(41.5s)。4 失败均为 §23.8 所列同一批 Windows 平台环境项,零回归。
### 24.4 待落地(波次 3~5) ### 24.4 待落地(波次 4~5)
波次 3 平台兼容(T3/T11/T25/T26/T27)、波次 4 LLM 健壮性(T10/T14/T19/T20/T21/T23,其中 T14 需先写 FTS5 复现用例)、波次 5 权限隔离(T1/T2/T22,含产品决策项)。 波次 4 LLM 健壮性(T10/T14/T19/T20/T21/T23,其中 T14 需先写 FTS5 复现用例)、波次 5 权限隔离(T1/T2/T22,含产品决策项)。
### 24.5 波次 3 落地(平台兼容批,2026-08-21)
范围:T3/T11/T25/T26/T27,共 5 项。
| # | 修复内容 | 文件/位置 |
|---|---|---|
| T3 | `basic_tools.py` 新增平台感知引用 `_quote_arg()`(POSIX=shlex.quote;Windows=双引号包裹 + 拒绝 `"`/换行/`%变量名%` 形态——cmd 变量展开是经典注入向量,URL 编码 %20 与 PowerShell `%{http_code}` 不匹配该模式不受影响)与 `_ps_quote()`(PowerShell 单引号字面量);四个工具按 `os.name` 分支生成指引:ReadFile 走 Get-Content/Select-Object(head/tail/行范围/wc 等价),WriteFile 走 Base64+Set-Content/Add-Content(heredoc 在 cmd 下不可用:命令行不能含换行;Base64 字符集不含任何 cmd/PowerShell 元字符,多行/引号/特殊字符全覆盖;超 6000 字符提示分块追加),ListFiles 走 Get-ChildItem,Web 走系统内置 curl.exe(分析类动作 text 给 PowerShell 管道等价,title/links/search 指引取原始 HTML 自行分析);`_proxy_curl_args`/search keyword 引用同步切换 | `tools/basic_tools.py:33-78,150-237,297-343,389-414,474-533` |
| T11 | executors 与 process_manager 两处 `_terminate_tree` Windows 分支改 `taskkill /PID {pid} /T /F` 递归终止整棵进程树(无 /F 仅对 GUI 进程有效,控制台进程杀不掉,故统一强制;force 参数在 Windows 不区分),taskkill 失败(进程已退出/无权限)兜底单进程 kill;POSIX 分支行为不变 | `subsystems/executors.py:109-157`;`subsystems/process_manager.py:604-634` |
| T25 | MemoryDatabase 新增 `_connect()` 统一连接工厂(`PRAGMA busy_timeout = 10000`),7 处直连 `sqlite3.connect` 全部换走工厂——与 MemoDatabase J9 修复对齐,两类共库时锁等待策略一致,CLI `--memo-add` 与服务进程并发写不再抛 `database is locked` | `subsystems/db.py:36-45`(工厂)+ 7 处调用点 |
| T26 | `SkillManager._load_skills` 本地构建 `new_skills` 完成后一次性原子替换 `self.skills`(对齐 PromptManager M6)——reload 期间(每个可执行技能 `--help` 最多 10s)并发读者不再看到空/半截技能表;目录不存在分支同样保持显式赋值语义 | `subsystems/skill_manager.py:125-213` |
| T27 | `atomic_write_text` 的 `os.replace` 捕获 `PermissionError` 做 3 次短退避重试(50/100ms)——Windows 下目标被杀毒扫描/编辑器/并发读者占用时一次冲突不再直接丢该条持久化(conversation_history 每条消息都走此路径);重试耗尽上抛原异常,临时文件清理逻辑不变;POSIX 下 `os.replace` 无条件成功不受影响 | `common/atomic.py:15-52` |
**Windows 分支验证方式注记**:Linux 环境以 `mock.patch("os.name", "nt")` 模拟验证命令生成逻辑(引用规则/拒绝路径/命令形态),生成的 PowerShell/curl.exe 命令未在真实 Windows 上执行验证,Windows 开发机建议抽测指引命令的实际可用性。
### 24.6 测试基线(Linux,`aibox/venv`)
波次 3 后:**566 passed / 1 failed**(30.1s)。新增 `tests/test_n_fixes.py` 30 项(T3 引用规则 6 + Windows 指引生成 10 + T11 taskkill 4 + T25 busy_timeout 2 + T26 原子 reload 2 + T27 重试 3 + POSIX 锚定 3);1 失败为 §11.3 所列 Windows 向用例(`test_execute_timeout`,本机无 ping 命令),与代码无关,零回归。