qing/blackbird
3
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
822 Commits 2 Branches 0 Tags
dbfb925f656d5022f7d4221565800cf21476ef4e
Commit Graph
16 Commits
Author SHA1 Message Date
qing 67acae708c fix(aibox): Batch N 波次4+5 LLM 健壮性与权限隔离，Batch N 收束 
波次 4 LLM 健壮性（T10/T19/T20/T21/T23 + T14 复现证伪）：
- T10 调度器 await 改 wait_for 硬超时兜底（task_timeout+30s 宽限），
  软取消够不到的卡死路径不再永久停摆调度循环
- T19 FailoverClient 异常收窄为瞬时白名单（Connection/Timeout/
  ChunkedEncoding），MissingSchema 等永久配置错误不再污染冷却表
- T20 流式未收到 [DONE] 抛 TransientLLMError，回答无声截断可感知
- T21 env 覆盖黑名单 filter_env_overrides（LD_*/DYLD_*/PATH 等），
  executors 两处 + process_manager 接入
- T23 压缩后 tool 配对双向修复（失配 tool_calls 剥离防 API 400）
- T14 FTS5 复现证伪：JOIN 别名场景表名 MATCH 正常，无需修复

波次 5 权限隔离（T22/T-L17/T-L18；T1/T2 产品决策：仅告警/维持现状）：
- T22a TOOL_PROFILE_TASK 定时任务精简工具面（无 update_identity/
  update_soul/set_proxy）
- T22b ProcessManager owner 隔离：bg_write/bg_kill 属主校验，
  观察面共享；DeepSeekChat 新增 owner_id 传导
- T1 绑定非回环地址时启动显著告警
- T-L17 model_config env 名按域名推荐（防跨厂商凭据外发）
- T-L18 list_profiles 过滤控制字符（防 ANSI 注入）

测试：test_n_fixes.py 增至 62 项；全量 598 passed / 1 failed
（Windows 向 ping 用例，环境项），零回归。审计报告补记 §24.7~24.11，
README 同步 owner 隔离语义。八轮审计 T 系列 28 项全部处置完毕。
 2026-08-21 14:53:42 +00:00
qing 2dc79f0637 fix(aibox): Batch N 波次3 平台兼容（T3/T11/T25/T26/T27） 
- T3 basic_tools 平台感知引用与 Windows 指引：_quote_arg 按 os.name
  分支（Windows 双引号包裹，拒绝双引号/换行/%变量名% 注入形态）；
  ReadFile/WriteFile/ListFiles/Web 四工具 Windows 走 PowerShell/
  curl.exe 等价命令（WriteFile 用 Base64 规避 cmd 换行/转义限制）
- T11 executors 与 process_manager 的 _terminate_tree Windows 分支
  改 taskkill /T /F 递归收割进程树，兜底单进程 kill
- T25 MemoryDatabase 统一 _connect 工厂（busy_timeout=10000），
  7 处直连替换，对齐 MemoDatabase J9
- T26 SkillManager._load_skills 原子替换 self.skills（对齐
  PromptManager M6），reload 期间不再读到空/半截技能表
- T27 atomic_write_text 对 PermissionError 做 3 次退避重试

测试：新增 tests/test_n_fixes.py 30 项；全量 566 passed / 1 failed
（Windows 向 ping 用例，环境项），零回归。审计报告补记 §24.5/§24.6。
 2026-08-21 13:56:33 +00:00
qing 5894e64287 fix(aibox): Batch N 波次1~2——第八轮审计 T 系列速赢与并发正确性修复 
波次1 速赢批（7 项）：
- T8 ProcessManager watermark 取号与回显记账原子化（entry.lock 改 RLock）
- T9 LLM 非 200 响应抛出前 close，流式路径清 _current_response
- T13 _sanitize_log_msg 增加裸 sk- key 脱敏，异常构造点统一过脱敏
- T12 流式判定改共享 saw_complete 标志（排空后 empty() 恒 False 死逻辑），
  超长截断前移堵住非流式直发绕过 N8 输出预算
- T16/T17 三处 Popen 显式 encoding=utf-8 + errors=replace
- T18 detach/log_file/force 统一过 _coerce_bool（字符串 "false" 误判）

波次2 并发正确性批（6 项）：
- T4 registry 输出队列入口快照，不再向共享工具实例传播（跨会话串扰）
- T5 executor 单槽 _current_process 改令牌多槽注册表，_terminate_tree
  显式传 (proc, pgid)，finally 只注销自己槽位（防并发误杀进程组）
- T6 FailoverClient 冷却键改 id(client)，与 P6 去重键语义一致
- T7 stream_chat 拆普通方法+内部生成器，消除生成器延迟副作用清除取消标志
- T15 工具结果统一 TOOL_MAX_RESULT_LENGTH 截断（两条分支同口径）
- T24 非流式 execute() timeout 下限夹取 1s（该路径无 3600s 兜底）

测试：532 passed / 4 failed（平台环境项）/ 1 skipped，零回归
审计报告补记 §23 第八轮审计 + §24 Batch N 波次1~2 修复记录
 2026-08-21 17:33:33 +08:00
qing 5cb35945a4 feat(aibox): 后台进程池 v3.8——智能体运行/管理长驻进程与交互式 I/O 
新增 ProcessManager 子系统 + 5 个 worker 执行面工具
（bg_run/bg_write/bg_list/bg_output/bg_kill），弥补 run_command
同步阻塞、超时即杀的不足；run_command 原路径零改动。

- 生命周期三决策：用户中断不影响后台进程；aibox 退出经 atexit
  收尾（非 detach 全杀，detach=true 豁免）；注册表跨会话/跨子代理
  全局共享
- 交互式 I/O：stdin 管道 + watermark 写读对齐（bg_output(since=
  watermark) 必然且仅为本条输入的响应）；wait_lines expect 式同步
  等待；写入 watchdog 线程兜底防管道写满阻塞
- 输出缓冲：每进程 256KB 环形缓冲（单行截断 16KB）；log_file=true
  同步 tee 无界全档，可被原生工具 tail/cat 读取
- 三级优雅终止：关 stdin → SIGTERM 宽限 → SIGKILL 兜底
- 安全边界：运行上限 16/条目上限 32/单次写入 64KB；background_
  processes 配置节双闸；proc_name 规整规避 execute_tool 首参撞名
- 测试：test_process_manager.py 39 项（生命周期/游标续读/环形淘汰/
  watermark 对齐/EOF 优雅终止/log tee/工具分发），全量 532 passed
  / 4 failed（平台环境项，与基线一致）零回归
 2026-08-21 11:17:11 +08:00
qing d1357ae483 fix(aibox): Batch M 波次4 命令面输入规整（W1~W8/W11/W12+W5） 
- W1: WS 三入口 action/command/content 非 str 规整；task title/content 类型拦截
- W2: task add delay float 转换+[0,10年]钳制+友好 error
- W3: task complete 缺 task_id 回 error（新增 i18n 键）
- W4: /search 空关键词回用法提示，不再 LIKE %% 全表刷屏
- W5: main() 暂存 _cli_overrides，/reload 重放并回包注明，不再静默丢 CLI 覆盖
- W6: 启动失败 sys.exit(1)；--port argparse 1~65535 校验
- W7: --scheduler-interval 统一 max(0.1,v) 钳制
- W8: --temperature main() 入口 [0.0,2.0] 钳制
- W11: config system/scheduler 节非 dict 回退默认节
- W12: --memo-add title strip/空串→None；db.add_memo 标题非空 ValueError

冒烟 12/12 通过；测试 493 passed/4 failed（既有平台项）/1 skipped 零回归
 2026-08-20 13:54:36 +08:00
qing 457bb900fb fix(aibox): round-5 audit batch K - max_turns clamp, task input validation, XSS hardening, atomic writes, broadcast timeout 
第五轮审计（J1~J16）全量修复 + 子代理 max_turns 钳制：
- 子代理 max_turns 钳制（0/None 回退默认 10，负数钳制 1）+ 14 项回归测试
- J1 任务系统投毒：add_memo 入库校验（时间/间隔/类型）、调度器预检与触发事务
  防御、展示路径兜底、register 移入 try、add_user_task 补 repeat_end_time 透传
- J2/J13 前端 XSS：三页面 marked 输出经 DOMPurify 净化、entry1 系统消息/任务
  分隔条改 textContent、CDN 脚本补 SRI
- J3 FTS 删除换序（先 fts 后内容表，索引不再残留）
- J5 原子写推广：新增 common/atomic.py（mkstemp+fsync+os.replace），config/prompt/
  history 全部接入；损坏配置备份后回退
- J6 单例初始化加锁（PromptManager/ConfigManager 并发首例不双重加载）
- J7 broadcast 并发发送 + wait_for 超时回收僵死连接
- J4/J8/J9~J12/J14~J16：验证异常脱敏、交互输入容错、MemoDatabase WAL+busy_timeout、
  memo 批量删除、/new 竞态、skills 引号转义、voice_client 清理、死代码移除、
  skill_manager 同名键/help 截断
 2026-08-19 07:01:13 +08:00
qing db54921066 fix(aibox): batch I audit fixes - prompt write cap, light-subagent cancel propagation, atomic history writes 
Remaining-item sweep (audit report §12):
- N9: cap prompt file writes (identity/soul/status, 10k chars) to bound
  persistent-injection surface
- F5: run_light_subagent switches to stream_chat + parent cancel_event
  (mid-stream interrupt); ExpressEmotionTool injects parent event
- Q4: busy_timeout=10s on mark_as_triggered (multi-process DB contention)
- Q5: malformed reminder_time tasks returned as due (were silently dropped
  by SQL-level string filter, which is now Python-side authoritative check)
- Q7b: load_conversation realpath prefix check (../ traversal defense)
- Q8: atomic JSON writes (tmp+os.replace) for index/current/archive
- F6: dedicated ws command executor for /new, /reload_prompts
- F9: --once hard-exit backstop after soft timeout
- PromptManager singleton lock; limit=0 semantics (None = no LIMIT)
 2026-08-18 03:09:05 +00:00
qing d3acccd146 fix(aibox): batch H audit fixes - subagent cancel propagation, scheduler timeout, exec process-group reaping 
Fourth audit round (see docs/reports/aibox_audit_20260817.md §11):
- H1: shared parent cancel event no longer cleared on subagent first think
  (N5 regression); parent-already-cancelled skips subagent start
- M1: /reload_prompts moved off event loop (run_in_executor, N3 sibling)
- M2: skill --help scan prewarmed before serving connections
- M3: scheduler task-level timeout (scheduler.task_timeout, default 600s)
- M4: get_recent_memories(None) capped instead of full-table load
- M5: memo delete now physically removes rows (was complete-only)
- M6: PromptManager.reload builds new list then atomically swaps
- L1-L12: read_file arg coercion, proxy save failure reported, SSE
  utf-8 tolerance, persona realpath check, truncation char boundary,
  compressor div0, skill path quoting, ask_user required, POSIX
  process-group kill (pgid captured at Popen), web scheme whitelist,
  registry redact substring match, Bearer redaction in failover logs
 2026-08-18 02:33:17 +00:00
qing cccc946660 fix(aibox): batch G audit fixes - llm hardening, thread lifecycle, cancel propagation, db transactions 
P-series (llm_client): truncate non-transient HTTP error body to 200 chars, cap Retry-After at 3600s, validate fallback profiles (missing model skipped with warning), dedupe fallback list, remove dead _cancelled flag, document FailoverClient concurrency contract.

N-series (aibox/registry): batch-level drain thread with try/finally reclamation (N2), subagent cancel propagation via shared parent event + watcher thread (N5), /new reset moved off event loop (N3), streaming forward bounded by TOOL_MAX_RESULT_LENGTH (N8), persona path boundary check (N12), broadcast except narrowed + discard (N13), per-connection task set registration (N14).

db/history: FTS5 quote escaping (N7), mark_as_triggered single BEGIN IMMEDIATE transaction (N10), LIMIT parameterization (L1), archive filename collision suffix (N16).

Tests: +20 cases in test_g_fixes.py, full suite 375 passed.
 2026-08-18 07:21:29 +08:00
qing 5bcf22f9a7 fix(aibox): registry timeout cleanup + reader thread finalization (F2/N6+N4) 
N6: both streaming-forward timeout paths (deadline hit, join(5)
expiry) previously returned early and skipped the
tool.streaming_output / tool.output_queue cleanup, leaving the daemon
thread writing through the instance attribute — zombie output could
leak into the next session's queue once the attribute got reassigned.
Now unified through a timed_out flag with cleanup at the single exit;
stream_tool_timeout becomes an instance attribute (3600s default,
test-tunable).

N4: reader-thread finalization after timeout/grace break. First
implementation followed the audit suggestion (close stdout + join),
but PoC regression proved close() blocks: CPython stream objects hold
an internal lock, so closing while readline is blocked parks until
readline returns (PoC-1 regressed 3s -> 15.2s, exactly the
Start-Sleep duration). Corrected strategy: when reader already got
EOF, close stdout to release the pipe fd; when reader is still
blocked, only join(timeout=2) then give up (daemon-bounded, thread
ends naturally when the grandchild exits and EOF arrives), keeping
the main flow prompt. Audit report updated with this correction.

Tests: new tests/test_f2_fixes.py (5 cases), PoC regression 5/5
(PoC-1/4 back to ~5s), full suite 272 passed.
 2026-08-17 22:47:37 +08:00
qing 177a9ac9a6 fix(aibox): close audit findings H1-H3/M1-M2 (command exec protection chain) 
- H1: CommandLineTool.execute_streaming now purely delegates to injected
  executor; removes 150-line duplicate Popen loop that bypassed 1MB cap,
  cancel() and timeout protections
- H2: thread the blocking readline into a daemon reader thread; main loop
  polls queue with 0.1s ticks so timeout/1MB checks fire even for silent
  commands (verified: 3s timeout on 15s sleep, was 15s+ before)
- H3: stdin=DEVNULL on both Popen sites; stdin-waiting commands now hit
  EOF instead of deadlocking the tool thread forever
- M1: heredoc delimiter uniquification in write_file guidance (content
  line equal to AIBOX_EOF no longer truncates into shell injection)
- M2: 3600s total deadline on registry streaming forward loop

Verification: PoC regression 5/5 PASS (timeout/1MB/cancel/delegation),
full unit suite 264 passed. Audit report updated with Batch E results.
 2026-08-17 17:28:41 +08:00
qing a5af4bb404 feat(aibox): batch C once mode hardening 
- SQLite WAL：subsystems/db.py 建库时 PRAGMA journal_mode=WAL，缓解多 CLI 进程并发写锁
- --worker-only CLI 参数：纯执行面模式（tool_profile=worker），注入精简 system prompt 避免人格/身份提示词污染
- --model / --temperature 覆盖参数：DeepSeekChat 构造透传，create_llm_client 与 stream_chat 两处生效
- 子代理工具失败透传：run_full_subagent(outcome) → SubAgentTool 打 _SUBAGENT_TOOL_FAILED_MARKER → _handle_tool_calls 检测置 orchestrator stats.tool_failed 并剥离标记
- Batch C 单测 5 项：worker profile 工具集、model/temperature 构造、子代理失败标记透传与剥离
 2026-08-17 16:32:22 +08:00
qing 7f21b65428 fix(aibox): FTS5 中文检索 + Token 压缩阈值生效——Compressor 抽离解耦 
- db.py: FTS5 改用 trigram 分词（中文可检索），旧 unicode61 表自动迁移回灌；search_memories 增加 LIKE 兜底（%/_ 转义、MATCH 异常防护）
- conversation_compressor.py: should_compress 补齐 Token 阈值判断（原为死配置），Token 触发时按比例压缩早期消息
- ConversationCompressor 抽离至 subsystems，注入 llm_client 打破与 DeepSeekChat 循环引用；aibox.py 3517→3018 行
- 新增 21 项测试（中文检索/旧表迁移/LIKE 转义/token 估算/压缩流程/失败回退），全量 232 项通过
 2026-08-15 09:51:59 +08:00
qing 85133868f3 refactor(aibox): 抽离ConversationHistory至subsystems(Wave2阶段1) 
- 新增 subsystems/conversation_history.py: 迁移对话历史管理器(318行)，MemoryDatabase用TYPE_CHECKING注解避免循环导入
- subsystems/__init__.py: 导出ConversationHistory，更新docstring含Wave2
- aibox.py: 移除ConversationHistory类定义(3706->3388行)，仅保留import与类型注解
- 新增 tests/test_conversation_history.py: 41项单测，覆盖构造/当前对话/归档/历史加载/memory_db集成
- 测试基线: 171->212项全过(12.13s)，无回归
 2026-08-13 16:54:32 +08:00
qing 791fac0efd refactor(aibox): 抽离B批12个工具类至tools包，附58个单测 
B批（tools/）：迁移依赖子系统的12个工具类（CommandLineTool/SearchSkills/ReloadSkills/InstallSkill/CreateSkill/MemoryTool/MemoTool/UpdateIdentity/UpdateSoul/SaveMemoryAndEndConversation/SetProxy/GetProxy/AskUser），aibox.py减少~1739行

修复MemoryDatabase.get_recent_memories(limit=None)的SQL错误（既有bug，None传给LIMIT ?导致datatype mismatch）

171个单测全绿，循环依赖障碍消除：B批工具直接依赖subsystems而非反向依赖aibox.py
 2026-08-13 16:09:31 +08:00
qing eadba81c26 refactor(aibox): 抽离工具类与子系统，附单测基线 
A 批（tools/basic_tools.py）：迁移 7 个无外部依赖工具类（FileToolBase/ReadFileTool/WriteFileTool/ListFilesTool/SimpleWebCrawlerTool/GetCurrentTimeTool/CalculatorTool）

Wave1（subsystems/）：迁移 5 个叶子子系统（CommandExecutor/SubprocessCommandExecutor/PromptManager/MemoryDatabase/MemoDatabase/SkillManager），aibox.py 减少 ~1440 行

新增 tests/ 单测 113 个用例全绿，依赖方向维持 common -> subsystems -> tools -> aibox 单向
 2026-08-13 15:48:55 +08:00