qing/blackbird
3
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
803 Commits 2 Branches 0 Tags
f7fc06c0ff7fa79d7f6f782502ea47daf0237217
Commit Graph
11 Commits
Author SHA1 Message Date
qing 457bb900fb fix(aibox): round-5 audit batch K - max_turns clamp, task input validation, XSS hardening, atomic writes, broadcast timeout 
第五轮审计（J1~J16）全量修复 + 子代理 max_turns 钳制：
- 子代理 max_turns 钳制（0/None 回退默认 10，负数钳制 1）+ 14 项回归测试
- J1 任务系统投毒：add_memo 入库校验（时间/间隔/类型）、调度器预检与触发事务
  防御、展示路径兜底、register 移入 try、add_user_task 补 repeat_end_time 透传
- J2/J13 前端 XSS：三页面 marked 输出经 DOMPurify 净化、entry1 系统消息/任务
  分隔条改 textContent、CDN 脚本补 SRI
- J3 FTS 删除换序（先 fts 后内容表，索引不再残留）
- J5 原子写推广：新增 common/atomic.py（mkstemp+fsync+os.replace），config/prompt/
  history 全部接入；损坏配置备份后回退
- J6 单例初始化加锁（PromptManager/ConfigManager 并发首例不双重加载）
- J7 broadcast 并发发送 + wait_for 超时回收僵死连接
- J4/J8/J9~J12/J14~J16：验证异常脱敏、交互输入容错、MemoDatabase WAL+busy_timeout、
  memo 批量删除、/new 竞态、skills 引号转义、voice_client 清理、死代码移除、
  skill_manager 同名键/help 截断
 2026-08-19 07:01:13 +08:00
qing db54921066 fix(aibox): batch I audit fixes - prompt write cap, light-subagent cancel propagation, atomic history writes 
Remaining-item sweep (audit report §12):
- N9: cap prompt file writes (identity/soul/status, 10k chars) to bound
  persistent-injection surface
- F5: run_light_subagent switches to stream_chat + parent cancel_event
  (mid-stream interrupt); ExpressEmotionTool injects parent event
- Q4: busy_timeout=10s on mark_as_triggered (multi-process DB contention)
- Q5: malformed reminder_time tasks returned as due (were silently dropped
  by SQL-level string filter, which is now Python-side authoritative check)
- Q7b: load_conversation realpath prefix check (../ traversal defense)
- Q8: atomic JSON writes (tmp+os.replace) for index/current/archive
- F6: dedicated ws command executor for /new, /reload_prompts
- F9: --once hard-exit backstop after soft timeout
- PromptManager singleton lock; limit=0 semantics (None = no LIMIT)
 2026-08-18 03:09:05 +00:00
qing d3acccd146 fix(aibox): batch H audit fixes - subagent cancel propagation, scheduler timeout, exec process-group reaping 
Fourth audit round (see docs/reports/aibox_audit_20260817.md §11):
- H1: shared parent cancel event no longer cleared on subagent first think
  (N5 regression); parent-already-cancelled skips subagent start
- M1: /reload_prompts moved off event loop (run_in_executor, N3 sibling)
- M2: skill --help scan prewarmed before serving connections
- M3: scheduler task-level timeout (scheduler.task_timeout, default 600s)
- M4: get_recent_memories(None) capped instead of full-table load
- M5: memo delete now physically removes rows (was complete-only)
- M6: PromptManager.reload builds new list then atomically swaps
- L1-L12: read_file arg coercion, proxy save failure reported, SSE
  utf-8 tolerance, persona realpath check, truncation char boundary,
  compressor div0, skill path quoting, ask_user required, POSIX
  process-group kill (pgid captured at Popen), web scheme whitelist,
  registry redact substring match, Bearer redaction in failover logs
 2026-08-18 02:33:17 +00:00
qing cccc946660 fix(aibox): batch G audit fixes - llm hardening, thread lifecycle, cancel propagation, db transactions 
P-series (llm_client): truncate non-transient HTTP error body to 200 chars, cap Retry-After at 3600s, validate fallback profiles (missing model skipped with warning), dedupe fallback list, remove dead _cancelled flag, document FailoverClient concurrency contract.

N-series (aibox/registry): batch-level drain thread with try/finally reclamation (N2), subagent cancel propagation via shared parent event + watcher thread (N5), /new reset moved off event loop (N3), streaming forward bounded by TOOL_MAX_RESULT_LENGTH (N8), persona path boundary check (N12), broadcast except narrowed + discard (N13), per-connection task set registration (N14).

db/history: FTS5 quote escaping (N7), mark_as_triggered single BEGIN IMMEDIATE transaction (N10), LIMIT parameterization (L1), archive filename collision suffix (N16).

Tests: +20 cases in test_g_fixes.py, full suite 375 passed.
 2026-08-18 07:21:29 +08:00
qing 5bcf22f9a7 fix(aibox): registry timeout cleanup + reader thread finalization (F2/N6+N4) 
N6: both streaming-forward timeout paths (deadline hit, join(5)
expiry) previously returned early and skipped the
tool.streaming_output / tool.output_queue cleanup, leaving the daemon
thread writing through the instance attribute — zombie output could
leak into the next session's queue once the attribute got reassigned.
Now unified through a timed_out flag with cleanup at the single exit;
stream_tool_timeout becomes an instance attribute (3600s default,
test-tunable).

N4: reader-thread finalization after timeout/grace break. First
implementation followed the audit suggestion (close stdout + join),
but PoC regression proved close() blocks: CPython stream objects hold
an internal lock, so closing while readline is blocked parks until
readline returns (PoC-1 regressed 3s -> 15.2s, exactly the
Start-Sleep duration). Corrected strategy: when reader already got
EOF, close stdout to release the pipe fd; when reader is still
blocked, only join(timeout=2) then give up (daemon-bounded, thread
ends naturally when the grandchild exits and EOF arrives), keeping
the main flow prompt. Audit report updated with this correction.

Tests: new tests/test_f2_fixes.py (5 cases), PoC regression 5/5
(PoC-1/4 back to ~5s), full suite 272 passed.
 2026-08-17 22:47:37 +08:00
qing 177a9ac9a6 fix(aibox): close audit findings H1-H3/M1-M2 (command exec protection chain) 
- H1: CommandLineTool.execute_streaming now purely delegates to injected
  executor; removes 150-line duplicate Popen loop that bypassed 1MB cap,
  cancel() and timeout protections
- H2: thread the blocking readline into a daemon reader thread; main loop
  polls queue with 0.1s ticks so timeout/1MB checks fire even for silent
  commands (verified: 3s timeout on 15s sleep, was 15s+ before)
- H3: stdin=DEVNULL on both Popen sites; stdin-waiting commands now hit
  EOF instead of deadlocking the tool thread forever
- M1: heredoc delimiter uniquification in write_file guidance (content
  line equal to AIBOX_EOF no longer truncates into shell injection)
- M2: 3600s total deadline on registry streaming forward loop

Verification: PoC regression 5/5 PASS (timeout/1MB/cancel/delegation),
full unit suite 264 passed. Audit report updated with Batch E results.
 2026-08-17 17:28:41 +08:00
qing a5af4bb404 feat(aibox): batch C once mode hardening 
- SQLite WAL：subsystems/db.py 建库时 PRAGMA journal_mode=WAL，缓解多 CLI 进程并发写锁
- --worker-only CLI 参数：纯执行面模式（tool_profile=worker），注入精简 system prompt 避免人格/身份提示词污染
- --model / --temperature 覆盖参数：DeepSeekChat 构造透传，create_llm_client 与 stream_chat 两处生效
- 子代理工具失败透传：run_full_subagent(outcome) → SubAgentTool 打 _SUBAGENT_TOOL_FAILED_MARKER → _handle_tool_calls 检测置 orchestrator stats.tool_failed 并剥离标记
- Batch C 单测 5 项：worker profile 工具集、model/temperature 构造、子代理失败标记透传与剥离
 2026-08-17 16:32:22 +08:00
qing 7f21b65428 fix(aibox): FTS5 中文检索 + Token 压缩阈值生效——Compressor 抽离解耦 
- db.py: FTS5 改用 trigram 分词（中文可检索），旧 unicode61 表自动迁移回灌；search_memories 增加 LIKE 兜底（%/_ 转义、MATCH 异常防护）
- conversation_compressor.py: should_compress 补齐 Token 阈值判断（原为死配置），Token 触发时按比例压缩早期消息
- ConversationCompressor 抽离至 subsystems，注入 llm_client 打破与 DeepSeekChat 循环引用；aibox.py 3517→3018 行
- 新增 21 项测试（中文检索/旧表迁移/LIKE 转义/token 估算/压缩流程/失败回退），全量 232 项通过
 2026-08-15 09:51:59 +08:00
qing 85133868f3 refactor(aibox): 抽离ConversationHistory至subsystems(Wave2阶段1) 
- 新增 subsystems/conversation_history.py: 迁移对话历史管理器(318行)，MemoryDatabase用TYPE_CHECKING注解避免循环导入
- subsystems/__init__.py: 导出ConversationHistory，更新docstring含Wave2
- aibox.py: 移除ConversationHistory类定义(3706->3388行)，仅保留import与类型注解
- 新增 tests/test_conversation_history.py: 41项单测，覆盖构造/当前对话/归档/历史加载/memory_db集成
- 测试基线: 171->212项全过(12.13s)，无回归
 2026-08-13 16:54:32 +08:00
qing 791fac0efd refactor(aibox): 抽离B批12个工具类至tools包，附58个单测 
B批（tools/）：迁移依赖子系统的12个工具类（CommandLineTool/SearchSkills/ReloadSkills/InstallSkill/CreateSkill/MemoryTool/MemoTool/UpdateIdentity/UpdateSoul/SaveMemoryAndEndConversation/SetProxy/GetProxy/AskUser），aibox.py减少~1739行

修复MemoryDatabase.get_recent_memories(limit=None)的SQL错误（既有bug，None传给LIMIT ?导致datatype mismatch）

171个单测全绿，循环依赖障碍消除：B批工具直接依赖subsystems而非反向依赖aibox.py
 2026-08-13 16:09:31 +08:00
qing eadba81c26 refactor(aibox): 抽离工具类与子系统，附单测基线 
A 批（tools/basic_tools.py）：迁移 7 个无外部依赖工具类（FileToolBase/ReadFileTool/WriteFileTool/ListFilesTool/SimpleWebCrawlerTool/GetCurrentTimeTool/CalculatorTool）

Wave1（subsystems/）：迁移 5 个叶子子系统（CommandExecutor/SubprocessCommandExecutor/PromptManager/MemoryDatabase/MemoDatabase/SkillManager），aibox.py 减少 ~1440 行

新增 tests/ 单测 113 个用例全绿，依赖方向维持 common -> subsystems -> tools -> aibox 单向
 2026-08-13 15:48:55 +08:00