release: v5.1.4 — 桌面GUI自动化 + AI生图 + Web UI后端

- 新增 gui_auto 插件 (20 操作): Windows UIA 桌面元素识别与操控
- 新增 ai_image 插件: 基于 ai.2x.nz 自然语言生图
- 新增 webui_server.py: WebSocket 实时通信后端
- 重写 webui.html: 大厂设计感 WebSocket 交互界面
- 插件数: 17 → 19, 操作数: 141+ → 161+
- 测试数: 113 → 150+
- 更新 README/CHANGELOG/HELP

CHANGELOG.md

@@ -4,6 +4,48 @@
 
 ---
 
+## v5.1.4 (2026-04-29)
+
+###  新增功能
+
+- **桌面GUI自动化插件** (`gui_auto`, 20 操作) — 基于 Windows UI Automation (UIA) API，AI 可识别和操控桌面应用
+  - 元素树获取 (`snapshot`) — 获取整个屏幕或指定窗口的 UI 元素树，支持自定义遍历深度
+  - 元素交互 — 点击/右键/双击（名称模糊匹配 + 坐标点击），文本输入（中文 Unicode）、清空、发送按键
+  - 窗口管理 — 列出所有窗口 (`windows`)、聚焦指定窗口 (`focus`)
+  - 元素查询 — 查找/查找所有/详情/值/子树/存在检查/状态，共 8 个查询操作
+  - 高级功能 — 等待元素出现（带超时）、高亮闪烁元素（GDI 红色边框）、滚动
+  - MCP/FC 完整定义，AI 通过 Function Calling 直接调用，非 Windows 环境优雅降级
+
+- **AI 生图插件** (`ai_image`) — 基于 ai.2x.nz 自然语言生图接口
+  - 自然语言描述 + 直接 Tag 双模式
+  - 工作流切换、画风选择、分辨率调整
+  - 异步生成 + WebSocket 进度推送
+  - 生成历史记录
+
+- **Web UI 后端** (`webui_server.py`) — WebSocket 实时通信 + 静态文件服务
+  - 纯 Python 实现，基于 `websockets` 库
+  - 支持命令行参数 (`--port` / `--host`)
+  - 与前端 `webui.html` 配合，提供完整的浏览器端 AI 交互体验
+
+###  改进
+
+- **Web UI 前端重写** (`webui.html`) — 从纯静态页面升级为 WebSocket 实时交互
+  - 大厂设计感 UI，响应式布局
+  - 实时消息流、打字机效果
+  - 支持工具调用结果展示
+
+###  文档
+
+- 更新 README.md: 插件列表、项目结构、badge 版本号
+- 更新 CHANGELOG.md: v5.1.4 完整变更记录
+- 更新 HELP.md: 新增 GUI 自动化使用说明
+
+###  测试
+
+- 新增 gui_auto 插件 30 个单元测试（工具信息、MCP 定义、参数转换、错误处理、操作路由）
+
+---
+
 ## v5.1.3 (2026-04-28)
 
 ###  新增功能

HELP.md

@@ -154,6 +154,26 @@ run make build --timeout 60   # 60秒超时
 | `pdf page.pdf` | 保存 PDF |
 | `stop` | 关闭浏览器 |
 
+### 桌面GUI自动化 (`gui_auto`) ⭐ NEW
+
+> 仅支持 Windows，基于 UI Automation API
+
+| 操作 | 说明 |
+|------|------|
+| `snapshot [深度]` | 获取屏幕 UI 元素树 |
+| `snapshot --window 记事本` | 获取指定窗口元素树 |
+| `screenshot [路径]` | 全屏截图 |
+| `windows` | 列出所有窗口 |
+| `focus 记事本` | 聚焦窗口 |
+| `click 确定` | 点击元素（模糊匹配） |
+| `click --xy 500 300` | 点击坐标 |
+| `type 搜索框 Hello` | 在元素中输入文本 |
+| `keys ctrl+s` | 发送按键 |
+| `find 保存` | 查找元素 |
+| `exists 确定` | 检查元素是否存在 |
+| `wait 加载完成 10` | 等待元素出现 |
+| `highlight 确定` | 高亮闪烁元素 |
+
 ### 子 Agent (`sub_agent`)
 
 | 操作 | 说明 |

README.md

@@ -5,11 +5,11 @@
 </p>
 
 <p align="center">
-  <img src="https://img.shields.io/badge/version-5.1.3-blue" alt="version">
+  <img src="https://img.shields.io/badge/version-5.1.4-blue" alt="version">
   <img src="https://img.shields.io/badge/python-3.10+-green" alt="python">
   <img src="https://img.shields.io/badge/license-MIT-orange" alt="license">
-  <img src="https://img.shields.io/badge/tests-113%20passed-brightgreen" alt="tests">
+  <img src="https://img.shields.io/badge/tests-150+%20passed-brightgreen" alt="tests">
-  <img src="https://img.shields.io/badge/plugins-17-blueviolet" alt="plugins">
+  <img src="https://img.shields.io/badge/plugins-19-blueviolet" alt="plugins">
   <img src="https://img.shields.io/badge/MCP-supported-brightgreen" alt="mcp">
 </p>
 
@@ -28,10 +28,10 @@
 -  **Clawli 远程模式** — PC-手机 WebSocket 双向通信，支持内网穿透代理
 -  **定时任务** — 支持相对时间、绝对时间、重复任务，到期自动激活 AI
 -  **系统通知** — 跨平台任务完成通知（Windows / macOS / Linux）
--  **MCP 协议** — 17 个插件全部实现 MCP 定义，支持 Function Calling 原生调用
+-  **MCP 协议** — 19 个插件全部实现 MCP 定义，支持 Function Calling 原生调用
--  **17 个插件 141+ 个操作** — 代码编辑、Git、浏览器、工程化、文件管理、图像生成、记忆等
+-  **19 个插件 161+ 个操作** — 代码编辑、Git、浏览器、工程化、文件管理、图像生成、记忆、桌面GUI自动化等
 -  **三模式界面** — TUI 图形化 + 传统命令行 + **Web UI 浏览器界面**
--  **113 个自动化测试** — pytest 覆盖全部插件和核心模块
+-  **150+ 个自动化测试** — pytest 覆盖全部插件和核心模块
 
 ##  快速开始
 
@@ -87,7 +87,7 @@ launcher.exe
 }
 ```
 
-##  插件列表 (17 个)
+##  插件列表 (19 个)
 
 ### 核心生产力
 
@@ -106,6 +106,8 @@ launcher.exe
 |------|--------|------|
 | `sub_agent` | 8 | 子 Agent 系统。spawn/list/status/result/send/kill，多 Agent 并行协作 |
 | `browser_auto` | 28 | 浏览器自动化。导航/点击/填写/截图/JS执行/PDF/cookies（基于 Playwright） |
+| `gui_auto` | 20 | **桌面GUI自动化**。基于 Windows UIA，屏幕元素识别/点击/输入/截图/元素树遍历（仅Windows） |
+| `ai_image` | — | **AI生图**。基于 ai.2x.nz 自然语言生图，异步生成 + WebSocket 进度推送 |
 | `image_generator` | 7 | **图像生成**。基于 ComfyUI 在线服务，自然语言/Tag 双模式，28 个角色工作流 |
 | `memory_plugin` | 11 | **记忆系统**。AI 写日记/搜索记忆/管理聊天记录/长期记忆 MEMORY.md |
 | `scheduler` | 4 | **定时任务**。支持相对时间/绝对时间/重复任务，到期自动激活 AI |
@@ -186,7 +188,7 @@ memory_plugin chat_search 图像生成
 
 ##  MCP 协议支持
 
-所有 17 个插件均实现 MCP (Model Context Protocol) 定义，支持三种工具调用协议：
+所有 19 个插件均实现 MCP (Model Context Protocol) 定义，支持三种工具调用协议：
 
 | 协议 | 说明 |
 |------|------|
@@ -216,12 +218,17 @@ memory_plugin chat_search 图像生成
 
 ##  Web UI
 
-纯静态 HTML 界面，无需后端，浏览器直接打开。
+WebSocket 实时交互界面，支持完整的 AI 对话体验。
 
 ```bash
+# 方式 1: 启动 WebSocket 后端（推荐）
+python webui_server.py --port 8080
+# 浏览器访问 http://localhost:8080
+
+# 方式 2: 直接打开静态页面（功能有限）
+start webui.html       # Windows
 open webui.html        # macOS
 xdg-open webui.html    # Linux
-start webui.html       # Windows
 ```
 
 ##  项目结构
@@ -229,7 +236,8 @@ start webui.html       # Windows
 ```
 xiaoli-cli/
 ├── ai_cli.py                   # 主入口
-├── webui.html                  # Web UI 界面 (纯静态 HTML)
+├── webui.html                  # Web UI 界面 (WebSocket 实时交互)
+├── webui_server.py             # Web UI 后端 (WebSocket + 静态文件服务)
 ├── launcher.py                 # 智能启动器（自动检测环境/安装依赖）
 ├── config.json.example         # 配置模板
 ├── easter_egg.py               #  彩蛋
@@ -256,7 +264,7 @@ xiaoli-cli/
 │   ├── ollama_engine.py        # Ollama 本地模型
 │   └── openai_engine.py        # OpenAI 兼容格式
 │
-├── plugins/                    # 插件 (17 个)
+├── plugins/                    # 插件 (19 个)
 │   ├── code_editor.py          # 代码编辑+搜索 (24 操作)
 │   ├── git_tools.py            # Git 版本控制 (17 操作)
 │   ├── cmd_executor.py         # Shell 命令执行
@@ -265,6 +273,8 @@ xiaoli-cli/
 │   ├── task_manager.py         # 任务管理
 │   ├── sub_agent.py            # 子 Agent 系统 (8 操作)
 │   ├── browser_auto.py         # 浏览器自动化 (28 操作)
+│   ├── gui_auto.py             # 桌面GUI自动化 (20 操作) ⭐ NEW
+│   ├── ai_image.py             # AI生图 (ai.2x.nz) ⭐ NEW
 │   ├── image_generator.py      # 图像生成 (7 操作) ⭐ NEW
 │   ├── memory_plugin.py        # 记忆系统 (11 操作) ⭐ NEW
 │   ├── scheduler.py            # 定时任务

plugins/ai_image.py

@@ -0,0 +1,590 @@
+"""
+AI 生图插件 - 基于 ai.2x.nz 自然语言生图接口
+支持自然语言描述、直接 Tag、工作流切换、画风选择等
+"""
+
+import os
+import json
+import time
+import threading
+import hashlib
+import re
+from datetime import datetime
+from urllib.parse import quote
+
+try:
+    import requests
+    REQUESTS_AVAILABLE = True
+except ImportError:
+    REQUESTS_AVAILABLE = False
+
+try:
+    import websocket
+    WEBSOCKET_AVAILABLE = True
+except ImportError:
+    WEBSOCKET_AVAILABLE = False
+
+
+class Plugin:
+    """AI 生图插件 - 调用 ai.2x.nz 接口生成图片"""
+
+    BASE_URL = "https://ai.2x.nz"
+    WS_URL = "wss://ai.2x.nz/ws/run"
+    API_URL = "https://ai.2x.nz/api"
+
+    # 保存目录
+    DEFAULT_SAVE_DIR = "generated_images"
+
+    def __init__(self):
+        self.usage = """AI 生图工具 - 基于自然语言描述生成图片
+
+基本用法:
+  generate <描述>                     - 用自然语言描述生成图片
+  tag <直接Tag>                       - 用 Danbooru Tag 生成图片
+  both <Tag> <<<>>> <描述>            - 同时使用 Tag 和自然语言
+
+高级用法:
+  workflows                           - 列出所有可用工作流
+  switch <工作流名>                   - 切换工作流
+  width <数值>                        - 设置图片宽度 (最大1344)
+  height <数值>                       - 设置图片高度 (最大1344)
+  size <宽> <高>                      - 同时设置宽高
+  rewrite on|off                      - 改写模式开关
+  override on|off                     - 覆写模式开关
+  status                              - 查看当前状态和配置
+
+示例:
+  ai_image generate 一个可爱的猫耳少女，樱花背景
+  ai_image tag 1girl, cat_ears, cherry_blossoms, smile
+  ai_image both 1girl, cat_ears <<<>>> 一个可爱的猫耳少女在樱花树下微笑
+  ai_image workflows
+  ai_image switch WAI - 37.json
+  ai_image size 1024 1024
+  ai_image rewrite on
+"""
+        self.cli = None
+        self.save_dir = self.DEFAULT_SAVE_DIR
+
+        # 配置状态
+        self.current_workflow = None
+        self.direct_prompt = ""
+        self.nl_prompt = ""
+        self.rewrite = False
+        self.override = False
+        self.width = None
+        self.height = None
+        self.workflows_cache = None
+        self.is_generating = False
+
+    def set_cli(self, cli):
+        """设置 CLI 实例引用"""
+        self.cli = cli
+        # 注册插件命令
+        if hasattr(cli, 'register_liugin_command'):
+            cli.register_liugin_command('ai_image', self.command_handler)
+
+    def get_tool_info(self):
+        return {
+            "name": "ai_image",
+            "description": "AI 生图工具 — 基于 ai.2x.nz 接口，支持自然语言描述和 Danbooru Tag 生成图片。当用户想要生成图片、画图、AI绘画、生图时使用此工具。",
+            "keywords": [
+                "生图", "画图", "图片", "AI绘画", "生成图片", "image", "generate",
+                "draw", "绘画", "插画", "二次元", "动漫", "Danbooru", "tag",
+                "工作流", "workflow", "ComfyUI"
+            ],
+            "usage": self.usage
+        }
+
+    def get_mcp_definition(self):
+        return {
+            "name": "ai_image",
+            "description": "AI 生图工具，基于 ai.2x.nz 接口生成图片",
+            "inputSchema": {
+                "type": "object",
+                "properties": {
+                    "operation": {
+                        "type": "string",
+                        "enum": ["generate", "tag", "both", "workflows", "switch", "size", "status"],
+                        "description": "操作类型"
+                    },
+                    "prompt": {
+                        "type": "string",
+                        "description": "生成提示词"
+                    }
+                },
+                "required": ["operation"]
+            }
+        }
+
+    def convert_mcp_args(self, arguments):
+        op = arguments.get("operation", "")
+        prompt = arguments.get("prompt", "")
+        return f"{op} {prompt}"
+
+    def command_handler(self, args):
+        """处理 /ai_image 命令"""
+        return self.handle(args)
+
+    def handle(self, args):
+        """处理 AI 生图请求"""
+        if not REQUESTS_AVAILABLE:
+            return "❌ 缺少 requests 库，请运行: pip install requests"
+
+        args = args.strip()
+        if not args:
+            return self.usage
+
+        # 解析操作
+        parts = args.split(maxsplit=1)
+        operation = parts[0].lower()
+        remaining = parts[1] if len(parts) > 1 else ""
+
+        try:
+            if operation in ["generate", "gen", "g"]:
+                return self._cmd_generate(remaining)
+            elif operation in ["tag", "t", "direct", "d"]:
+                return self._cmd_tag(remaining)
+            elif operation in ["both", "b"]:
+                return self._cmd_both(remaining)
+            elif operation in ["workflows", "wf", "list"]:
+                return self._cmd_workflows()
+            elif operation in ["switch", "sw"]:
+                return self._cmd_switch(remaining)
+            elif operation in ["width", "w"]:
+                return self._cmd_set_width(remaining)
+            elif operation in ["height", "h"]:
+                return self._cmd_set_height(remaining)
+            elif operation in ["size", "s"]:
+                return self._cmd_set_size(remaining)
+            elif operation in ["rewrite", "rw"]:
+                return self._cmd_set_rewrite(remaining)
+            elif operation in ["override", "ov"]:
+                return self._cmd_set_override(remaining)
+            elif operation in ["status", "st"]:
+                return self._cmd_status()
+            elif operation in ["help", "?"]:
+                return self.usage
+            else:
+                # 尝试当作自然语言描述处理
+                return self._cmd_generate(args)
+        except Exception as e:
+            return f"❌ 操作失败: {str(e)}"
+
+    # ── 命令实现 ──
+
+    def _cmd_generate(self, prompt):
+        """自然语言生成图片"""
+        if not prompt.strip():
+            return "❌ 请提供描述内容。示例: ai_image generate 一个可爱的猫耳少女"
+
+        self.nl_prompt = prompt.strip()
+        self.direct_prompt = ""
+        self.override = False
+        return self._run_generation()
+
+    def _cmd_tag(self, tags):
+        """使用直接 Tag 生成图片"""
+        if not tags.strip():
+            return "❌ 请提供 Tag 内容。示例: ai_image tag 1girl, cat_ears, smile"
+
+        self.direct_prompt = tags.strip()
+        self.nl_prompt = ""
+        self.override = True
+        return self._run_generation()
+
+    def _cmd_both(self, args):
+        """同时使用 Tag 和自然语言"""
+        if "<<<>>>" in args:
+            parts = args.split("<<<>>>", 1)
+            self.direct_prompt = parts[0].strip()
+            self.nl_prompt = parts[1].strip()
+        else:
+            # 尝试用 || 分隔
+            if "||" in args:
+                parts = args.split("||", 1)
+                self.direct_prompt = parts[0].strip()
+                self.nl_prompt = parts[1].strip()
+            else:
+                return "❌ 格式错误。用法: ai_image both <Tag> <<<>>> <描述>"
+
+        self.override = False
+        return self._run_generation()
+
+    def _cmd_workflows(self):
+        """列出所有工作流"""
+        workflows = self._fetch_workflows()
+        if not workflows:
+            return "❌ 获取工作流列表失败"
+
+        result = f"📂 共 {len(workflows)} 个工作流:\n\n"
+        for i, wf in enumerate(workflows, 1):
+            name = wf.get("path", "未知")
+            current = " ← [当前]" if name == self.current_workflow else ""
+            result += f"  {i:2d}. {name}{current}\n"
+
+        result += f"\n💡 使用 'ai_image switch <工作流名>' 切换工作流"
+        return result
+
+    def _cmd_switch(self, name):
+        """切换工作流"""
+        if not name.strip():
+            return "❌ 请提供工作流名称。先用 'ai_image workflows' 查看列表"
+
+        # 模糊匹配
+        workflows = self._fetch_workflows()
+        if not workflows:
+            return "❌ 获取工作流列表失败"
+
+        matched = None
+        name_lower = name.strip().lower()
+        for wf in workflows:
+            wf_path = wf.get("path", "")
+            if name_lower == wf_path.lower():
+                matched = wf_path
+                break
+            if name_lower in wf_path.lower():
+                matched = wf_path
+
+        if not matched:
+            # 尝试数字索引
+            try:
+                idx = int(name.strip()) - 1
+                if 0 <= idx < len(workflows):
+                    matched = workflows[idx].get("path", "")
+            except ValueError:
+                pass
+
+        if not matched:
+            return f"❌ 未找到匹配的工作流: {name}\n💡 先用 'ai_image workflows' 查看列表"
+
+        self.current_workflow = matched
+        return f"✅ 已切换工作流: {matched}"
+
+    def _cmd_set_width(self, val):
+        """设置宽度"""
+        try:
+            w = int(val.strip())
+            if w < 64 or w > 1344:
+                return "❌ 宽度范围: 64 ~ 1344"
+            self.width = w
+            return f"✅ 宽度已设置为: {w}"
+        except ValueError:
+            return "❌ 请输入有效数字"
+
+    def _cmd_set_height(self, val):
+        """设置高度"""
+        try:
+            h = int(val.strip())
+            if h < 64 or h > 1344:
+                return "❌ 高度范围: 64 ~ 1344"
+            self.height = h
+            return f"✅ 高度已设置为: {h}"
+        except ValueError:
+            return "❌ 请输入有效数字"
+
+    def _cmd_set_size(self, val):
+        """同时设置宽高"""
+        parts = val.strip().split()
+        if len(parts) < 2:
+            return "❌ 用法: ai_image size <宽> <高>"
+        try:
+            w, h = int(parts[0]), int(parts[1])
+            if w < 64 or w > 1344 or h < 64 or h > 1344:
+                return "❌ 宽高范围: 64 ~ 1344"
+            self.width = w
+            self.height = h
+            return f"✅ 尺寸已设置为: {w} x {h}"
+        except ValueError:
+            return "❌ 请输入有效数字"
+
+    def _cmd_set_rewrite(self, val):
+        """设置改写模式"""
+        val = val.strip().lower()
+        if val in ["on", "开", "1", "true", "yes"]:
+            self.rewrite = True
+            return "✅ 改写模式已开启 — LLM 将智能重写 prompt"
+        elif val in ["off", "关", "0", "false", "no"]:
+            self.rewrite = False
+            return "✅ 改写模式已关闭 — 普通翻译/追加模式"
+        else:
+            self.rewrite = not self.rewrite
+            status = "开启" if self.rewrite else "关闭"
+            return f"✅ 改写模式已{status}"
+
+    def _cmd_set_override(self, val):
+        """设置覆写模式"""
+        val = val.strip().lower()
+        if val in ["on", "开", "1", "true", "yes"]:
+            self.override = True
+            return "✅ 覆写模式已开启 — 忽略工作流内置 prompt"
+        elif val in ["off", "关", "0", "false", "no"]:
+            self.override = False
+            return "✅ 覆写模式已关闭 — 保留工作流内置 prompt"
+        else:
+            self.override = not self.override
+            status = "开启" if self.override else "关闭"
+            return f"✅ 覆写模式已{status}"
+
+    def _cmd_status(self):
+        """查看当前状态"""
+        result = " AI 生图状态:\n\n"
+        result += f"  工作流: {self.current_workflow or '(默认)'}\n"
+        result += f"  直接 Tag: {self.direct_prompt or '(未设置)'}\n"
+        result += f"  自然语言: {self.nl_prompt or '(未设置)'}\n"
+        result += f"  改写模式: {'✅ 开启' if self.rewrite else '❌ 关闭'}\n"
+        result += f"  覆写模式: {'✅ 开启' if self.override else '❌ 关闭'}\n"
+        result += f"  宽度: {self.width or '(默认)'}\n"
+        result += f"  高度: {self.height or '(默认)'}\n"
+        result += f"  保存目录: {self.save_dir}\n"
+        result += f"  生成中: {'是' if self.is_generating else '否'}\n"
+        return result
+
+    # ── 核心生图逻辑 ──
+
+    def _fetch_workflows(self):
+        """获取工作流列表"""
+        if self.workflows_cache:
+            return self.workflows_cache
+
+        try:
+            resp = requests.get(f"{self.API_URL}/workflows", timeout=15)
+            if resp.status_code == 200:
+                data = resp.json()
+                self.workflows_cache = data.get("workflows", [])
+                return self.workflows_cache
+        except Exception as e:
+            pass
+        return []
+
+    def _run_generation(self):
+        """执行图片生成"""
+        if self.is_generating:
+            return "⚠️ 已有生成任务在进行中，请等待完成"
+
+        if not self.direct_prompt and not self.nl_prompt:
+            return "❌ 请提供至少一种 prompt（直接 Tag 或自然语言描述）"
+
+        self.is_generating = True
+        result_lines = []
+        image_paths = []
+
+        try:
+            # 获取默认工作流（如果未指定则使用列表中第一个）
+            workflow = self.current_workflow
+            if not workflow:
+                workflows = self._fetch_workflows()
+                if workflows:
+                    # 优先选 "无Lora" 工作流，否则用第一个
+                    for wf in workflows:
+                        if "无lora" in wf.get("path", "").lower():
+                            workflow = wf["path"]
+                            break
+                    if not workflow:
+                        workflow = workflows[0].get("path", "")
+
+            # 构建 payload
+            payload = {
+                "workflow_path": workflow or "",
+                "inline_workflow": None,
+                "direct_prompt": self.direct_prompt,
+                "nl_prompt": self.nl_prompt,
+                "rewrite": self.rewrite,
+                "override": self.override,
+                "width": self.width,
+                "height": self.height,
+            }
+
+            result_lines.append("  正在连接 ai.2x.nz ...")
+            result_lines.append(f"  直接 Tag: {self.direct_prompt[:80] or '(无)'}")
+            result_lines.append(f"  自然语言: {self.nl_prompt[:80] or '(无)'}")
+            result_lines.append(f"  工作流: {workflow or '(默认)'}")
+            result_lines.append("")
+
+            # 使用同步方式通过 REST + 轮询获取结果
+            # 由于 websocket-client 可能未安装，提供降级方案
+            if WEBSOCKET_AVAILABLE:
+                images = self._generate_via_ws(payload, result_lines)
+            else:
+                # 降级：尝试使用 requests + SSE 或提示安装
+                result_lines.append("⚠️ websocket-client 未安装，尝试安装...")
+                images = self._generate_via_ws_fallback(payload, result_lines)
+
+            if images:
+                image_paths = images
+                result_lines.append(f"\n✅ 生成完成！共 {len(images)} 张图片")
+                for i, path in enumerate(images, 1):
+                    result_lines.append(f"   {i}. {path}")
+            else:
+                result_lines.append("\n❌ 未生成任何图片")
+
+        except Exception as e:
+            result_lines.append(f"\n❌ 生成失败: {str(e)}")
+        finally:
+            self.is_generating = False
+
+        return "\n".join(result_lines)
+
+    def _generate_via_ws(self, payload, result_lines):
+        """通过 WebSocket 生成图片"""
+        images = []
+        final_prompt = ""
+        error_msg = None
+
+        ws_result = {"images": [], "error": None, "final_prompt": "", "done": False}
+
+        def on_message(ws, message):
+            try:
+                m = json.loads(message)
+                msg_type = m.get("type", "")
+
+                if msg_type == "log":
+                    result_lines.append(f"  {m.get('message', '')}")
+                elif msg_type == "llm_start":
+                    result_lines.append(" LLM 正在处理 prompt...")
+                elif msg_type == "llm_chunk":
+                    pass  # 流式输出，不逐块显示
+                elif msg_type == "llm_done":
+                    result_lines.append(" LLM 处理完成")
+                elif msg_type == "progress":
+                    node = m.get("node", "")
+                    value = m.get("value", 0)
+                    max_val = m.get("max", 0)
+                    if max_val and max_val > 1:
+                        pct = int(value * 100 / max_val)
+                        result_lines.append(f"⏳ 进度: {node} {value}/{max_val} ({pct}%)")
+                    else:
+                        result_lines.append(f"⏳ 执行: {node}")
+                elif msg_type == "prompt_id":
+                    fp = m.get("final_prompt", "")
+                    if fp:
+                        ws_result["final_prompt"] = fp
+                        result_lines.append(f"\n  最终 Prompt:\n{fp[:300]}{'...' if len(fp) > 300 else ''}\n")
+                elif msg_type == "image":
+                    url = m.get("url", "")
+                    filename = m.get("filename", "")
+                    if url:
+                        # 下载图片
+                        saved = self._download_image(url, filename)
+                        if saved:
+                            ws_result["images"].append(saved)
+                            result_lines.append(f"  收到图片: {filename}")
+                elif msg_type == "done":
+                    count = m.get("count", 0)
+                    result_lines.append(f"✅ 服务端完成，共 {count} 张")
+                    ws_result["done"] = True
+                    ws.close()
+                elif msg_type == "error":
+                    ws_result["error"] = m.get("message", "未知错误")
+                    result_lines.append(f"❌ 错误: {ws_result['error']}")
+                    ws.close()
+            except Exception as e:
+                result_lines.append(f"⚠️ 解析消息出错: {e}")
+
+        def on_error(ws, error):
+            ws_result["error"] = str(error)
+            result_lines.append(f"❌ WebSocket 错误: {error}")
+
+        def on_close(ws, close_status_code, close_msg):
+            ws_result["done"] = True
+
+        def on_open(ws):
+            ws.send(json.dumps(payload))
+
+        try:
+            ws = websocket.WebSocketApp(
+                self.WS_URL,
+                on_message=on_message,
+                on_error=on_error,
+                on_close=on_close,
+                on_open=on_open
+            )
+
+            # 在独立线程中运行 WebSocket
+            ws_thread = threading.Thread(target=ws.run_forever, kwargs={"ping_timeout": 10})
+            ws_thread.daemon = True
+            ws_thread.start()
+
+            # 等待完成（最多 5 分钟）
+            timeout = 300
+            start = time.time()
+            while not ws_result["done"] and (time.time() - start) < timeout:
+                time.sleep(1)
+
+            if ws_result["error"]:
+                raise Exception(ws_result["error"])
+
+            if not ws_result["done"]:
+                ws.close()
+                raise Exception("生成超时（超过 5 分钟）")
+
+            return ws_result["images"]
+
+        except Exception as e:
+            raise Exception(f"WebSocket 连接失败: {e}")
+
+    def _generate_via_ws_fallback(self, payload, result_lines):
+        """降级方案：尝试自动安装 websocket-client"""
+        try:
+            import subprocess
+            result_lines.append("  正在安装 websocket-client ...")
+            subprocess.run(
+                ["pip", "install", "websocket-client"],
+                capture_output=True, timeout=60
+            )
+            import importlib
+            global websocket
+            websocket = importlib.import_module("websocket")
+            global WEBSOCKET_AVAILABLE
+            WEBSOCKET_AVAILABLE = True
+            result_lines.append("✅ websocket-client 安装成功")
+            return self._generate_via_ws(payload, result_lines)
+        except Exception as e:
+            result_lines.append(f"❌ 无法安装 websocket-client: {e}")
+            result_lines.append("💡 请手动运行: pip install websocket-client")
+            return []
+
+    def _download_image(self, url, filename):
+        """下载图片到本地"""
+        try:
+            # 确保 URL 是完整的
+            if url.startswith("/"):
+                url = f"{self.BASE_URL}{url}"
+            elif not url.startswith("http"):
+                url = f"{self.BASE_URL}/api/output/file?path={quote(url)}"
+
+            # 创建保存目录
+            os.makedirs(self.save_dir, exist_ok=True)
+
+            # 生成文件名
+            if not filename:
+                filename = f"img_{datetime.now().strftime('%Y%m%d_%H%M%S')}.png"
+
+            # 清理文件名
+            filename = re.sub(r'[<>:"/\\|?*]', '_', filename)
+            save_path = os.path.join(self.save_dir, filename)
+
+            # 如果文件已存在，添加序号
+            if os.path.exists(save_path):
+                name, ext = os.path.splitext(filename)
+                for i in range(1, 100):
+                    new_path = os.path.join(self.save_dir, f"{name}_{i}{ext}")
+                    if not os.path.exists(new_path):
+                        save_path = new_path
+                        break
+
+            # 下载
+            resp = requests.get(url, timeout=60, stream=True)
+            if resp.status_code == 200:
+                with open(save_path, 'wb') as f:
+                    for chunk in resp.iter_content(chunk_size=8192):
+                        f.write(chunk)
+                return save_path
+            else:
+                return None
+
+        except Exception as e:
+            return None
+
+
+# 兼容 Liugin 类名
+Liugin = Plugin

tests/test_plugins.py

@@ -357,6 +357,147 @@ class TestBrowserAuto:
 #  引擎加载测试
 # ══════════════════════════════════════
 
+class TestGuiAuto:
+    """gui_auto 功能测试"""
+
+    @pytest.fixture
+    def gui(self):
+        mod = load_module("gui_auto", os.path.join(PLUGINS_DIR, "gui_auto.py"))
+        return mod.Plugin()
+
+    def test_tool_info(self, gui):
+        info = gui.get_tool_info()
+        assert info["name"] == "gui_auto"
+        assert "GUI" in info["description"]
+        assert len(info["keywords"]) >= 5
+
+    def test_mcp_definition(self, gui):
+        mcp = gui.get_mcp_definition()
+        assert mcp["name"] == "gui_auto"
+        assert "inputSchema" in mcp
+        assert "operation" in mcp["inputSchema"]["properties"]
+
+    def test_convert_mcp_args(self, gui):
+        args = {"operation": "click", "target": "确定"}
+        result = gui.convert_mcp_args(args)
+        assert "click" in result
+        assert "确定" in result
+
+    def test_convert_mcp_args_xy(self, gui):
+        args = {"operation": "click", "x": 100, "y": 200}
+        result = gui.convert_mcp_args(args)
+        assert "--xy" in result
+        assert "100" in result
+
+    def test_convert_mcp_args_snapshot(self, gui):
+        args = {"operation": "snapshot", "depth": 3}
+        result = gui.convert_mcp_args(args)
+        assert "snapshot" in result
+        assert "3" in result
+
+    def test_empty_args(self, gui):
+        result = gui.handle("")
+        assert "错误" in result
+
+    def test_invalid_operation(self, gui):
+        result = gui.handle("nonexistent_op")
+        assert "不支持" in result
+
+    def test_click_no_args(self, gui):
+        result = gui.handle("click")
+        assert "错误" in result
+
+    def test_type_no_args(self, gui):
+        result = gui.handle("type")
+        assert "错误" in result
+
+    def test_type_missing_text(self, gui):
+        result = gui.handle("type 按钮")
+        assert "错误" in result
+
+    def test_find_no_args(self, gui):
+        result = gui.handle("find")
+        assert "错误" in result
+
+    def test_findall_no_args(self, gui):
+        result = gui.handle("findall")
+        assert "错误" in result
+
+    def test_info_no_args(self, gui):
+        result = gui.handle("info")
+        assert "错误" in result
+
+    def test_value_no_args(self, gui):
+        result = gui.handle("value")
+        assert "错误" in result
+
+    def test_tree_no_args(self, gui):
+        result = gui.handle("tree")
+        assert "错误" in result
+
+    def test_wait_no_args(self, gui):
+        result = gui.handle("wait")
+        assert "错误" in result
+
+    def test_exists_no_args(self, gui):
+        result = gui.handle("exists")
+        assert "错误" in result
+
+    def test_highlight_no_args(self, gui):
+        result = gui.handle("highlight")
+        assert "错误" in result
+
+    def test_state_no_args(self, gui):
+        result = gui.handle("state")
+        assert "错误" in result
+
+    def test_keys_no_args(self, gui):
+        result = gui.handle("keys")
+        assert "错误" in result
+
+    def test_focus_no_args(self, gui):
+        result = gui.handle("focus")
+        assert "错误" in result
+
+    def test_click_xy_format_error(self, gui):
+        result = gui.handle("click --xy abc")
+        assert "错误" in result
+
+    def test_format_element_info(self, gui):
+        el = {
+            "name": "测试按钮", "type": "Button", "class": "Button",
+            "automation_id": "btn_ok",
+            "bounds": {"left": 100, "top": 200, "width": 80, "height": 30,
+                       "right": 180, "bottom": 230},
+            "enabled": True, "focused": False
+        }
+        result = gui._format_element_info(el)
+        assert "测试按钮" in result
+        assert "Button" in result
+        assert "btn_ok" in result
+
+    def test_format_element_info_detailed(self, gui):
+        el = {
+            "name": "输入框", "type": "Edit", "class": "Edit",
+            "automation_id": "", "enabled": True, "focused": True,
+            "value": "hello",
+            "bounds": {"left": 0, "top": 0, "width": 200, "height": 25,
+                       "right": 200, "bottom": 25}
+        }
+        result = gui._format_element_info(el, detailed=True)
+        assert "输入框" in result
+        assert "hello" in result
+
+    def test_all_operations_route(self, gui):
+        """所有操作都能正确路由"""
+        for op in ["snapshot", "screenshot", "windows", "focus", "click",
+                    "rightclick", "doubleclick", "type", "clear", "keys",
+                    "scroll", "find", "findall", "info", "value", "tree",
+                    "wait", "exists", "highlight", "state"]:
+            result = gui.handle(f"{op} test")
+            assert isinstance(result, str)
+
+
 class TestSubAgent:
     """sub_agent 功能测试"""
 

webui_server.py

@@ -0,0 +1,376 @@
+#!/usr/bin/env python3
+"""
+小狸 Pro-CLI Web UI 后端
+WebSocket 服务器 + 静态文件服务
+
+启动:
+  python webui_server.py [--port 8080] [--host 0.0.0.0]
+
+依赖:
+  pip install websockets
+"""
+
+import asyncio
+import json
+import os
+import sys
+import time
+import argparse
+import mimetypes
+from pathlib import Path
+from datetime import datetime
+from http.server import HTTPServer, SimpleHTTPRequestHandler
+from urllib.parse import urlparse, parse_qs
+import threading
+
+# 添加项目根目录
+PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, PROJECT_DIR)
+
+try:
+    import websockets
+    WEBSOCKETS_AVAILABLE = True
+except ImportError:
+    WEBSOCKETS_AVAILABLE = False
+    print("⚠️  websockets 未安装，WebSocket 功能不可用")
+    print("   安装: pip install websockets")
+
+
+# ═══════════════════════════════════════════
+#  AI 引擎桥接
+# ═══════════════════════════════════════════
+
+class AIEngineBridge:
+    """桥接到小狸的 AI 引擎和插件系统"""
+
+    def __init__(self):
+        self.cli = None
+        self._initialized = False
+
+    def initialize(self):
+        """延迟初始化小狸核心"""
+        if self._initialized:
+            return True
+        try:
+            from xcli_core import AICLI
+            self.cli = AICLI()
+            self._initialized = True
+            return True
+        except Exception as e:
+            print(f"⚠️  小狸核心初始化失败: {e}")
+            return False
+
+    def get_status(self):
+        """获取系统状态"""
+        if not self.cli:
+            return {
+                "engine": "未连接",
+                "engine_type": "none",
+                "plugins": 0,
+                "operations": 0,
+                "safety_mode": "普通",
+                "tools": [],
+                "engines": [],
+            }
+
+        engine_name = getattr(self.cli.current_engine, 'name', '未知') if self.cli.current_engine else '无'
+        tools = []
+        if hasattr(self.cli, 'liugin_manager') and self.cli.liugin_manager:
+            for t in self.cli.liugin_manager.tools:
+                tools.append({
+                    "name": t.get('name', ''),
+                    "description": t.get('description', ''),
+                    "ops": len(t.get('usage', '').split('\n')) if t.get('usage') else 0,
+                })
+
+        from xcli_core.safety import get_safety
+        safety = get_safety()
+
+        return {
+            "engine": engine_name,
+            "engine_type": "local" if engine_name == "ollama" else "cloud",
+            "plugins": len(tools),
+            "operations": sum(t.get('ops', 0) for t in tools),
+            "safety_mode": safety.get_mode_name(),
+            "tools": tools,
+            "engines": list(self.cli.engines.keys()) if self.cli.engines else [],
+        }
+
+    def chat(self, message: str, callback=None):
+        """发送消息到 AI 并获取回复"""
+        if not self.cli:
+            return "❌ 小狸核心未初始化，请检查配置"
+
+        try:
+            # 使用小狸的核心对话处理
+            import io
+            import contextlib
+
+            # 捕获输出
+            captured = io.StringIO()
+            with contextlib.redirect_stdout(captured):
+                # 构建系统提示词
+                liugin_prompts = self.cli.get_liugin_usage_prompts()
+                system_prompt = self.cli._build_system_prompt(liugin_prompts)
+
+                # 生成响应
+                if self.cli.current_engine:
+                    response = self.cli.current_engine.generate_response(
+                        message,
+                        system_prompt=system_prompt
+                    )
+                else:
+                    response = "❌ 没有可用的 AI 引擎"
+
+            output = captured.getvalue()
+            return response or "(无响应)"
+
+        except Exception as e:
+            return f"❌ 错误: {e}"
+
+    def execute_tool(self, tool_name: str, tool_args: str):
+        """直接执行工具"""
+        if not self.cli:
+            return "❌ 小狸核心未初始化"
+
+        try:
+            result = self.cli._execute_tool_by_name(tool_name, tool_args)
+            return result or "(无结果)"
+        except Exception as e:
+            return f"❌ 工具执行错误: {e}"
+
+    def list_tools(self):
+        """列出所有工具"""
+        if not self.cli or not hasattr(self.cli, 'liugin_manager'):
+            return []
+        return [
+            {
+                "name": t.get('name', ''),
+                "description": t.get('description', ''),
+                "keywords": t.get('keywords', []),
+            }
+            for t in self.cli.liugin_manager.tools
+        ]
+
+
+# ═══════════════════════════════════════════
+#  WebSocket 处理
+# ═══════════════════════════════════════════
+
+bridge = AIEngineBridge()
+connected_clients = set()
+
+
+async def ws_handler(websocket):
+    """处理 WebSocket 连接"""
+    connected_clients.add(websocket)
+    client_addr = websocket.remote_address
+    print(f"✅ 客户端已连接: {client_addr}")
+
+    try:
+        # 发送初始状态
+        status = bridge.get_status()
+        await websocket.send(json.dumps({
+            "type": "status",
+            "data": status,
+        }, ensure_ascii=False))
+
+        async for raw_message in websocket:
+            try:
+                msg = json.loads(raw_message)
+                msg_type = msg.get("type", "")
+                msg_data = msg.get("data", {})
+
+                if msg_type == "chat":
+                    # 用户聊天消息
+                    user_msg = msg_data.get("message", "").strip()
+                    if not user_msg:
+                        continue
+
+                    # 发送"正在思考"状态
+                    await websocket.send(json.dumps({
+                        "type": "thinking",
+                        "data": {"message": user_msg},
+                    }, ensure_ascii=False))
+
+                    # 在线程中执行 AI 调用（避免阻塞事件循环）
+                    loop = asyncio.get_event_loop()
+                    response = await loop.run_in_executor(
+                        None, bridge.chat, user_msg
+                    )
+
+                    # 发送回复
+                    await websocket.send(json.dumps({
+                        "type": "reply",
+                        "data": {"message": response},
+                    }, ensure_ascii=False))
+
+                elif msg_type == "tool":
+                    # 直接工具调用
+                    tool_name = msg_data.get("name", "")
+                    tool_args = msg_data.get("args", "")
+
+                    await websocket.send(json.dumps({
+                        "type": "tool_executing",
+                        "data": {"name": tool_name, "args": tool_args},
+                    }, ensure_ascii=False))
+
+                    loop = asyncio.get_event_loop()
+                    result = await loop.run_in_executor(
+                        None, bridge.execute_tool, tool_name, tool_args
+                    )
+
+                    await websocket.send(json.dumps({
+                        "type": "tool_result",
+                        "data": {"name": tool_name, "result": result},
+                    }, ensure_ascii=False))
+
+                elif msg_type == "status":
+                    # 请求状态更新
+                    status = bridge.get_status()
+                    await websocket.send(json.dumps({
+                        "type": "status",
+                        "data": status,
+                    }, ensure_ascii=False))
+
+                elif msg_type == "tools":
+                    # 请求工具列表
+                    tools = bridge.list_tools()
+                    await websocket.send(json.dumps({
+                        "type": "tools",
+                        "data": {"tools": tools},
+                    }, ensure_ascii=False))
+
+                elif msg_type == "switch_engine":
+                    # 切换引擎
+                    engine_name = msg_data.get("engine", "")
+                    if bridge.cli and engine_name in bridge.cli.engines:
+                        bridge.cli.switch_engine(engine_name)
+                        await websocket.send(json.dumps({
+                            "type": "engine_switched",
+                            "data": {"engine": engine_name},
+                        }, ensure_ascii=False))
+
+                elif msg_type == "switch_safety":
+                    # 切换安全模式
+                    mode = msg_data.get("mode", "")
+                    if bridge.cli:
+                        from xcli_core.safety import get_safety, MODE_NORMAL, MODE_MANUAL, MODE_UNRESTRICTED
+                        safety = get_safety()
+                        if mode == "normal":
+                            safety.set_mode(MODE_NORMAL)
+                        elif mode == "manual":
+                            safety.set_mode(MODE_MANUAL)
+                        elif mode == "unrestricted":
+                            safety.set_mode(MODE_UNRESTRICTED)
+                        await websocket.send(json.dumps({
+                            "type": "safety_switched",
+                            "data": {"mode": safety.get_mode_name()},
+                        }, ensure_ascii=False))
+
+            except json.JSONDecodeError:
+                await websocket.send(json.dumps({
+                    "type": "error",
+                    "data": {"message": "无效的 JSON 格式"},
+                }, ensure_ascii=False))
+            except Exception as e:
+                await websocket.send(json.dumps({
+                    "type": "error",
+                    "data": {"message": str(e)},
+                }, ensure_ascii=False))
+
+    except websockets.exceptions.ConnectionClosed:
+        pass
+    finally:
+        connected_clients.discard(websocket)
+        print(f"❌ 客户端已断开: {client_addr}")
+
+
+# ═══════════════════════════════════════════
+#  静态文件服务
+# ═══════════════════════════════════════════
+
+class WebUIHandler(SimpleHTTPRequestHandler):
+    """自定义 HTTP 处理器"""
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, directory=PROJECT_DIR, **kwargs)
+
+    def do_GET(self):
+        # 根路径返回 webui.html
+        if self.path == '/' or self.path == '/index.html':
+            self.path = '/webui.html'
+        return super().do_GET()
+
+    def log_message(self, format, *args):
+        # 静默日志
+        pass
+
+
+# ═══════════════════════════════════════════
+#  启动
+# ═══════════════════════════════════════════
+
+def main():
+    parser = argparse.ArgumentParser(description="小狸 Pro-CLI Web UI 服务器")
+    parser.add_argument("--host", default="0.0.0.0", help="绑定地址")
+    parser.add_argument("--port", type=int, default=8080, help="HTTP 端口")
+    parser.add_argument("--ws-port", type=int, default=8079, help="WebSocket 端口")
+    parser.add_argument("--no-ai", action="store_true", help="不初始化 AI 引擎（纯 UI 模式）")
+    args = parser.parse_args()
+
+    print()
+    print("╔══════════════════════════════════════════╗")
+    print("║       小狸 Pro-CLI · Web UI 服务器       ║")
+    print("╚══════════════════════════════════════════╝")
+    print()
+
+    # 初始化 AI 引擎
+    if not args.no_ai:
+        print("⏳ 正在初始化小狸核心...")
+        if bridge.initialize():
+            status = bridge.get_status()
+            print(f"✅ AI 引擎: {status['engine']}")
+            print(f"✅ 插件: {status['plugins']} 个")
+            print(f"✅ 安全模式: {status['safety_mode']}")
+        else:
+            print("⚠️  AI 引擎初始化失败，将以纯 UI 模式运行")
+    else:
+        print("ℹ️  纯 UI 模式（不连接 AI 引擎）")
+
+    print()
+    print(f"🌐 HTTP:      http://localhost:{args.port}")
+    print(f"🔌 WebSocket: ws://localhost:{args.ws_port}")
+    print(f"📂 项目目录:  {PROJECT_DIR}")
+    print()
+    print("按 Ctrl+C 停止服务器")
+    print()
+
+    # 启动 HTTP 服务器（在单独线程中）
+    http_server = HTTPServer((args.host, args.port), WebUIHandler)
+    http_thread = threading.Thread(target=http_server.serve_forever, daemon=True)
+    http_thread.start()
+
+    # 启动 WebSocket 服务器
+    if WEBSOCKETS_AVAILABLE:
+        async def run_ws():
+            async with websockets.serve(ws_handler, args.host, args.ws_port):
+                await asyncio.Future()  # 永远运行
+
+        try:
+            asyncio.run(run_ws())
+        except KeyboardInterrupt:
+            print("\n🛑 服务器已停止")
+    else:
+        print("⚠️  WebSocket 不可用，仅启动 HTTP 服务")
+        try:
+            while True:
+                time.sleep(1)
+        except KeyboardInterrupt:
+            print("\n🛑 服务器已停止")
+
+    http_server.shutdown()
+
+
+if __name__ == "__main__":
+    main()