feat: enhance Milkdown editor and file system functionality
- Normalize line endings in Markdown export for DOCX files. - Improve selection serialization to Markdown with better handling of empty documents. - Add a new `updateFile` function to the file system for updating file properties. - Introduce video transcoding capabilities using FFmpeg, supporting various video formats. - Update AGENTS.md for clearer plugin structure and responsibilities. - Add scoped styles for TreeNodeItem component to improve UI consistency. - Implement cross-origin isolation headers in Vite configuration for enhanced security. - Remove obsolete test_cross.py file.
This commit is contained in:
@@ -1,9 +0,0 @@
|
||||
{
|
||||
"shortcuts": [
|
||||
{
|
||||
"label": "Run",
|
||||
"command": "npm run dev",
|
||||
"icon": "play"
|
||||
}
|
||||
]
|
||||
}
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -23,6 +23,8 @@ env/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
htmlcov/
|
||||
api_performance_report.md
|
||||
|
||||
# Env files
|
||||
.env
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
# 导出按钮缺失修复计划
|
||||
|
||||
## 问题分析
|
||||
当前 `action-buttons` 区域只有以下按钮可见：
|
||||
- 上传文件
|
||||
- 导入 Markdown
|
||||
- 导出 Markdown
|
||||
- 上传图片
|
||||
- AI 切换按钮
|
||||
|
||||
**缺失功能**：DOCX 和 PDF 导出按钮
|
||||
|
||||
## 调查结果
|
||||
1. ✅ 翻译文件中已存在 `exportDocx` 和 `exportPdf` 键名（src/utils/i18n.js）
|
||||
2. ❌ 模板中**完全缺失**这两个按钮的 HTML 代码
|
||||
3. ❓ 导出功能后端已实现，前端只需要添加调用接口的按钮
|
||||
4. ✅ 相关 CSS 样式已存在，按钮外观无需额外调整
|
||||
|
||||
## 实施计划
|
||||
|
||||
### 1. 添加 UI 按钮
|
||||
在 `src/components/MilkdownEditor.vue:79` 之后添加两个新按钮：
|
||||
- DOCX 导出按钮
|
||||
- PDF 导出按钮
|
||||
|
||||
按钮位置：
|
||||
```
|
||||
导出 Markdown → 导出 DOCX → 导出 PDF → 上传图片
|
||||
```
|
||||
|
||||
### 2. 实现前端导出功能
|
||||
使用已安装的依赖库：
|
||||
- `docx` 库：用于 DOCX 导出
|
||||
- `html2pdf.js` 库：用于 PDF 导出
|
||||
|
||||
需要添加的函数：
|
||||
```javascript
|
||||
const exportDocx = async () => {
|
||||
// 使用 docx 库实现导出
|
||||
}
|
||||
|
||||
const exportPdf = async () => {
|
||||
// 使用 html2pdf.js 实现导出
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 按钮图标
|
||||
- DOCX：使用文档图标
|
||||
- PDF：使用 PDF 专用图标
|
||||
|
||||
### 4. 状态管理
|
||||
添加加载状态和错误处理，与现有按钮保持一致风格
|
||||
@@ -1,11 +0,0 @@
|
||||
# rules.md
|
||||
|
||||
在构建这个LLM应用网页时，你需要基于VUE3开发。我需要前端只运行渲染和数据回传，后端负责llm api调用，类似copilet的auto inline suggustions实现和数据解析。
|
||||
|
||||
## 指导原则
|
||||
|
||||
- 不要擅自用npm或者yarn运行网页，你既看不到网页的内容，也无法阻止命令暂停。但是，你可以用npm run build检查代码。
|
||||
- 应该保证代码效率，不多定义变量，不写冗余注释，把降低延迟放在第一位。
|
||||
- 每次完成任务前都要反复阅读检查代码，确保代码准确无误。
|
||||
- 尽量不要搜索关键字，而是了解代码结构后查询整个问题代码明确问题所在。
|
||||
- @/milkdown-docs/ 代表milkdown的最新官方文档，不要修改，涉及到前端编辑器的指令时要核对官方文档。
|
||||
155
AGENTS.md
155
AGENTS.md
@@ -1,90 +1,101 @@
|
||||
# LLM in Text 项目知识库
|
||||
# LLM in Text 仓库指引
|
||||
|
||||
**生成时间:** 2025-04-10
|
||||
**Commit:** 2fdc996
|
||||
**Branch:** main
|
||||
本文件适用于整个仓库。进入更深层目录后，子目录中的 AGENTS.md 优先于本文件。
|
||||
|
||||
## 概述
|
||||
## 项目定位
|
||||
|
||||
智能 Markdown 编辑器，集成 LLM 实时补全建议。前端 Vue3 + Vite + Milkdown，后端 FastAPI + Python + Ollama。核心功能：AI 补全、OCR 图片识别、文档转换、TTS/ASR 语音功能。
|
||||
- 这是一个智能 Markdown 编辑器，前端负责编辑器 UI、上传导出、补全交互和设置状态，后端负责 LLM、OCR、文件转换和 TTS 接口。
|
||||
- 前端技术栈：Vue 3 + Vite + Milkdown/Crepe + Pinia + Vue Router。
|
||||
- 后端技术栈：FastAPI + Python + Ollama。
|
||||
- 当前代码中可以确认的主功能是：AI 补全、OCR、文档转 Markdown、TTS、Markdown/DOCX/PDF 导入导出。
|
||||
- 历史文档中有一部分 TTS/ASR、Apple Silicon、Whisper、离线模式说明已经落后于当前代码；出现冲突时以实际代码和测试为准。
|
||||
|
||||
## 结构
|
||||
## 先看哪里
|
||||
|
||||
```
|
||||
llm-in-text/
|
||||
├── backend/ # FastAPI 后端 (Python)
|
||||
│ ├── main.py # API 入口，路由定义
|
||||
│ ├── llm.py # Ollama 调用封装
|
||||
│ ├── prompt.py # Prompt 构建逻辑
|
||||
│ ├── prompts/ # JSON 格式的提示模板
|
||||
│ └── tests/ # pytest 测试套件
|
||||
├── src/ # 前端源码 (Vue3 + Vite)
|
||||
│ ├── main.js # Vue 入口
|
||||
│ ├── App.vue # 根组件
|
||||
│ ├── components/ # Vue 组件
|
||||
│ ├── plugins/ # Milkdown/Copilot 插件
|
||||
│ ├── stores/ # Pinia 状态管理
|
||||
│ ├── views/ # 页面视图
|
||||
│ └── utils/ # 工具函数
|
||||
├── public/ # 静态资源
|
||||
├── milkdown-docs/ # Milkdown 官方文档（只读）
|
||||
└── index.html # HTML 入口
|
||||
```
|
||||
- 项目概览和运行说明：README.md
|
||||
- 前端入口：src/main.js
|
||||
- 路由：src/router/index.js
|
||||
- 编辑器主组件：src/components/MilkdownEditor.vue
|
||||
- AI 补全核心：src/plugins/copilotPlugin.ts
|
||||
- 前端请求层：src/utils/api.js
|
||||
- 前端配置：src/utils/config.js
|
||||
- 设置状态：src/stores/settings.js
|
||||
- 后端入口和主路由：backend/main.py
|
||||
- LLM 和 OCR 调用：backend/llm.py
|
||||
- Prompt 组装：backend/prompt.py
|
||||
- TTS 路由：backend/tts_asr.py
|
||||
- 测试配置和入口：pytest.ini、backend/tests/run_tests.py
|
||||
|
||||
## 查找指南
|
||||
## 稳定事实
|
||||
|
||||
| 任务 | 位置 | 说明 |
|
||||
|------|------|------|
|
||||
| 后端 API 入口 | `backend/main.py` | FastAPI 路由、CORS、启动逻辑 |
|
||||
| LLM 调用 | `backend/llm.py` | Ollama 异步调用、超时控制 |
|
||||
| Prompt 构建 | `backend/prompt.py` | 系统提示、上下文准备 |
|
||||
| AI 补全核心 | `src/plugins/copilotPlugin.ts` | ProseMirror Mark、ghost text |
|
||||
| 编辑器组件 | `src/components/MilkdownEditor.vue` | Milkdown 编辑器封装 |
|
||||
| 状态管理 | `src/stores/settings.js` | 用户设置、主题、偏好 |
|
||||
| API 调用 | `src/utils/api.js` | fetchSuggestion、TTS 接口 |
|
||||
| 测试运行 | `pytest.ini` + `backend/tests/` | 测试配置与用例 |
|
||||
- 补全接口当前不是 SSE；前端用普通 POST 请求拿 JSON 响应。
|
||||
- 前端会生成 X-Request-Id，并在请求被中止时额外调用 /v1/completions/cancel。
|
||||
- 文档超过 32 KB 时，AI 补全会在前端和插件层被禁用。
|
||||
- OCR 文本和文档块内容会被注入补全上下文，但这些内容属于隐藏上下文，不应被直接当作用户可见文本重复输出。
|
||||
- /v1/convert 当前支持 txt、docx、pptx、pdf，非 txt 文件通过 MarkItDown 转成 Markdown，之后会清理图片标记。
|
||||
- 前端存在 /v1/export/pdf 调用点，但当前后端主路由中看不到同名端点；排查 PDF 导出问题前先确认服务端是否真正提供该接口。
|
||||
- 当前 tts_asr.py 主要提供 TTS 相关能力。不要直接沿用 README 或历史修复文档里关于 ASR、Whisper、MPS/offline 的描述。
|
||||
|
||||
## 约定（项目特定）
|
||||
## 常用命令
|
||||
|
||||
- **前端入口**：`src/main.js`（非 TypeScript），使用 Vue3 + Pinia + Vue Router
|
||||
- **后端入口**：`backend/main.py`，端口 8001，uvicorn 启动
|
||||
- **代理配置**：开发时 `/v1` 代理到远程 API，生产需调整
|
||||
- **文件命名**：全小写+短横线（`my-module.py`、`my-component.vue`）
|
||||
- **语言**：UI 默认中文，响应必须使用中文
|
||||
- 前端安装：npm install
|
||||
- 前端开发：npm run dev
|
||||
- 前端构建：npm run build
|
||||
- 后端安装：pip install -r backend/requirements.txt
|
||||
- 后端启动：python backend/main.py
|
||||
- 可选启动方式：uvicorn backend.main:app --reload --port 8001
|
||||
- 全量测试：pytest
|
||||
- 常用窄测试：
|
||||
- pytest backend/tests/test_main_endpoints.py -v
|
||||
- pytest backend/tests/test_main_cancel.py -v
|
||||
- pytest backend/tests/test_prompt.py -v
|
||||
- pytest backend/tests/test_llm.py -v
|
||||
|
||||
## 反模式（本项目禁止）
|
||||
## 代码约定
|
||||
|
||||
- ❌ 硬编码 API_KEY（必须从环境变量读取）
|
||||
- ❌ 在前端暴露密钥（应通过后端代理）
|
||||
- ❌ `npm run dev` 运行网页（无法看到内容）
|
||||
- ❌ 修改 `milkdown-docs/` 目录
|
||||
- ❌ 类型错误使用 `as any` / `@ts-ignore`
|
||||
- ❌ 空的 catch 块
|
||||
- 不要把整个仓库当成“全小写+短横线命名”项目。当前实际情况是：
|
||||
- Vue 组件和视图多为 PascalCase
|
||||
- 前端工具模块多为小写 .js
|
||||
- 插件层使用 TypeScript
|
||||
- Python 使用 snake_case
|
||||
- 以就地风格为准，不要顺手做全仓格式统一。
|
||||
- UI 文案和代理回复默认使用中文。
|
||||
- 不要修改 milkdown-docs/，它是只读参考资料。
|
||||
- 不要新增硬编码密钥、空 catch/except、as any、@ts-ignore 之类的扩散式技术债。
|
||||
- 代理在这个仓库里应优先做局部、可验证的修改，不要做无关重构。
|
||||
|
||||
## 命令
|
||||
## 调试路径
|
||||
|
||||
```bash
|
||||
# 前端开发
|
||||
npm install
|
||||
npm run dev # 端口 5173
|
||||
npm run build # 构建到 dist/
|
||||
- 补全问题：
|
||||
src/components/MilkdownEditor.vue
|
||||
-> src/plugins/copilotPlugin.ts
|
||||
-> src/utils/api.js
|
||||
-> backend/main.py
|
||||
-> backend/prompt.py / backend/llm.py
|
||||
|
||||
# 后端运行
|
||||
pip install -r backend/requirements.txt
|
||||
python backend/main.py # 端口 8001
|
||||
# 或
|
||||
uvicorn backend.main:app --reload --port 8001
|
||||
- OCR 问题：
|
||||
src/components/MilkdownEditor.vue
|
||||
-> backend/main.py
|
||||
-> backend/llm.py
|
||||
|
||||
# 测试
|
||||
pytest # 运行所有测试，覆盖率要求 90%
|
||||
python backend/tests/run_tests.py unit # 单元测试
|
||||
python backend/tests/run_tests.py integration # 集成测试
|
||||
```
|
||||
- 文档转换问题：
|
||||
src/utils/convert.js
|
||||
-> backend/main.py
|
||||
|
||||
## 注意事项
|
||||
- TTS 问题：
|
||||
src/components/TTSMenu.vue / src/components/TTSPlayer.vue / src/components/MilkdownEditor.vue
|
||||
-> src/utils/api.js
|
||||
-> backend/tts_asr.py
|
||||
|
||||
- **架构分离**：前端仅渲染和数据回传，后端负责 LLM API 调用和数据解析
|
||||
- **延迟优先**：代码效率优先，降低延迟放在第一位
|
||||
- **大小限制**：文档超过 32KB 自动禁用 AI 补全
|
||||
- **milkdown-docs/**：官方文档参考，不可修改，编辑器相关问题需核对此目录
|
||||
## 测试和产物
|
||||
|
||||
- pytest.ini 对 backend.main、backend.llm、backend.prompt、backend.geoip、backend.prompts、backend.tts_asr 设了覆盖率门槛，低于 90% 会失败。
|
||||
- 默认测试目录是 backend/tests。
|
||||
- 常见生成产物包括 dist、htmlcov、.pytest_cache、api_performance_report.md；它们不是源代码。
|
||||
|
||||
## 文档注意事项
|
||||
|
||||
- README.md 对产品功能有参考价值，但其中补全、TTS/ASR 和部分接口说明已经比代码旧。
|
||||
- backend/TTS_ASR_MACOS_FIX.md 和 backend/tests/TESTING_GUIDE.md 更适合作为历史背景，不应在与代码冲突时被当成事实来源。
|
||||
- 修改行为时，优先参考实现代码和对应测试，再决定是否同步普通文档。
|
||||
|
||||
|
||||
@@ -1,40 +1,121 @@
|
||||
# Backend 模块指南
|
||||
# Backend 后端指引
|
||||
|
||||
## OVERVIEW
|
||||
FastAPI 后端，处理 AI 补全、OCR、文档转换、TTS/ASR。
|
||||
本文件适用于 backend/ 下的后端实现。进入 backend/tests/ 后，以子目录 AGENTS.md 为准。
|
||||
|
||||
## STRUCTURE
|
||||
- main.py - API 入口、路由、CORS、启动逻辑
|
||||
- llm.py - Ollama 异步调用、超时控制、日志
|
||||
- prompt.py - Prompt 构建、上下文准备、语言处理
|
||||
- geoip.py - IP 地理位置查询
|
||||
- tts_asr.py - TTS/ASR 处理、Apple Silicon 优化
|
||||
- prompts/ - JSON 格式提示模板（PromptManager 单例）
|
||||
- tests/ - pytest 测试套件（见子目录 AGENTS.md）
|
||||
## 后端职责
|
||||
|
||||
## WHERE TO LOOK
|
||||
- 对外提供补全、取消补全、OCR、文档转换和 TTS 相关接口。
|
||||
- 组织 Prompt，上下文清洗，调用 Ollama 模型。
|
||||
- 负责 API Key 校验、日志记录和部分启动预热逻辑。
|
||||
|
||||
| 任务 | 文件 | 说明 |
|
||||
|------|------|------|
|
||||
| API 路由定义 | main.py | /v1/completions、/v1/ocr、/v1/convert 等 |
|
||||
| LLM 调用封装 | llm.py | call_ollama、call_vlm_ocr、超时控制 |
|
||||
| Prompt 构建 | prompt.py | build_completion_prompts、语言处理 |
|
||||
| 提示模板 | prompts/__init__.py | PromptManager、JSON 模板加载 |
|
||||
| TTS/ASR | tts_asr.py | 模型预热、设备检测、音频处理 |
|
||||
| 测试 | tests/ | pytest 测试套件 |
|
||||
## 先看哪里
|
||||
|
||||
## CONVENTIONS
|
||||
- Python 4 空格缩进
|
||||
- 函数/变量：snake_case
|
||||
- 类：PascalCase
|
||||
- 文件名：全小写+短横线
|
||||
- API 入口和路由：main.py
|
||||
- Ollama 调用封装：llm.py
|
||||
- Prompt 清洗和拼装：prompt.py
|
||||
- 数据模型：models.py
|
||||
- 地理位置：geoip.py
|
||||
- TTS 路由：tts_asr.py
|
||||
- Prompt 模板：prompts/
|
||||
- 后端测试：tests/
|
||||
|
||||
## ANTI-PATTERNS
|
||||
- 硬编码 API_KEY（必须从环境变量读取）
|
||||
- 空 catch 块
|
||||
- 类型错误使用 as any / @ts-ignore
|
||||
## 当前接口面
|
||||
|
||||
## 注意事项
|
||||
- 端口：8001
|
||||
- 启动：`python backend/main.py` 或 `uvicorn backend.main:app --reload`
|
||||
- 依赖：`pip install -r backend/requirements.txt`
|
||||
- POST /v1/completions
|
||||
- POST /v1/completions/cancel
|
||||
- POST /v1/ocr
|
||||
- POST /v1/convert
|
||||
- /v1/tts-asr/* 由 tts_asr.py 延迟注册
|
||||
|
||||
## 请求流转
|
||||
|
||||
### /v1/completions
|
||||
|
||||
- 读取或生成 request_id。
|
||||
- privacy_mode 为 false 时，尝试根据客户端 IP 生成 location 文本。
|
||||
- 调用 prepare_prompt_context 清洗 prefix 和 suffix。
|
||||
- 调用 build_completion_prompts 生成 system_prompt 和 user_prompt。
|
||||
- 创建异步任务调用 call_ollama。
|
||||
- 用 request_id 把任务登记到 ACTIVE_COMPLETIONS。
|
||||
- 成功时返回 JSON：content 和 request_id。
|
||||
- finally 中清理当前 request_id 对应任务。
|
||||
|
||||
### /v1/completions/cancel
|
||||
|
||||
- 通过 request_id 在 ACTIVE_COMPLETIONS 中查找任务。
|
||||
- 未找到返回 not_found。
|
||||
- 已完成返回 already_done。
|
||||
- 仍在执行则调用 task.cancel() 并返回 ok。
|
||||
|
||||
### /v1/ocr
|
||||
|
||||
- 把 base64 图片解码成字节。
|
||||
- 调用 call_vlm_ocr。
|
||||
- 返回识别文本和原始文件名。
|
||||
|
||||
### /v1/convert
|
||||
|
||||
- 接收 base64 文件内容和文件名。
|
||||
- 当前允许的扩展名只有 txt、docx、pptx、pdf。
|
||||
- txt 直接解码后清洗。
|
||||
- 其他格式写入临时文件，用 MarkItDown 转换，再做 Markdown 清洗。
|
||||
- 清洗逻辑会移除图片 Markdown 和 img HTML 标签，并压缩多余空行。
|
||||
|
||||
### /v1/tts-asr/*
|
||||
|
||||
- 通过 _register_tts_asr_routes 延迟导入并挂到主应用。
|
||||
- 当前代码里的 tts_asr.py 主要是 TTS 能力，不要自行假设存在完整 ASR 实现。
|
||||
|
||||
## 开发命令
|
||||
|
||||
- 安装依赖：pip install -r backend/requirements.txt
|
||||
- 启动：python backend/main.py
|
||||
- 开发启动：uvicorn backend.main:app --reload --port 8001
|
||||
- 路由相关测试：
|
||||
- pytest backend/tests/test_main_endpoints.py -v
|
||||
- pytest backend/tests/test_main_cancel.py -v
|
||||
- Prompt 测试：
|
||||
- pytest backend/tests/test_prompt.py -v
|
||||
- pytest backend/tests/test_prompt_extended.py -v
|
||||
- LLM 测试：
|
||||
- pytest backend/tests/test_llm.py -v
|
||||
- pytest backend/tests/test_llm_extended.py -v
|
||||
|
||||
## 编码约定
|
||||
|
||||
- Python 使用 4 空格缩进。
|
||||
- 函数、变量使用 snake_case，类使用 PascalCase。
|
||||
- 新逻辑优先保留显式类型和明确的输入输出。
|
||||
- 异步边界要清晰；阻塞操作优先放进 asyncio.to_thread，而不是直接阻塞事件循环。
|
||||
- 异常要么转成 HTTPException，要么转成结构化 JSONResponse；不要静默吞掉后端错误。
|
||||
- 日志尽量带 request_id 或短 tag，便于把前后端一次请求串起来。
|
||||
|
||||
## 容易误判的点
|
||||
|
||||
- 补全接口当前不是流式响应，不要按 SSE 方式改造周边代码。
|
||||
- ACTIVE_COMPLETIONS 在补全和取消路径里都被读写，任务生命周期要谨慎处理。
|
||||
- main.py 里虽然有 _convert_docx_to_pdf 辅助函数，但当前 /v1/convert 路径实际走的是 MarkItDown，不要误以为 DOCX 转 PDF 桥接脚本已接入主流程。
|
||||
- API_KEY 存在占位默认值，这更像本地开发兜底，不是推荐的安全模式。
|
||||
- 历史 TTS/ASR 文档和部分测试覆盖的是旧实现；代码与文档冲突时，先确认产品方向，再决定修代码还是修文档。
|
||||
|
||||
## 改动时的定位建议
|
||||
|
||||
- 如果问题是补全结果不对，先查 prompt.py，再查 llm.py，不要只盯着 main.py。
|
||||
- 如果问题是取消不生效，先查 main.py 里的 request_id 生命周期，再对照前端的 X-Request-Id 和 cancel 调用。
|
||||
- 如果问题是 OCR 识别为空，先看 main.py 的 base64 解码，再看 llm.py 的 call_vlm_ocr。
|
||||
- 如果问题是转换结果脏，重点看 main.py 里的 _sanitize_converted_markdown。
|
||||
- 如果问题是 TTS 行为和文档不一致，以 tts_asr.py 为准，不要以 README 为准。
|
||||
|
||||
## 测试映射
|
||||
|
||||
- 路由主行为：tests/test_main_endpoints.py
|
||||
- 取消逻辑：tests/test_main_cancel.py
|
||||
- Prompt 逻辑：tests/test_prompt.py、tests/test_prompt_extended.py
|
||||
- LLM 包装层：tests/test_llm.py、tests/test_llm_extended.py
|
||||
- GeoIP：tests/test_geoip.py
|
||||
- TTS 相关：tests/test_tts_asr_*.py
|
||||
|
||||
## 文档使用原则
|
||||
|
||||
- README.md、TTS_ASR_MACOS_FIX.md、tests/TESTING_GUIDE.md 可以作为背景材料。
|
||||
- 一旦这些文档和 main.py、llm.py、prompt.py、tts_asr.py 冲突，以代码为准。
|
||||
|
||||
@@ -1,196 +0,0 @@
|
||||
# API Benchmarking Report (2026-04-06 13:45:31)
|
||||
|
||||
**Base URL:** `https://api.imageteach.tech:8002`
|
||||
|
||||
## Executive Summary
|
||||
| Task | Success Rate | Avg TTFB | Avg Latency | P95 Latency | TPS | RPS |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| Completion-Short | 90.0% | 9123.6ms | 9123.8ms | 20222.9ms | 7.9 | 0.08 |
|
||||
| Completion-Normal | 10.0% | 10559.3ms | 10559.6ms | 10559.6ms | 66.4 | 0.66 |
|
||||
| Completion-Long | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 8.97 |
|
||||
| OCR-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 6.75 |
|
||||
| TTS-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 10.17 |
|
||||
| ASR-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 13.02 |
|
||||
| Convert-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 5.98 |
|
||||
|
||||
## Stability & Context Analysis
|
||||
Detailed analysis of how context length affects TTFB and overall performance.
|
||||
|
||||
### Completion-Short Details
|
||||
- **Total Samples:** 10
|
||||
- **Duration:** 123.93s
|
||||
- **Top Errors:**
|
||||
- `[0]`
|
||||
|
||||
### Completion-Normal Details
|
||||
- **Total Samples:** 10
|
||||
- **Duration:** 15.21s
|
||||
- **Top Errors:**
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
|
||||
### Completion-Long Details
|
||||
- **Total Samples:** 10
|
||||
- **Duration:** 1.11s
|
||||
- **Top Errors:**
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
|
||||
### OCR-Concurrent Details
|
||||
- **Total Samples:** 10
|
||||
- **Duration:** 1.48s
|
||||
- **Top Errors:**
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
|
||||
### TTS-Concurrent Details
|
||||
- **Total Samples:** 10
|
||||
- **Duration:** 0.98s
|
||||
- **Top Errors:**
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
|
||||
<body>
|
||||
|
||||
<center><h1>502 Bad Gateway</h1></center>
|
||||
|
||||
<hr><center>openresty</center>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
|
||||
|
||||
- `[502]` <html>
|
||||
|
||||
<head><title>502 Bad Gateway</title></head>
|
||||
@@ -1,4 +1,4 @@
|
||||
import asyncio
|
||||
import asyncio
|
||||
import base64
|
||||
import logging
|
||||
import os
|
||||
@@ -9,9 +9,9 @@ import tempfile
|
||||
import uuid
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import FastAPI, HTTPException, Request, Security, File, UploadFile
|
||||
from fastapi import FastAPI, HTTPException, Request, Security
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse, Response
|
||||
from fastapi.responses import JSONResponse
|
||||
from fastapi.security import APIKeyHeader
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -133,8 +133,7 @@ def _sanitize_converted_markdown(text: str) -> str:
|
||||
value = (text or "").replace("\r\n", "\n").replace("\r", "\n")
|
||||
value = IMAGE_MARKDOWN_RE.sub("", value)
|
||||
value = IMAGE_HTML_RE.sub("", value)
|
||||
value = re.sub(r"\n{3,}", "\n\n", value)
|
||||
return value.strip()
|
||||
return value
|
||||
|
||||
|
||||
def get_client_ip(request: Request) -> str:
|
||||
|
||||
@@ -302,23 +302,24 @@ Requirements:
|
||||
- Non-empty and meaningful
|
||||
- Concise unless structure needs more
|
||||
- Follows markdown rules in system prompt
|
||||
- Use real line breaks instead of spelled-out escape sequences unless PREFIX or SUFFIX clearly requires that text
|
||||
|
||||
=== BOUNDARY DECISION GUIDE ===
|
||||
|
||||
Step 1: Check PREFIX_ENDS_WITH_NEWLINE
|
||||
If false, ask: "Does output need to start on a new line?"
|
||||
- YES if PREFIX ends with: ":", "steps:", "items:", heading text, or complete sentence before heading
|
||||
- If YES: start output with \\n
|
||||
- If YES: make the first character of OUTPUT a real newline
|
||||
|
||||
Step 2: Check SUFFIX_STARTS_WITH_NEWLINE
|
||||
If false, ask: "Does output need to end with a newline?"
|
||||
- YES if SUFFIX starts with: heading (##), new paragraph, or list marker
|
||||
- If YES: end output with \\n
|
||||
- If YES: make the last character of OUTPUT a real newline
|
||||
|
||||
Step 3: Choose newline type
|
||||
- Use \\n\\n for: new paragraphs, before headings, starting lists
|
||||
- Use \\n for: continuing within blocks, list items, table cells
|
||||
- Exception: inside code fences, use \\n freely
|
||||
- Use a blank line for: new paragraphs, before headings, starting lists
|
||||
- Use a single line break for: continuing within blocks, list items, table cells
|
||||
- Exception: inside code fences, use real newline characters freely
|
||||
|
||||
=== CONTEXT NOTES ===
|
||||
- OCR metadata (e.g., <OCR:description>) is hidden context, never copy to output
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
"template": "You are an inline completion engine for a {language_id} editor with ghost-text suggestions.\n\nReturn only the insertion text that should be placed between PREFIX and SUFFIX.\n\nCORE PRINCIPLE: Output insertion text only. No explanations, no meta labels, no wrapper quotes.\n\nPRIORITY 1: CONTEXT AWARENESS (Read these flags from user prompt)\n- CURSOR_IN_FENCED_CODE_BLOCK: Are you inside a code fence?\n- CURSOR_FENCE_LANGUAGE: What language is the current fence?\n- PREFIX_ENDS_WITH_NEWLINE: Does prefix end with newline?\n- SUFFIX_STARTS_WITH_NEWLINE: Does suffix start with newline?\n- MERMAID_CONTEXT: Is this a Mermaid diagram context?\n\nPRIORITY 2: SPECIALIZED CONTENT RULES\n\n2.1 Code Block Handling:\nIf CURSOR_IN_FENCED_CODE_BLOCK=true:\n- You are inside a code fence\n- Output code lines ONLY (no triple backticks)\n- Use single \\n for code line separation\n\nIf CURSOR_IN_FENCED_CODE_BLOCK=false and code needed:\n- Wrap code in fenced block with language tag:\n```{language}\ncode here\n```\n- Never use inline backticks for code snippets\n\n2.2 Math Formatting (KaTeX):\n- Inline math: wrap with $...$\n- Block math: wrap with $$...$$\n- Never output bare formulas\n- Exception: inside latex/tex/katex fence, output raw LaTeX\n\n2.3 Mermaid Diagrams:\nIf CURSOR_FENCE_LANGUAGE=mermaid:\n- Output Mermaid syntax ONLY\n- No backticks, no explanations\n\nIf MERMAID_CONTEXT=true and outside fence:\n- Output complete fenced block:\n```mermaid\ndiagram syntax\n```\n\nPRIORITY 3: MARKDOWN STRUCTURE\n\n3.1 Newline Semantics:\n- Single \\n: soft break (same paragraph, renders as space or <br>)\n- Double \\n\\n: hard break (new paragraph/block)\n- Use \\n\\n for: new paragraphs, before headings, starting lists/tables\n- Use \\n for: continuation within blocks (list items, table cells)\n- Exception: inside code blocks, use \\n freely for code lines\n\n3.2 Boundary Management:\nCheck PREFIX_ENDS_WITH_NEWLINE and SUFFIX_STARTS_WITH_NEWLINE:\n- If PREFIX lacks needed newline: start OUTPUT with \\n\n- If SUFFIX lacks needed newline: end OUTPUT with \\n\n- Common cases requiring leading \\n:\n* Starting a list after \"Steps:\"\n* Creating new paragraph after text\n* Adding heading after paragraph\n- Common cases requiring trailing \\n:\n* Before new heading\n* End of section\n\n3.3 Context Stitching:\n- Never repeat text from SUFFIX beginning\n- Match PREFIX tone, style, indentation\n- Continue structures: lists, tables, quotes, headings\n\nPRIORITY 4: HIDDEN CONTEXT\n- OCR metadata like <OCR:...> is hidden context\n- Never copy OCR tags to output\n- Use OCR content as semantic hint only"
|
||||
"template": "You are an inline completion engine for a {language_id} editor with ghost-text suggestions.\n\nReturn only the insertion text that should be placed between PREFIX and SUFFIX.\n\nCORE PRINCIPLE: Output insertion text only. No explanations, no meta labels, no wrapper quotes.\n\nPRIORITY 1: CONTEXT AWARENESS (Read these flags from user prompt)\n- CURSOR_IN_FENCED_CODE_BLOCK: Are you inside a code fence?\n- CURSOR_FENCE_LANGUAGE: What language is the current fence?\n- PREFIX_ENDS_WITH_NEWLINE: Does prefix end with newline?\n- SUFFIX_STARTS_WITH_NEWLINE: Does suffix start with newline?\n- MERMAID_CONTEXT: Is this a Mermaid diagram context?\n\nPRIORITY 2: SPECIALIZED CONTENT RULES\n\n2.1 Code Block Handling:\nIf CURSOR_IN_FENCED_CODE_BLOCK=true:\n- You are inside a code fence\n- Output code lines ONLY (no triple backticks)\n- Separate code lines with actual newline characters\n\nIf CURSOR_IN_FENCED_CODE_BLOCK=false and code needed:\n- Wrap code in fenced block with language tag:\n```{language}\ncode here\n```\n- Never use inline backticks for code snippets\n\n2.2 Math Formatting (KaTeX):\n- Inline math: wrap with $...$\n- Block math: wrap with $$...$$\n- Never output bare formulas\n- Exception: inside latex/tex/katex fence, output raw LaTeX\n\n2.3 Mermaid Diagrams:\nIf CURSOR_FENCE_LANGUAGE=mermaid:\n- Output Mermaid syntax ONLY\n- No backticks, no explanations\n\nIf MERMAID_CONTEXT=true and outside fence:\n- Output complete fenced block:\n```mermaid\ndiagram syntax\n```\n\nPRIORITY 3: MARKDOWN STRUCTURE\n\n3.1 Newline Semantics:\n- Use actual line breaks in output, not spelled-out escape sequences, unless the surrounding content explicitly needs that text\n- A single line break usually continues the current block\n- A blank line starts a new paragraph or block\n- Use blank lines for: new paragraphs, before headings, starting lists/tables\n- Use single line breaks for: continuation within blocks (list items, table cells)\n- Exception: inside code blocks, use actual newline characters freely for code lines\n\n3.2 Boundary Management:\nCheck PREFIX_ENDS_WITH_NEWLINE and SUFFIX_STARTS_WITH_NEWLINE:\n- If PREFIX lacks needed newline: start OUTPUT on a new line\n\n- If SUFFIX lacks needed newline: end OUTPUT with a trailing line break\n\n- Common cases requiring a leading line break:\n* Starting a list after \"Steps:\"\n* Creating new paragraph after text\n* Adding heading after paragraph\n- Common cases requiring a trailing line break:\n* Before new heading\n* End of section\n\n3.3 Context Stitching:\n- Never repeat text from SUFFIX beginning\n- Match PREFIX tone, style, indentation\n- Continue structures: lists, tables, quotes, headings\n\nPRIORITY 4: HIDDEN CONTEXT\n- OCR metadata like <OCR:...> is hidden context\n- Never copy OCR tags to output\n- Use OCR content as semantic hint only"
|
||||
}
|
||||
|
||||
@@ -1,295 +0,0 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
import argparse
|
||||
import uuid
|
||||
import sys
|
||||
import statistics
|
||||
import os
|
||||
from datetime import datetime
|
||||
from typing import List, Dict, Any, Optional, Tuple
|
||||
import httpx
|
||||
from pydantic import BaseModel
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
handlers=[logging.StreamHandler(sys.stdout)]
|
||||
)
|
||||
logger = logging.getLogger("api_benchmarker")
|
||||
|
||||
# Constants
|
||||
DEFAULT_BASE_URL = "http://localhost:8001"
|
||||
DEFAULT_API_KEY = "your-secret-key-here"
|
||||
CHARS_PER_TOKEN = 4
|
||||
|
||||
# Data Generators
|
||||
def get_dummy_base64_image():
|
||||
return "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII="
|
||||
|
||||
def get_dummy_base64_audio():
|
||||
# A bit longer dummy audio to pass validation (44 bytes header + some data)
|
||||
return "UklGRigAAABXQVZFZm10IBAAAAABAAEARKwAAIhYAQACABAAZGF0YQQAAAAAAA" + "A" * 100 + "=="
|
||||
|
||||
def generate_context_text(tokens: int) -> str:
|
||||
"""Generate synthetic text of approximately 'tokens' tokens."""
|
||||
base_phrase = "The quick brown fox jumps over the lazy dog. "
|
||||
repeat_count = (tokens * CHARS_PER_TOKEN) // len(base_phrase) + 1
|
||||
return (base_phrase * repeat_count)[:tokens * CHARS_PER_TOKEN]
|
||||
|
||||
# Metric Models
|
||||
class RequestMetric(BaseModel):
|
||||
task_name: str
|
||||
endpoint: str
|
||||
status_code: int
|
||||
ttfb_ms: float
|
||||
total_ms: float
|
||||
success: bool
|
||||
tokens: int
|
||||
error: Optional[str] = None
|
||||
|
||||
class BenchStats:
|
||||
def __init__(self, name: str):
|
||||
self.name = name
|
||||
self.metrics: List[RequestMetric] = []
|
||||
self.start_time = 0.0
|
||||
self.end_time = 0.0
|
||||
|
||||
def add(self, m: RequestMetric):
|
||||
self.metrics.append(m)
|
||||
|
||||
def get_summary(self) -> Dict[str, Any]:
|
||||
if not self.metrics:
|
||||
return {}
|
||||
|
||||
total = len(self.metrics)
|
||||
successes = [m for m in self.metrics if m.success]
|
||||
success_count = len(successes)
|
||||
fail_count = total - success_count
|
||||
|
||||
total_latencies = [m.total_ms for m in successes] if successes else [0]
|
||||
ttfb_latencies = [m.ttfb_ms for m in successes] if successes else [0]
|
||||
|
||||
duration = self.end_time - self.start_time
|
||||
total_tokens = sum(m.tokens for m in successes)
|
||||
|
||||
return {
|
||||
"name": self.name,
|
||||
"total_requests": total,
|
||||
"success_rate": (success_count / total) * 100 if total > 0 else 0,
|
||||
"avg_latency": statistics.mean(total_latencies),
|
||||
"p50_latency": statistics.median(total_latencies),
|
||||
"p95_latency": sorted(total_latencies)[int(len(total_latencies)*0.95)] if total_latencies else 0,
|
||||
"avg_ttfb": statistics.mean(ttfb_latencies),
|
||||
"tps": total_tokens / duration if duration > 0 else 0,
|
||||
"rps": total / duration if duration > 0 else 0,
|
||||
"duration": duration
|
||||
}
|
||||
|
||||
# Benchmarking Engine
|
||||
class ApiBenchmarker:
|
||||
def __init__(self, base_url: str, api_key: str):
|
||||
self.base_url = base_url
|
||||
self.api_key = api_key
|
||||
self.headers = {"X-API-Key": api_key}
|
||||
self.semaphores = {
|
||||
"completions": asyncio.Semaphore(5),
|
||||
"ocr": asyncio.Semaphore(2),
|
||||
"convert": asyncio.Semaphore(2),
|
||||
"tts-asr": asyncio.Semaphore(3)
|
||||
}
|
||||
self.results: Dict[str, BenchStats] = {}
|
||||
|
||||
async def _execute_request(self, client: httpx.AsyncClient, name: str, method: str, path: str, **kwargs) -> RequestMetric:
|
||||
url = f"{self.base_url}{path}"
|
||||
start = time.perf_counter()
|
||||
ttfb = 0.0
|
||||
tokens_count = 0
|
||||
|
||||
# Estimate input + output tokens (mock for output)
|
||||
if "json" in kwargs:
|
||||
input_text = str(kwargs["json"].get("prefix", "")) + str(kwargs["json"].get("text", ""))
|
||||
tokens_count += len(input_text) // CHARS_PER_TOKEN
|
||||
|
||||
try:
|
||||
async with client.stream(method, url, **kwargs) as response:
|
||||
ttfb = (time.perf_counter() - start) * 1000
|
||||
|
||||
body = await response.aread()
|
||||
total_ms = (time.perf_counter() - start) * 1000
|
||||
success = 200 <= response.status_code < 300
|
||||
|
||||
error_msg = None
|
||||
if not success:
|
||||
error_msg = body.decode(errors="ignore")[:200]
|
||||
else:
|
||||
# Estimate output tokens from response content
|
||||
try:
|
||||
resp_json = json.loads(body)
|
||||
content = resp_json.get("content", "") or resp_json.get("text", "") or resp_json.get("markdown", "")
|
||||
tokens_count += len(content) // CHARS_PER_TOKEN
|
||||
except:
|
||||
pass
|
||||
|
||||
return RequestMetric(
|
||||
task_name=name,
|
||||
endpoint=path,
|
||||
status_code=response.status_code,
|
||||
ttfb_ms=ttfb,
|
||||
total_ms=total_ms,
|
||||
success=success,
|
||||
tokens=tokens_count,
|
||||
error=error_msg
|
||||
)
|
||||
except Exception as e:
|
||||
total_ms = (time.perf_counter() - start) * 1000
|
||||
return RequestMetric(
|
||||
task_name=name,
|
||||
endpoint=path,
|
||||
status_code=0,
|
||||
ttfb_ms=ttfb or total_ms,
|
||||
total_ms=total_ms,
|
||||
success=False,
|
||||
tokens=tokens_count,
|
||||
error=str(e)
|
||||
)
|
||||
|
||||
async def run_task(self, client: httpx.AsyncClient, task_type: str, name: str, iterations: int):
|
||||
if name not in self.results:
|
||||
self.results[name] = BenchStats(name)
|
||||
|
||||
stats = self.results[name]
|
||||
stats.start_time = time.perf_counter()
|
||||
|
||||
sem = self.semaphores.get(task_type, self.semaphores["completions"])
|
||||
|
||||
async def worker():
|
||||
async with sem:
|
||||
if task_type == "completions":
|
||||
# Stability Test Variation
|
||||
prefix_len = 100
|
||||
if "Normal" in name: prefix_len = 1000
|
||||
if "Long" in name: prefix_len = 4000
|
||||
|
||||
metric = await self._execute_request(client, name, "POST", "/v1/completions", json={
|
||||
"prefix": generate_context_text(prefix_len),
|
||||
"suffix": "End of document.",
|
||||
"model_thinking": "low"
|
||||
})
|
||||
elif task_type == "ocr":
|
||||
metric = await self._execute_request(client, name, "POST", "/v1/ocr", json={
|
||||
"image": get_dummy_base64_image(),
|
||||
"filename": "bench.png"
|
||||
})
|
||||
elif task_type == "convert":
|
||||
metric = await self._execute_request(client, name, "POST", "/v1/convert", json={
|
||||
"file": base64.b64encode(b"Performance test data").decode(),
|
||||
"filename": "bench.txt"
|
||||
})
|
||||
elif task_type == "tts":
|
||||
metric = await self._execute_request(client, name, "POST", "/v1/tts-asr/tts", json={
|
||||
"text": "This is a performance benchmark for the text to speech engine.",
|
||||
"voice": "v2/en_speaker_6",
|
||||
"format": "wav"
|
||||
})
|
||||
elif task_type == "asr":
|
||||
metric = await self._execute_request(client, name, "POST", "/v1/tts-asr/asr", json={
|
||||
"audio_base64": get_dummy_base64_audio(),
|
||||
"language": "en"
|
||||
})
|
||||
else:
|
||||
metric = await self._execute_request(client, name, "GET", "/v1/tts-asr/status")
|
||||
|
||||
stats.add(metric)
|
||||
|
||||
tasks = [worker() for _ in range(iterations)]
|
||||
await asyncio.gather(*tasks)
|
||||
stats.end_time = time.perf_counter()
|
||||
|
||||
def generate_report(self, output_file: str):
|
||||
report = []
|
||||
report.append(f"# API Benchmarking Report ({datetime.now().strftime('%Y-%m-%d %H:%M:%S')})")
|
||||
report.append(f"\n**Base URL:** `{self.base_url}`")
|
||||
|
||||
# Summary Table
|
||||
report.append("\n## Executive Summary")
|
||||
report.append("| Task | Success Rate | Avg TTFB | Avg Latency | P95 Latency | TPS | RPS |")
|
||||
report.append("| :--- | :--- | :--- | :--- | :--- | :--- | :--- |")
|
||||
|
||||
for name, stats in self.results.items():
|
||||
s = stats.get_summary()
|
||||
if not s: continue
|
||||
report.append(f"| {s['name']} | {s['success_rate']:.1f}% | {s['avg_ttfb']:.1f}ms | {s['avg_latency']:.1f}ms | {s['p95_latency']:.1f}ms | {s['tps']:.1f} | {s['rps']:.2f} |")
|
||||
|
||||
# Stability Analysis
|
||||
report.append("\n## Stability & Context Analysis")
|
||||
report.append("Detailed analysis of how context length affects TTFB and overall performance.")
|
||||
|
||||
# Details per category
|
||||
for name, stats in self.results.items():
|
||||
s = stats.get_summary()
|
||||
if not s: continue
|
||||
report.append(f"\n### {name} Details")
|
||||
report.append(f"- **Total Samples:** {s['total_requests']}")
|
||||
report.append(f"- **Duration:** {s['duration']:.2f}s")
|
||||
failures = [m for m in stats.metrics if not m.success]
|
||||
if failures:
|
||||
report.append(f"- **Top Errors:**")
|
||||
for f in failures[:3]:
|
||||
report.append(f" - `[{f.status_code}]` {f.error}")
|
||||
|
||||
with open(output_file, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(report))
|
||||
logger.info(f"Report generated: {output_file}")
|
||||
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="Advanced LLM API Benchmarker")
|
||||
parser.add_argument("--url", default=DEFAULT_BASE_URL, help="Base URL")
|
||||
parser.add_argument("--key", default=DEFAULT_API_KEY, help="API Key")
|
||||
parser.add_argument("--c-comp", type=int, default=5, help="Completion Concurrency")
|
||||
parser.add_argument("--c-ocr", type=int, default=2, help="OCR Concurrency")
|
||||
parser.add_argument("--c-audio", type=int, default=2, help="TTS/ASR Concurrency")
|
||||
parser.add_argument("--iters", type=int, default=10, help="Iterations per test suite")
|
||||
parser.add_argument("--output", default="api_performance_report.md", help="Output report file")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
bench = ApiBenchmarker(args.url, args.key)
|
||||
bench.semaphores["completions"] = asyncio.Semaphore(args.c_comp)
|
||||
bench.semaphores["ocr"] = asyncio.Semaphore(args.c_ocr)
|
||||
bench.semaphores["tts-asr"] = asyncio.Semaphore(args.c_audio)
|
||||
|
||||
async with httpx.AsyncClient(headers=bench.headers, timeout=120.0) as client:
|
||||
logger.info("Starting Benchmark Suites...")
|
||||
|
||||
# Suite 1: Stability - Completion Contexts
|
||||
logger.info("Running Stability Suite (Short Context)...")
|
||||
await bench.run_task(client, "completions", "Completion-Short", args.iters)
|
||||
|
||||
logger.info("Running Stability Suite (Normal Context)...")
|
||||
await bench.run_task(client, "completions", "Completion-Normal", args.iters)
|
||||
|
||||
logger.info("Running Stability Suite (Long Context)...")
|
||||
await bench.run_task(client, "completions", "Completion-Long", args.iters)
|
||||
|
||||
# Suite 2: Functional Concurrency
|
||||
logger.info("Running OCR Concurrency Suite...")
|
||||
await bench.run_task(client, "ocr", "OCR-Concurrent", args.iters)
|
||||
|
||||
logger.info("Running TTS Concurrency Suite...")
|
||||
await bench.run_task(client, "tts", "TTS-Concurrent", args.iters)
|
||||
|
||||
logger.info("Running ASR Concurrency Suite...")
|
||||
await bench.run_task(client, "asr", "ASR-Concurrent", args.iters)
|
||||
|
||||
logger.info("Running File Transformation Suite...")
|
||||
await bench.run_task(client, "convert", "Convert-Concurrent", args.iters)
|
||||
|
||||
bench.generate_report(args.output)
|
||||
print(f"\nBenchmark Complete! View the report at: {args.output}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -1,80 +0,0 @@
|
||||
"""
|
||||
GeoIP2 IP归属地查询测试脚本
|
||||
|
||||
使用方法：
|
||||
1. 安装依赖：pip install geoip2
|
||||
2. 下载数据库：https://dev.maxmind.com/geoip/geoip2/geolite2/
|
||||
3. 运行测试：python test_geoip.py
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
import geoip2.database
|
||||
except ImportError:
|
||||
print("请先安装 geoip2: pip install geoip2")
|
||||
sys.exit(1)
|
||||
|
||||
DB_PATH = os.path.join(os.path.dirname(__file__), "GeoLite2-City.mmdb")
|
||||
|
||||
TEST_IPS = [
|
||||
"8.8.8.8", # Google DNS (美国)
|
||||
"114.114.114.114", # 114 DNS (中国南京)
|
||||
"223.5.5.5", # 阿里DNS (中国杭州)
|
||||
"1.1.1.1", # Cloudflare DNS (澳大利亚)
|
||||
"119.29.29.29", # 腾讯DNS (中国)
|
||||
]
|
||||
|
||||
|
||||
def get_location(reader, ip: str) -> dict:
|
||||
try:
|
||||
response = reader.city(ip)
|
||||
return {
|
||||
"ip": ip,
|
||||
"country": response.country.name,
|
||||
"country_code": response.country.iso_code,
|
||||
"region": response.subdivisions.most_specific.name if response.subdivisions else None,
|
||||
"city": response.city.name,
|
||||
"latitude": response.location.latitude,
|
||||
"longitude": response.location.longitude,
|
||||
"timezone": response.location.time_zone,
|
||||
}
|
||||
except geoip2.errors.AddressNotFoundError:
|
||||
return {"ip": ip, "error": "IP未在数据库中找到"}
|
||||
except Exception as e:
|
||||
return {"ip": ip, "error": str(e)}
|
||||
|
||||
|
||||
def main():
|
||||
if not os.path.exists(DB_PATH):
|
||||
print(f"数据库文件不存在: {DB_PATH}")
|
||||
print("请从 https://dev.maxmind.com/geoip/geoip2/geolite2/ 下载 GeoLite2-City.mmdb")
|
||||
return
|
||||
|
||||
print(f"加载数据库: {DB_PATH}")
|
||||
reader = geoip2.database.Reader(DB_PATH)
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("IP归属地查询测试")
|
||||
print("=" * 60)
|
||||
|
||||
for ip in TEST_IPS:
|
||||
result = get_location(reader, ip)
|
||||
if "error" in result:
|
||||
print(f"\n{ip}: {result['error']}")
|
||||
else:
|
||||
print(f"\n{ip}:")
|
||||
print(f" 国家: {result['country']} ({result['country_code']})")
|
||||
print(f" 地区: {result['region'] or '未知'}")
|
||||
print(f" 城市: {result['city'] or '未知'}")
|
||||
print(f" 坐标: {result['latitude']}, {result['longitude']}")
|
||||
print(f" 时区: {result['timezone']}")
|
||||
|
||||
reader.close()
|
||||
print("\n" + "=" * 60)
|
||||
print("测试完成")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,45 +1,62 @@
|
||||
OVERVIEW: pytest 测试套件，覆盖率要求 90%
|
||||
STRUCTURE：
|
||||
- test_*.py - 各模块测试
|
||||
- run_tests.py - 测试执行脚本（unit/integration/all）
|
||||
- simulate_macos.py - macOS 环境模拟
|
||||
- TESTING_GUIDE.md - 测试指南文档
|
||||
# Backend Tests 测试指引
|
||||
|
||||
WHERE TO LOOK
|
||||
表格
|
||||
本文件适用于 backend/tests/ 下的测试和测试脚本。
|
||||
|
||||
| Area | Path |
|
||||
|---|---|
|
||||
| 单元测试 | backend/tests/ |
|
||||
| 集成测试 | backend/tests/ |
|
||||
| 测试执行脚本 | backend/tests/run_tests.py |
|
||||
| macOS 模拟 | backend/tests/simulate_macos.py |
|
||||
| 测试指南 | backend/tests/TESTING_GUIDE.md |
|
||||
## 测试入口
|
||||
|
||||
运行命令：
|
||||
- pytest - 运行所有测试
|
||||
- python backend/tests/run_tests.py unit - 单元测试
|
||||
- python backend/tests/run_tests.py integration - 集成测试
|
||||
- pytest.ini 指定默认测试目录为 backend/tests，并设置后端覆盖率门槛为 90%。
|
||||
- run_tests.py 提供 unit、integration、simulate、all 几种快捷入口。
|
||||
- 默认优先使用 pytest 跑窄测试；只有在需要脚本封装参数时再用 run_tests.py。
|
||||
|
||||
测试命名约定：test_*.py、Test* 类、test_* 函数
|
||||
## 测试分布
|
||||
|
||||
ANTI-PATTERNS：删除测试以通过覆盖率
|
||||
- test_main_endpoints.py：主 API 路由行为
|
||||
- test_main_cancel.py：补全取消和任务生命周期
|
||||
- test_prompt.py、test_prompt_extended.py：Prompt 上下文与规则
|
||||
- test_llm.py、test_llm_extended.py：LLM 包装层
|
||||
- test_geoip.py：GeoIP 逻辑
|
||||
- test_tts_asr_*.py：TTS 相关与历史 TTS/ASR 面
|
||||
- simulate_macos.py：历史模拟脚本
|
||||
- quick_verify.py、verify_cross.py、play_audio.py：人工验证或辅助脚本
|
||||
|
||||
验证
|
||||
- 保证测试覆盖率≥90% 时，报告合格
|
||||
- 使用 CI 运行 pytest，确保通过率
|
||||
## 常用命令
|
||||
|
||||
注意事项
|
||||
- 不要重复父目录内容
|
||||
- 不要超过 60 行
|
||||
- pytest
|
||||
- pytest backend/tests/test_main_endpoints.py -v
|
||||
- pytest backend/tests/test_main_cancel.py -v
|
||||
- pytest backend/tests/test_prompt.py -v
|
||||
- pytest backend/tests/test_llm.py -v
|
||||
- python backend/tests/run_tests.py unit
|
||||
- python backend/tests/run_tests.py integration --url http://localhost:8001 --key your-secret-key-here
|
||||
|
||||
测试应尽量独立，不要依赖全局状态
|
||||
- 运行单元测试时应使用 unit 标签
|
||||
- 运行集成测试时应使用 integration 标签
|
||||
## 测试原则
|
||||
|
||||
区分环境
|
||||
- unit 测试应尽量快速、稳定
|
||||
- integration 测试应覆盖接口和数据库交互
|
||||
- 优先跑与改动直接对应的窄测试，不要动不动全量跑。
|
||||
- 单元测试尽量 mock 掉外部依赖，不要直连真实 Ollama。
|
||||
- 涉及 main.py 时，优先用 monkeypatch 或 fake 对象替代：
|
||||
- call_ollama
|
||||
- call_vlm_ocr
|
||||
- MarkItDown
|
||||
- GeoIP 查询
|
||||
- TTS 模型加载
|
||||
- 测试要保持确定性，不依赖全局状态、环境顺序或人工输入。
|
||||
|
||||
维护
|
||||
- 如扩展新模块，优先增加 test_*.py 文件并在其中添加对应的测试类和方法
|
||||
## 容易误判的点
|
||||
|
||||
- 覆盖率门槛是针对多个 backend 模块一起算的，改核心文件时，即使单测通过也可能因为覆盖率不够失败。
|
||||
- htmlcov、.pytest_cache、api_performance_report.md 属于生成产物，不是需要维护的源码。
|
||||
- 这一目录里有一批 TTS/ASR 测试和说明明显继承自旧实现；当它们与当前 backend/tts_asr.py 冲突时，不要默认代码错了，先确认目标产品面。
|
||||
- integration 脚本通常假设本地服务在 http://localhost:8001，且默认 API Key 还是占位值。
|
||||
|
||||
## 改动定位建议
|
||||
|
||||
- 路由返回值不对：先看 test_main_endpoints.py 和 test_main_cancel.py
|
||||
- Prompt 规则不对：先看 test_prompt.py 和 test_prompt_extended.py
|
||||
- Ollama 调用包装不对：先看 test_llm.py 和 test_llm_extended.py
|
||||
- TTS 面变化：先确认当前 backend/tts_asr.py 是不是仍然以旧文档描述为目标，再决定修测试还是修实现
|
||||
|
||||
## 维护原则
|
||||
|
||||
- 新增后端行为时，优先给对应模块补测试，不要只依赖全量回归。
|
||||
- 如果变更的是历史 TTS/ASR 面，先把“当前规范是什么”确定下来，再批量修测试。
|
||||
- 如果覆盖率策略变化，记得同步这个文件，而不是只改 pytest.ini。
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../..')))
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
|
||||
|
||||
from backend.tts_asr import _tts_sync_with_retry
|
||||
|
||||
async def play_audio():
|
||||
print("生成测试音频中，请稍候...")
|
||||
test_text = "这是一段用以测试新语音模型音质的中文合成音频。"
|
||||
|
||||
try:
|
||||
audio_bytes, sr = await _tts_sync_with_retry(test_text, rate=1.0)
|
||||
|
||||
# 保存到本地文件
|
||||
wav_path = os.path.join(os.path.dirname(__file__), "test_audio.wav")
|
||||
with open(wav_path, "wb") as f:
|
||||
f.write(audio_bytes)
|
||||
|
||||
print(f"音频已生成并保存到: {wav_path}")
|
||||
print("正在尝试在 macOS 上播放...")
|
||||
|
||||
# Mac OS 的播放命令
|
||||
os.system(f"afplay '{wav_path}'")
|
||||
|
||||
print("播放完成。")
|
||||
|
||||
except Exception as e:
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
print(f"音频生成失败: {str(e)}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(play_audio())
|
||||
@@ -12,7 +12,6 @@ macOS环境模拟测试工具
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import importlib
|
||||
import os
|
||||
import platform
|
||||
import sys
|
||||
|
||||
Binary file not shown.
@@ -1,5 +1,4 @@
|
||||
import sys
|
||||
import os
|
||||
import types
|
||||
import pathlib
|
||||
import pytest
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import os
|
||||
import sys
|
||||
import base64
|
||||
import asyncio
|
||||
import types
|
||||
import pytest
|
||||
from unittest.mock import MagicMock
|
||||
from fastapi.testclient import TestClient
|
||||
@@ -11,6 +11,10 @@ BACKEND_DIR = os.path.abspath(os.path.join(CURRENT_DIR, ".."))
|
||||
if BACKEND_DIR not in sys.path:
|
||||
sys.path.insert(0, BACKEND_DIR)
|
||||
|
||||
fake_tts_asr = types.ModuleType("tts_asr")
|
||||
fake_tts_asr.register_tts_asr_routes = lambda app: None
|
||||
sys.modules.setdefault("tts_asr", fake_tts_asr)
|
||||
|
||||
import main # type: ignore
|
||||
|
||||
API_KEY = main.API_KEY
|
||||
@@ -61,13 +65,13 @@ def test_sanitize_markdown_strips_img_tag():
|
||||
assert "<img" not in main._sanitize_converted_markdown("<img src='x.png'/>")
|
||||
|
||||
|
||||
def test_sanitize_markdown_collapse_newlines():
|
||||
assert main._sanitize_converted_markdown("a\n\n\nb\n\n\n\nc") == "a\n\nb\n\nc"
|
||||
def test_sanitize_markdown_preserves_extra_newlines():
|
||||
assert main._sanitize_converted_markdown("a\n\n\nb\n\n\n\nc") == "a\n\n\nb\n\n\n\nc"
|
||||
|
||||
|
||||
def test_sanitize_markdown_normalize_crlf():
|
||||
result = main._sanitize_converted_markdown("line1\r\nline2\r\n")
|
||||
assert "line1\nline2" in result
|
||||
assert result == "line1\nline2\n"
|
||||
assert "\r" not in result
|
||||
|
||||
|
||||
|
||||
@@ -28,6 +28,13 @@ def test_prompt_builds_system_and_user():
|
||||
assert "MERMAID_CONTEXT" in user_prompt
|
||||
assert "PREFIX_ENDS_WITH_NEWLINE" in user_prompt
|
||||
assert "SUFFIX_STARTS_WITH_NEWLINE" in user_prompt
|
||||
assert "actual line breaks" in system_prompt
|
||||
assert "start OUTPUT on a new line" in system_prompt
|
||||
assert "Use real line breaks instead of spelled-out escape sequences" in user_prompt
|
||||
assert "make the first character of OUTPUT a real newline" in user_prompt
|
||||
assert "make the last character of OUTPUT a real newline" in user_prompt
|
||||
assert "start output with \\n" not in user_prompt
|
||||
assert "Use single \\n" not in system_prompt
|
||||
|
||||
|
||||
def test_cursor_in_fence_detection():
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@@ -15,9 +15,7 @@ TTS/ASR模块集成测试
|
||||
python backend/tests/test_tts_asr_integration.py --test config
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
@@ -12,8 +12,7 @@ TTS/ASR模块单元测试
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import Mock, MagicMock, patch
|
||||
import tempfile
|
||||
from unittest.mock import patch
|
||||
import numpy as np
|
||||
|
||||
# 确保可以导入backend和tts_asr模块
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import os
|
||||
import sys
|
||||
|
||||
# 确保能找到backend模块
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../..')))
|
||||
|
||||
from backend.tts_asr import _tts_sync_with_retry, _load_asr_pipeline_with_retry, _get_asr_pipeline
|
||||
|
||||
async def verify_tts_asr_cross():
|
||||
print("====================================")
|
||||
print(" 开始严格交叉验证: TTS 生成 -> ASR 解析")
|
||||
print("====================================")
|
||||
|
||||
test_text = "苹果设备支持离线大模型运算"
|
||||
print(f"\n[1] 正在调用 TTS 引擎 (suno/bark-small)...")
|
||||
print(f"目标文本: '{test_text}'")
|
||||
|
||||
try:
|
||||
# TTS生成
|
||||
audio_bytes, sr = await _tts_sync_with_retry(test_text, rate=1.0)
|
||||
print(f"-> TTS 成功生成音频数据，大小: {len(audio_bytes)} Bytes, 采样率: {sr}Hz")
|
||||
except Exception as e:
|
||||
print(f"-> TTS 失败: {str(e)}")
|
||||
sys.exit(1)
|
||||
|
||||
print("\n[2] 正在调用 ASR 引擎 (Whisper)...")
|
||||
try:
|
||||
loaded = await _load_asr_pipeline_with_retry()
|
||||
if not loaded:
|
||||
print("-> ASR 核心加载失败！")
|
||||
sys.exit(1)
|
||||
|
||||
print("-> ASR 加载成功，开始解析音频...")
|
||||
|
||||
# 将生成的wav bytes传递给ASR进行语音识别
|
||||
asr_pipeline = _get_asr_pipeline()
|
||||
result = asr_pipeline(audio_bytes, generate_kwargs={"task": "transcribe"})
|
||||
recognized_text = result.get('text', '')
|
||||
print(f"-> ASR 识别结果: '{recognized_text.strip()}'")
|
||||
|
||||
if len(recognized_text.strip()) > 0:
|
||||
print("\n结论: ✅ 验证成功！TTS和ASR模块功能链路闭环完成。")
|
||||
else:
|
||||
print("\n结论: ❌ ASR输出为空字符，闭环失败。")
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
print(f"-> ASR 分析阶段失败: {str(e)}")
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(verify_tts_asr_cross())
|
||||
1780
package-lock.json
generated
1780
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@@ -12,6 +12,10 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@blocknote/xl-docx-exporter": "^0.47.3",
|
||||
"@ffmpeg/core": "^0.12.10",
|
||||
"@ffmpeg/core-mt": "^0.12.10",
|
||||
"@ffmpeg/ffmpeg": "^0.12.15",
|
||||
"@ffmpeg/util": "^0.12.2",
|
||||
"@milkdown/core": "^7.18.0",
|
||||
"@milkdown/crepe": "^7.18.0",
|
||||
"@milkdown/kit": "^7.18.0",
|
||||
|
||||
BIN
sample-video.mp4
Normal file
BIN
sample-video.mp4
Normal file
Binary file not shown.
162
src/AGENTS.md
162
src/AGENTS.md
@@ -1,42 +1,134 @@
|
||||
# Src 前端模块指南
|
||||
# Src 前端指引
|
||||
|
||||
## OVERVIEW
|
||||
Vue3 前端核心，Milkdown 编辑器，AI 补全插件。
|
||||
本文件适用于 src/ 下的前端代码。进入 src/plugins/ 后，以子目录 AGENTS.md 为准。
|
||||
|
||||
## STRUCTURE
|
||||
- main.js - Vue 入口，Pinia + Router 挂载
|
||||
- App.vue - 根组件，主题/背景控制
|
||||
- plugins/ - Milkdown/Copilot 插件（见子目录 AGENTS.md）
|
||||
- components/ - Vue 组件（MilkdownEditor、SettingsPanel 等）
|
||||
- stores/ - Pinia 状态管理（settings.js）
|
||||
- views/ - 页面视图（EditorView、DocsView）
|
||||
- utils/ - 工具函数（api.js、config.js）
|
||||
- router/ - 路由定义
|
||||
## 前端职责
|
||||
|
||||
## WHERE TO LOOK
|
||||
- 启动 Vue 应用并挂载路由和 Pinia。
|
||||
- 提供编辑器界面、上传导出、OCR 触发、TTS 菜单、设置面板和多语言 UI。
|
||||
- 把编辑器状态、设置和网络请求接到后端接口。
|
||||
|
||||
| 任务 | 文件 | 说明 |
|
||||
|------|------|------|
|
||||
| Vue 入口 | main.js | createApp、Pinia、Router 挂载 |
|
||||
| 根组件 | App.vue | 主题切换、背景设置 |
|
||||
| 编辑器组件 | components/MilkdownEditor.vue | Milkdown 编辑器封装 |
|
||||
| AI 补全核心 | plugins/copilotPlugin.ts | ghost text、防抖、交互 |
|
||||
| 状态管理 | stores/settings.js | 用户设置、主题、偏好 |
|
||||
| API 调用 | utils/api.js | fetchSuggestion、TTS 接口 |
|
||||
## 先看哪里
|
||||
|
||||
## CONVENTIONS
|
||||
- JS/TS 2 空格缩进
|
||||
- 变量/函数：camelCase
|
||||
- Vue 组件：PascalCase
|
||||
- 文件名：全小写+短横线
|
||||
- 应用入口：main.js
|
||||
- 根组件：App.vue
|
||||
- 路由：router/index.js
|
||||
- 编辑页：views/EditorView.vue
|
||||
- 文档页：views/DocsView.vue
|
||||
- Univer 页：views/UniverView.vue
|
||||
- 编辑器主组件：components/MilkdownEditor.vue
|
||||
- 设置面板：components/SettingsPanel.vue
|
||||
- TTS 组件：components/TTSMenu.vue、components/TTSPlayer.vue
|
||||
- 插件层：plugins/
|
||||
- 设置状态：stores/settings.js
|
||||
- 请求层：utils/api.js
|
||||
- 环境配置：utils/config.js
|
||||
- 文件转换：utils/convert.js
|
||||
- OCR 缓存：utils/ocrCache.js
|
||||
- 文档块工具：utils/docBlock.js
|
||||
|
||||
## ANTI-PATTERNS
|
||||
- 在前端暴露密钥
|
||||
- 空 catch 块
|
||||
- 类型错误使用 as any
|
||||
## 路由事实
|
||||
|
||||
## 注意事项
|
||||
- 端口：5173
|
||||
- 启动：`npm run dev`
|
||||
- 构建：`npm run build`
|
||||
- UI 默认中文
|
||||
- / -> EditorView
|
||||
- /docs -> DocsView
|
||||
- /univer -> UniverView
|
||||
|
||||
## 核心行为
|
||||
|
||||
### 编辑器主控
|
||||
|
||||
- MilkdownEditor.vue 是前端最重要的控制点。
|
||||
- 它负责：
|
||||
- 创建 Crepe 编辑器
|
||||
- 注册 copilot、docBlock、mermaid 插件
|
||||
- 上传图片和文档
|
||||
- 触发 OCR
|
||||
- 导入导出 Markdown
|
||||
- 导出 DOCX 和 PDF
|
||||
- AI 开关
|
||||
- 32 KB 大小限制
|
||||
- TTS 菜单和播放器
|
||||
- 把 Markdown 更新回父组件
|
||||
|
||||
### 设置状态
|
||||
|
||||
- settings.js 负责持久化用户设置到 localStorage。
|
||||
- 当前重要字段包括：
|
||||
- theme
|
||||
- modelThinking
|
||||
- debounceMs
|
||||
- privacyMode
|
||||
- language
|
||||
- currency
|
||||
- backgroundType
|
||||
- backgroundImage
|
||||
- backgroundOpacity
|
||||
- ttsInstruct
|
||||
- initialMarkdown 也是由 store 推导出来的，编辑器初始化会消费它。
|
||||
|
||||
### 请求层
|
||||
|
||||
- utils/config.js 负责从 VITE_* 环境变量拼接接口地址。
|
||||
- utils/api.js 负责补全请求、取消补全、TTS 请求和状态请求。
|
||||
- fetchSuggestion 会：
|
||||
- 生成 request_id
|
||||
- 绑定 AbortSignal
|
||||
- 触发中止时的 cancel 请求
|
||||
- 读取 settings store 中的 thinking、privacy、language、currency、timezone 信息
|
||||
- 向后端发 POST 并读取 JSON
|
||||
|
||||
## 当前真实约定
|
||||
|
||||
- 这个目录不是纯 JS，也不是纯 TS，而是 Vue SFC + JS + TS 混合。
|
||||
- 组件、视图文件多为 PascalCase。
|
||||
- 工具和配置模块多为小写 .js。
|
||||
- 不要根据旧文档把前端文件统一改成另一种命名风格。
|
||||
- 代码风格存在历史混合，优先跟随当前文件周围风格，不要做无关格式化。
|
||||
|
||||
## 容易踩坑的点
|
||||
|
||||
- 文档超过 32 KB 时，AI 补全会被禁用。
|
||||
- 在文档块、Mermaid、LaTeX 等特定上下文中，部分 AI 行为和上传行为会被禁用或改道。
|
||||
- OCR 文本和文档块摘录会被注入补全上下文，但这些内容不应直接作为用户可见输出回写到文档。
|
||||
- 前端存在 /v1/export/pdf 调用，但调试前先确认后端是否真的实现了这个端点。
|
||||
- 当前前端多处仍保留占位 API Key 或默认值；不要把这种写法继续扩散到新代码。
|
||||
- 需要多语言 UI 时，优先走现有 i18n 结构，而不是在组件里新增硬编码文案。
|
||||
|
||||
## 调试路径
|
||||
|
||||
- 补全问题：
|
||||
components/MilkdownEditor.vue
|
||||
-> plugins/copilotPlugin.ts
|
||||
-> utils/api.js
|
||||
-> backend
|
||||
|
||||
- OCR 问题：
|
||||
components/MilkdownEditor.vue
|
||||
-> OCR 请求
|
||||
-> backend/main.py
|
||||
|
||||
- 文档导入转换问题：
|
||||
components/MilkdownEditor.vue
|
||||
-> utils/convert.js
|
||||
-> backend/main.py
|
||||
|
||||
- TTS 问题：
|
||||
components/TTSMenu.vue / components/TTSPlayer.vue / components/MilkdownEditor.vue
|
||||
-> utils/api.js
|
||||
-> backend/tts_asr.py
|
||||
|
||||
## 改动建议
|
||||
|
||||
- 编辑器相关问题优先从 MilkdownEditor.vue 入手，不要先到处搜。
|
||||
- 插件逻辑问题优先看 plugins/，尤其是 copilotPlugin.ts。
|
||||
- 状态问题优先看 stores/settings.js。
|
||||
- 网络问题优先看 utils/config.js 和 utils/api.js。
|
||||
- 改前端时尽量避免同时改样式、文案、结构和网络逻辑，多做局部可验证修改。
|
||||
|
||||
## 不要做的事
|
||||
|
||||
- 不要修改 milkdown-docs/。
|
||||
- 不要在组件里继续新增硬编码密钥。
|
||||
- 不要绕开现有 store 直接分散持久化设置。
|
||||
- 不要把隐藏上下文数据直接渲染进用户文档。
|
||||
- 不要因为某个文件风格不统一就顺手改整个 src/。
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
<template>
|
||||
<div class="doc-block" :class="{ collapsed: isCollapsed }">
|
||||
<!-- 深色条：文件头 -->
|
||||
<div class="doc-header">
|
||||
<!-- 最左边：文件类型icon -->
|
||||
<div class="doc-icon">
|
||||
<!-- PDF icon -->
|
||||
<svg v-if="docType === 'pdf'" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
|
||||
<polyline points="14 2 14 8 20 8"/>
|
||||
<path d="M9 15v-2h6v2"/>
|
||||
<path d="M12 13v4"/>
|
||||
</svg>
|
||||
<!-- Word icon -->
|
||||
<svg v-else-if="docType === 'doc'" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
|
||||
<polyline points="14 2 14 8 20 8"/>
|
||||
<path d="M16 13H8"/>
|
||||
<path d="M16 17H8"/>
|
||||
<path d="M10 9H8"/>
|
||||
</svg>
|
||||
<!-- PPT icon -->
|
||||
<svg v-else-if="docType === 'ppt'" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<rect x="2" y="3" width="20" height="14" rx="2"/>
|
||||
<path d="M8 21h8"/>
|
||||
<path d="M12 17v4"/>
|
||||
</svg>
|
||||
<!-- TXT icon -->
|
||||
<svg v-else width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
|
||||
<polyline points="14 2 14 8 20 8"/>
|
||||
<path d="M16 13H8"/>
|
||||
<path d="M16 17H8"/>
|
||||
</svg>
|
||||
</div>
|
||||
|
||||
<!-- 中间：文件名 -->
|
||||
<div class="doc-name">{{ docName }}</div>
|
||||
|
||||
<!-- 最右边：下载按钮 + 折叠按钮 -->
|
||||
<div class="doc-actions">
|
||||
<button @click="downloadDoc" class="action-btn" title="下载文档">
|
||||
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/>
|
||||
<polyline points="7 10 12 15 17 10"/>
|
||||
<line x1="12" y1="15" x2="12" y2="3"/>
|
||||
</svg>
|
||||
</button>
|
||||
<button @click="toggleCollapse" class="action-btn collapse-btn" :title="isCollapsed ? '展开' : '折叠'">
|
||||
<svg v-if="isCollapsed" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<polyline points="9 18 15 12 9 6"/>
|
||||
</svg>
|
||||
<svg v-else width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<polyline points="6 9 12 15 18 9"/>
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 浅色块：文档内容（非折叠状态显示） -->
|
||||
<div class="doc-content" v-show="!isCollapsed">
|
||||
<pre>{{ content }}</pre>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { ref } from 'vue'
|
||||
|
||||
const props = defineProps({
|
||||
docType: {
|
||||
type: String,
|
||||
default: 'text'
|
||||
},
|
||||
docName: {
|
||||
type: String,
|
||||
default: 'document.txt'
|
||||
},
|
||||
uploadTime: {
|
||||
type: String,
|
||||
default: ''
|
||||
},
|
||||
content: {
|
||||
type: String,
|
||||
default: ''
|
||||
}
|
||||
})
|
||||
|
||||
const isCollapsed = ref(false)
|
||||
|
||||
const toggleCollapse = () => {
|
||||
isCollapsed.value = !isCollapsed.value
|
||||
}
|
||||
|
||||
const downloadDoc = () => {
|
||||
const blob = new Blob([props.content], { type: 'text/plain;charset=utf-8' })
|
||||
const url = URL.createObjectURL(blob)
|
||||
const a = document.createElement('a')
|
||||
a.href = url
|
||||
a.download = props.docName
|
||||
document.body.appendChild(a)
|
||||
a.click()
|
||||
a.remove()
|
||||
URL.revokeObjectURL(url)
|
||||
}
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.doc-block {
|
||||
margin: 8px 0;
|
||||
border-radius: 10px;
|
||||
overflow: hidden;
|
||||
background: rgba(255, 255, 255, 0.8);
|
||||
backdrop-filter: blur(12px);
|
||||
border: 1px solid rgba(59, 130, 246, 0.15);
|
||||
box-shadow: 0 2px 8px rgba(59, 130, 246, 0.06), 0 1px 2px rgba(0, 0, 0, 0.04);
|
||||
}
|
||||
|
||||
:root[data-theme='dark'] .doc-block {
|
||||
background: rgba(26, 30, 39, 0.8);
|
||||
border-color: rgba(96, 165, 250, 0.15);
|
||||
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.3), 0 1px 2px rgba(0, 0, 0, 0.2);
|
||||
}
|
||||
|
||||
.doc-block.collapsed .doc-content {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.doc-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding: 6px 10px;
|
||||
background: rgba(255, 255, 255, 0.8);
|
||||
border-bottom: 1px solid rgba(59, 130, 246, 0.12);
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
:root[data-theme='dark'] .doc-header {
|
||||
background: rgba(26, 30, 39, 0.8);
|
||||
border-bottom-color: rgba(96, 165, 250, 0.15);
|
||||
}
|
||||
|
||||
.doc-icon {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: #3b82f6;
|
||||
}
|
||||
|
||||
.doc-name {
|
||||
flex: 1;
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
color: #1e293b;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
:root[data-theme='dark'] .doc-name {
|
||||
color: #e5e7eb;
|
||||
}
|
||||
|
||||
.doc-actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 2px;
|
||||
}
|
||||
|
||||
.action-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
padding: 0;
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: #64748b;
|
||||
cursor: pointer;
|
||||
border-radius: 6px;
|
||||
opacity: 0.75;
|
||||
}
|
||||
|
||||
.action-btn:hover {
|
||||
background: rgba(59, 130, 246, 0.1);
|
||||
color: #3b82f6;
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.doc-content {
|
||||
padding: 8px 10px;
|
||||
background: rgba(248, 250, 252, 0.8);
|
||||
max-height: 240px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
:root[data-theme='dark'] .doc-content {
|
||||
background: rgba(18, 22, 30, 0.8);
|
||||
}
|
||||
|
||||
.doc-content pre {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Fira Mono', monospace;
|
||||
font-size: 12px;
|
||||
line-height: 1.5;
|
||||
color: #334155;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
:root[data-theme='dark'] .doc-content pre {
|
||||
color: #cbd5e1;
|
||||
}
|
||||
</style>
|
||||
@@ -2,6 +2,8 @@
|
||||
import { computed, onBeforeUnmount, ref, watch } from 'vue'
|
||||
import MarkdownIt from 'markdown-it'
|
||||
import { isOfficeFile, getOfficeFormat } from '../services/officeDetection'
|
||||
import { canPlayVideoNatively, editVideoBlob, isVideoFile, transcodeVideoToMp4 } from '../utils/ffmpeg'
|
||||
import ImageEditorComponent from './ImageEditorComponent.vue'
|
||||
import UniverPreview from './UniverPreview.vue'
|
||||
|
||||
const props = defineProps({
|
||||
@@ -9,10 +11,12 @@ const props = defineProps({
|
||||
breadcrumb: { type: Array, default: () => [] },
|
||||
rootNodes: { type: Array, default: () => [] },
|
||||
getFileIcon: { type: Function, default: () => 'file' },
|
||||
getFileBlob: { type: Function, default: () => null }
|
||||
getFileBlob: { type: Function, default: () => null },
|
||||
updateFile: { type: Function, default: null },
|
||||
showSidebarToggle: { type: Boolean, default: false }
|
||||
})
|
||||
|
||||
const emit = defineEmits(['navigate'])
|
||||
const emit = defineEmits(['navigate', 'toggle-sidebar'])
|
||||
|
||||
const md = new MarkdownIt({
|
||||
html: false,
|
||||
@@ -20,7 +24,27 @@ const md = new MarkdownIt({
|
||||
linkify: true
|
||||
})
|
||||
|
||||
const objectUrl = ref('')
|
||||
const basePreviewUrl = ref('')
|
||||
const editedVideoUrl = ref('')
|
||||
const editedVideoBlob = ref(null)
|
||||
const imageEditorRef = ref(null)
|
||||
const imageEditorError = ref('')
|
||||
const isEditingImage = ref(false)
|
||||
const isSavingImage = ref(false)
|
||||
const isTranscodingVideo = ref(false)
|
||||
const transcodeProgress = ref(0)
|
||||
const videoPreviewError = ref('')
|
||||
const isUsingTranscodedVideo = ref(false)
|
||||
const isApplyingVideoEdit = ref(false)
|
||||
const videoEditProgress = ref(0)
|
||||
const videoEditError = ref('')
|
||||
const videoElement = ref(null)
|
||||
const videoDurationSeconds = ref(0)
|
||||
const videoCurrentTime = ref(0)
|
||||
const clipStartSeconds = ref(0)
|
||||
const clipEndSeconds = ref(0)
|
||||
const muteVideoAudio = ref(false)
|
||||
let previewTaskId = 0
|
||||
|
||||
const isRoot = computed(() => !props.node)
|
||||
const isFolder = computed(() => props.node?.type === 'folder')
|
||||
@@ -38,6 +62,7 @@ const isPdf = computed(() => {
|
||||
const mime = String(props.node?.mimeType || '')
|
||||
return mime === 'application/pdf' || fileExt.value === 'pdf'
|
||||
})
|
||||
const isVideo = computed(() => isVideoFile(props.node))
|
||||
const isText = computed(() => {
|
||||
const mime = String(props.node?.mimeType || '')
|
||||
return Boolean(props.node?.content || props.node?.previewText) || mime.startsWith('text/') || mime.includes('json') || ['txt', 'json', 'js', 'jsx', 'ts', 'tsx', 'css', 'html', 'htm', 'py', 'vue', 'xml', 'yaml', 'yml', 'csv', 'log', 'sql', 'toml', 'ini', 'cfg', 'conf', 'sh', 'bat', 'ps1', 'java', 'c', 'cpp', 'h', 'hpp', 'go', 'rs'].includes(fileExt.value)
|
||||
@@ -68,18 +93,161 @@ const locLabel = computed(() => {
|
||||
const count = lineCount.value
|
||||
return count ? `${count} 行` : '二进制文件'
|
||||
})
|
||||
const objectUrl = computed(() => editedVideoUrl.value || basePreviewUrl.value)
|
||||
const canEditImage = computed(() => isImage.value && typeof props.updateFile === 'function')
|
||||
const hasEditedVideo = computed(() => editedVideoBlob.value instanceof Blob)
|
||||
const normalizedClipStart = computed(() => clampNumber(clipStartSeconds.value, 0, videoDurationSeconds.value || 0))
|
||||
const normalizedClipEnd = computed(() => resolveClipEnd(clipEndSeconds.value, normalizedClipStart.value, videoDurationSeconds.value || 0))
|
||||
const selectedClipDuration = computed(() => Math.max(0, normalizedClipEnd.value - normalizedClipStart.value))
|
||||
const hasPendingVideoEdit = computed(() => {
|
||||
if (!isVideo.value || !videoDurationSeconds.value) return false
|
||||
const duration = videoDurationSeconds.value
|
||||
return normalizedClipStart.value > 0.05 || Math.abs(normalizedClipEnd.value - duration) > 0.05 || muteVideoAudio.value
|
||||
})
|
||||
|
||||
function clampNumber(value, min, max) {
|
||||
const numeric = Number(value)
|
||||
if (!Number.isFinite(numeric)) return min
|
||||
return Math.min(Math.max(numeric, min), max)
|
||||
}
|
||||
|
||||
function roundSeconds(value = 0) {
|
||||
return Math.round((Number(value) || 0) * 10) / 10
|
||||
}
|
||||
|
||||
function resolveClipEnd(value, start, duration) {
|
||||
if (!duration) return 0
|
||||
const numeric = Number(value)
|
||||
if (!Number.isFinite(numeric) || numeric <= 0) return duration
|
||||
return clampNumber(numeric, start, duration)
|
||||
}
|
||||
|
||||
function stripExtension(name = '') {
|
||||
return String(name).replace(/\.[^.]+$/, '')
|
||||
}
|
||||
|
||||
function buildEditedVideoName(name = 'video.mp4') {
|
||||
return `${stripExtension(name)}-edited.mp4`
|
||||
}
|
||||
|
||||
function revokeUrl(url = '') {
|
||||
if (!url) return
|
||||
URL.revokeObjectURL(url)
|
||||
}
|
||||
|
||||
function clearEditedVideoResult() {
|
||||
revokeUrl(editedVideoUrl.value)
|
||||
editedVideoUrl.value = ''
|
||||
editedVideoBlob.value = null
|
||||
}
|
||||
|
||||
function clearBasePreview() {
|
||||
revokeUrl(basePreviewUrl.value)
|
||||
basePreviewUrl.value = ''
|
||||
}
|
||||
|
||||
function resetVideoPreviewState() {
|
||||
isTranscodingVideo.value = false
|
||||
transcodeProgress.value = 0
|
||||
videoPreviewError.value = ''
|
||||
isUsingTranscodedVideo.value = false
|
||||
}
|
||||
|
||||
function resetVideoEditState() {
|
||||
isApplyingVideoEdit.value = false
|
||||
videoEditProgress.value = 0
|
||||
videoEditError.value = ''
|
||||
videoDurationSeconds.value = 0
|
||||
videoCurrentTime.value = 0
|
||||
clipStartSeconds.value = 0
|
||||
clipEndSeconds.value = 0
|
||||
muteVideoAudio.value = false
|
||||
clearEditedVideoResult()
|
||||
}
|
||||
|
||||
function normalizeVideoEditRange() {
|
||||
if (!videoDurationSeconds.value) {
|
||||
clipStartSeconds.value = 0
|
||||
clipEndSeconds.value = 0
|
||||
return
|
||||
}
|
||||
|
||||
clipStartSeconds.value = roundSeconds(normalizedClipStart.value)
|
||||
clipEndSeconds.value = roundSeconds(normalizedClipEnd.value)
|
||||
}
|
||||
|
||||
function assignBasePreviewUrl(url = '') {
|
||||
clearBasePreview()
|
||||
basePreviewUrl.value = url
|
||||
}
|
||||
|
||||
function setVideoRangeFromDuration(duration = 0) {
|
||||
videoDurationSeconds.value = duration
|
||||
videoCurrentTime.value = 0
|
||||
clipStartSeconds.value = 0
|
||||
clipEndSeconds.value = roundSeconds(duration)
|
||||
}
|
||||
|
||||
watch(
|
||||
fileBlob,
|
||||
(blob) => {
|
||||
if (objectUrl.value) URL.revokeObjectURL(objectUrl.value)
|
||||
objectUrl.value = blob instanceof Blob ? URL.createObjectURL(blob) : ''
|
||||
[fileBlob, () => props.node?.id],
|
||||
async ([blob]) => {
|
||||
const currentTaskId = ++previewTaskId
|
||||
clearBasePreview()
|
||||
resetVideoPreviewState()
|
||||
resetVideoEditState()
|
||||
|
||||
if (!(blob instanceof Blob)) return
|
||||
|
||||
if (!isVideo.value) {
|
||||
assignBasePreviewUrl(URL.createObjectURL(blob))
|
||||
return
|
||||
}
|
||||
|
||||
if (canPlayVideoNatively(props.node)) {
|
||||
assignBasePreviewUrl(URL.createObjectURL(blob))
|
||||
return
|
||||
}
|
||||
|
||||
isTranscodingVideo.value = true
|
||||
try {
|
||||
const transcodedBlob = await transcodeVideoToMp4(blob, props.node?.name || 'video', {
|
||||
onProgress(progress) {
|
||||
if (currentTaskId !== previewTaskId) return
|
||||
transcodeProgress.value = Math.max(transcodeProgress.value, Math.round(progress * 100))
|
||||
}
|
||||
})
|
||||
|
||||
if (currentTaskId !== previewTaskId) return
|
||||
|
||||
assignBasePreviewUrl(URL.createObjectURL(transcodedBlob))
|
||||
isUsingTranscodedVideo.value = true
|
||||
} catch (error) {
|
||||
if (currentTaskId !== previewTaskId) return
|
||||
videoPreviewError.value = error instanceof Error && error.message
|
||||
? error.message
|
||||
: '当前视频暂时无法在浏览器内转换预览'
|
||||
} finally {
|
||||
if (currentTaskId === previewTaskId) {
|
||||
isTranscodingVideo.value = false
|
||||
}
|
||||
}
|
||||
},
|
||||
{ immediate: true }
|
||||
)
|
||||
|
||||
watch(
|
||||
() => props.node?.id,
|
||||
() => {
|
||||
isEditingImage.value = false
|
||||
isSavingImage.value = false
|
||||
imageEditorError.value = ''
|
||||
}
|
||||
)
|
||||
|
||||
onBeforeUnmount(() => {
|
||||
if (objectUrl.value) URL.revokeObjectURL(objectUrl.value)
|
||||
previewTaskId += 1
|
||||
clearBasePreview()
|
||||
clearEditedVideoResult()
|
||||
})
|
||||
|
||||
function formatBytes(bytes = 0) {
|
||||
@@ -103,6 +271,166 @@ function navigateTo(id) {
|
||||
emit('navigate', id)
|
||||
}
|
||||
|
||||
function toggleSidebar() {
|
||||
emit('toggle-sidebar')
|
||||
}
|
||||
|
||||
function getImageExportFormat() {
|
||||
if (fileExt.value === 'jpg' || fileExt.value === 'jpeg') return 'jpeg'
|
||||
if (fileExt.value === 'webp') return 'webp'
|
||||
return 'png'
|
||||
}
|
||||
|
||||
function startImageEditing() {
|
||||
if (!canEditImage.value || !objectUrl.value || isSavingImage.value) return
|
||||
imageEditorError.value = ''
|
||||
isEditingImage.value = true
|
||||
}
|
||||
|
||||
function cancelImageEditing() {
|
||||
if (isSavingImage.value) return
|
||||
imageEditorError.value = ''
|
||||
isEditingImage.value = false
|
||||
}
|
||||
|
||||
function handleImageEditorError(message) {
|
||||
imageEditorError.value = String(message || '图片编辑器加载失败')
|
||||
}
|
||||
|
||||
async function saveImageEdits() {
|
||||
if (!props.node?.id || typeof props.updateFile !== 'function' || !imageEditorRef.value) return
|
||||
|
||||
isSavingImage.value = true
|
||||
imageEditorError.value = ''
|
||||
|
||||
try {
|
||||
const editedBlob = await imageEditorRef.value.exportImageBlob({
|
||||
format: getImageExportFormat(),
|
||||
quality: 0.92
|
||||
})
|
||||
|
||||
const updated = await props.updateFile(props.node.id, editedBlob, {
|
||||
mimeType: editedBlob.type || props.node?.mimeType || '',
|
||||
storageKind: 'blob',
|
||||
previewText: '',
|
||||
isTruncatedPreview: false
|
||||
})
|
||||
|
||||
if (updated === false) {
|
||||
throw new Error('保存图片失败')
|
||||
}
|
||||
|
||||
isEditingImage.value = false
|
||||
} catch (error) {
|
||||
imageEditorError.value = error instanceof Error && error.message
|
||||
? error.message
|
||||
: '保存图片失败'
|
||||
} finally {
|
||||
isSavingImage.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function formatDuration(seconds = 0) {
|
||||
const total = Math.max(0, Number(seconds) || 0)
|
||||
const hours = Math.floor(total / 3600)
|
||||
const minutes = Math.floor((total % 3600) / 60)
|
||||
const secs = total % 60
|
||||
const paddedSeconds = secs >= 10 ? secs.toFixed(1).padStart(4, '0') : secs.toFixed(1).padStart(3, '0')
|
||||
|
||||
if (hours > 0) {
|
||||
return `${String(hours).padStart(2, '0')}:${String(minutes).padStart(2, '0')}:${paddedSeconds.padStart(4, '0')}`
|
||||
}
|
||||
|
||||
return `${String(minutes).padStart(2, '0')}:${paddedSeconds.padStart(4, '0')}`
|
||||
}
|
||||
|
||||
function downloadBlob(blob, fileName) {
|
||||
if (!(blob instanceof Blob)) return
|
||||
const url = URL.createObjectURL(blob)
|
||||
const anchor = document.createElement('a')
|
||||
anchor.href = url
|
||||
anchor.download = fileName
|
||||
document.body.appendChild(anchor)
|
||||
anchor.click()
|
||||
anchor.remove()
|
||||
setTimeout(() => URL.revokeObjectURL(url), 0)
|
||||
}
|
||||
|
||||
function handleVideoLoadedMetadata(event) {
|
||||
const duration = Number(event.target?.duration || 0)
|
||||
if (!Number.isFinite(duration)) return
|
||||
setVideoRangeFromDuration(duration)
|
||||
}
|
||||
|
||||
function handleVideoTimeUpdate(event) {
|
||||
const currentTime = Number(event.target?.currentTime || 0)
|
||||
if (!Number.isFinite(currentTime)) return
|
||||
videoCurrentTime.value = currentTime
|
||||
}
|
||||
|
||||
function setClipStartFromCurrentTime() {
|
||||
if (!videoElement.value) return
|
||||
clipStartSeconds.value = roundSeconds(videoElement.value.currentTime || 0)
|
||||
if (clipEndSeconds.value < clipStartSeconds.value) {
|
||||
clipEndSeconds.value = roundSeconds(videoDurationSeconds.value || clipStartSeconds.value)
|
||||
}
|
||||
normalizeVideoEditRange()
|
||||
}
|
||||
|
||||
function setClipEndFromCurrentTime() {
|
||||
if (!videoElement.value) return
|
||||
clipEndSeconds.value = roundSeconds(videoElement.value.currentTime || 0)
|
||||
if (clipEndSeconds.value < clipStartSeconds.value) {
|
||||
clipStartSeconds.value = roundSeconds(Math.max(0, clipEndSeconds.value - 0.1))
|
||||
}
|
||||
normalizeVideoEditRange()
|
||||
}
|
||||
|
||||
async function applyVideoEdits() {
|
||||
if (!isVideo.value) return
|
||||
normalizeVideoEditRange()
|
||||
|
||||
const sourceBlob = hasEditedVideo.value ? editedVideoBlob.value : fileBlob.value
|
||||
if (!(sourceBlob instanceof Blob) || !videoDurationSeconds.value || !hasPendingVideoEdit.value) return
|
||||
|
||||
isApplyingVideoEdit.value = true
|
||||
videoEditProgress.value = 0
|
||||
videoEditError.value = ''
|
||||
|
||||
try {
|
||||
const editedBlob = await editVideoBlob(sourceBlob, hasEditedVideo.value ? buildEditedVideoName(props.node?.name || 'video.mp4') : props.node?.name || 'video.mp4', {
|
||||
startTime: normalizedClipStart.value,
|
||||
endTime: normalizedClipEnd.value,
|
||||
muteAudio: muteVideoAudio.value,
|
||||
onProgress(progress) {
|
||||
videoEditProgress.value = Math.max(videoEditProgress.value, Math.round(progress * 100))
|
||||
}
|
||||
})
|
||||
|
||||
clearEditedVideoResult()
|
||||
editedVideoBlob.value = editedBlob
|
||||
editedVideoUrl.value = URL.createObjectURL(editedBlob)
|
||||
} catch (error) {
|
||||
videoEditError.value = error instanceof Error && error.message
|
||||
? error.message
|
||||
: '视频编辑失败'
|
||||
} finally {
|
||||
isApplyingVideoEdit.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function restoreOriginalVideo() {
|
||||
clearEditedVideoResult()
|
||||
videoEditError.value = ''
|
||||
videoEditProgress.value = 0
|
||||
muteVideoAudio.value = false
|
||||
}
|
||||
|
||||
function downloadEditedVideo() {
|
||||
if (!(editedVideoBlob.value instanceof Blob)) return
|
||||
downloadBlob(editedVideoBlob.value, buildEditedVideoName(props.node?.name || 'video.mp4'))
|
||||
}
|
||||
|
||||
async function copyText() {
|
||||
if (!previewText.value) return
|
||||
try {
|
||||
@@ -120,13 +448,7 @@ function openRaw() {
|
||||
function downloadFile() {
|
||||
const blob = fileBlob.value
|
||||
if (!blob) return
|
||||
const url = objectUrl.value || URL.createObjectURL(blob)
|
||||
const anchor = document.createElement('a')
|
||||
anchor.href = url
|
||||
anchor.download = props.node?.name || 'download'
|
||||
document.body.appendChild(anchor)
|
||||
anchor.click()
|
||||
anchor.remove()
|
||||
downloadBlob(blob, props.node?.name || 'download')
|
||||
}
|
||||
</script>
|
||||
|
||||
@@ -182,35 +504,58 @@ function downloadFile() {
|
||||
|
||||
<template v-else-if="node">
|
||||
<div class="github-file-header">
|
||||
<div class="file-path-bar">
|
||||
<div class="file-path">
|
||||
<span v-for="(item, index) in breadcrumb" :key="item.id || item.name" class="file-path-item">
|
||||
<button
|
||||
v-if="index < breadcrumb.length - 1"
|
||||
type="button"
|
||||
class="breadcrumb-link"
|
||||
@click="navigateTo(item.id)"
|
||||
>
|
||||
{{ item.name }}
|
||||
</button>
|
||||
<span v-else class="breadcrumb-current">{{ item.name }}</span>
|
||||
<span v-if="index < breadcrumb.length - 1" class="path-sep">/</span>
|
||||
</span>
|
||||
</div>
|
||||
<button class="top-link" type="button">Top</button>
|
||||
</div>
|
||||
<div class="file-header-row">
|
||||
<button
|
||||
v-if="showSidebarToggle"
|
||||
class="sidebar-toggle-btn"
|
||||
type="button"
|
||||
title="切换左侧栏"
|
||||
@click="toggleSidebar"
|
||||
>
|
||||
<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true"><path d="M2.75 2A1.75 1.75 0 001 3.75v8.5C1 13.216 1.784 14 2.75 14h10.5A1.75 1.75 0 0015 12.25v-8.5A1.75 1.75 0 0013.25 2H2.75zm0 1.5h2.5v9h-2.5a.25.25 0 01-.25-.25v-8.5a.25.25 0 01.25-.25zm4 9v-9h6.5a.25.25 0 01.25.25v8.5a.25.25 0 01-.25.25h-6.5z"/></svg>
|
||||
</button>
|
||||
|
||||
<div class="file-meta-row">
|
||||
<div class="meta-tabs">
|
||||
<button class="meta-tab active" type="button">{{ isImage || isPdf ? 'Preview' : 'Code' }}</button>
|
||||
<button class="meta-tab" type="button" disabled>Blame</button>
|
||||
<span class="file-meta">{{ locLabel }}</span>
|
||||
<span class="file-meta">{{ fileSizeLabel }}</span>
|
||||
<span v-if="node.isTruncatedPreview" class="truncated-pill">仅预览前 2MB</span>
|
||||
<div class="file-header-main">
|
||||
<div class="file-path">
|
||||
<button type="button" class="breadcrumb-link" @click="navigateTo(null)">workspace</button>
|
||||
<template v-for="(item, index) in breadcrumb" :key="item.id || item.name">
|
||||
<span class="path-sep">/</span>
|
||||
<button
|
||||
v-if="index < breadcrumb.length - 1"
|
||||
type="button"
|
||||
class="breadcrumb-link"
|
||||
@click="navigateTo(item.id)"
|
||||
>
|
||||
{{ item.name }}
|
||||
</button>
|
||||
<span v-else class="breadcrumb-current">{{ item.name }}</span>
|
||||
</template>
|
||||
</div>
|
||||
|
||||
<div class="file-header-meta">
|
||||
<span class="file-meta">{{ fileSizeLabel }}</span>
|
||||
<span class="file-meta">{{ locLabel }}</span>
|
||||
<span v-if="node.isTruncatedPreview" class="truncated-pill">仅预览前 2MB</span>
|
||||
<span v-else-if="isTranscodingVideo" class="truncated-pill">视频转换中 {{ transcodeProgress }}%</span>
|
||||
<span v-else-if="isApplyingVideoEdit" class="truncated-pill">编辑处理中 {{ videoEditProgress }}%</span>
|
||||
<span v-else-if="isUsingTranscodedVideo" class="truncated-pill">已转为 MP4 预览</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="file-actions">
|
||||
<button class="action-btn" type="button" @click="openRaw" :disabled="!objectUrl">Raw</button>
|
||||
<button
|
||||
v-if="canEditImage && !isEditingImage"
|
||||
class="action-btn"
|
||||
type="button"
|
||||
@click="startImageEditing"
|
||||
:disabled="!objectUrl"
|
||||
>
|
||||
编辑图片
|
||||
</button>
|
||||
<template v-else-if="canEditImage && isEditingImage">
|
||||
<button class="action-btn" type="button" @click="cancelImageEditing" :disabled="isSavingImage">取消</button>
|
||||
<button class="action-btn primary-btn" type="button" @click="saveImageEdits" :disabled="isSavingImage">{{ isSavingImage ? '保存中' : '保存编辑' }}</button>
|
||||
</template>
|
||||
<button class="action-btn" type="button" @click="copyText" :disabled="!previewText">复制</button>
|
||||
<button class="action-btn" type="button" @click="downloadFile" :disabled="!fileBlob">下载</button>
|
||||
</div>
|
||||
@@ -231,11 +576,101 @@ function downloadFile() {
|
||||
</div>
|
||||
|
||||
<div v-else-if="isImage && objectUrl" class="content-preview">
|
||||
<div class="preview-surface image-surface">
|
||||
<div class="preview-surface image-surface" :class="{ 'image-editor-active': isEditingImage }">
|
||||
<div v-if="imageEditorError" class="image-inline-error">{{ imageEditorError }}</div>
|
||||
<ImageEditorComponent
|
||||
v-if="isEditingImage"
|
||||
ref="imageEditorRef"
|
||||
:image-url="objectUrl"
|
||||
:image-name="node.name"
|
||||
@error="handleImageEditorError"
|
||||
/>
|
||||
<img :src="objectUrl" :alt="node.name" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div v-else-if="isVideo" class="content-preview">
|
||||
<div class="preview-surface video-surface">
|
||||
<div v-if="isTranscodingVideo" class="video-state-card">
|
||||
<h3>正在准备视频预览</h3>
|
||||
<p>当前格式需要先转换为浏览器兼容的 MP4，转换完成后会自动开始播放。</p>
|
||||
<div class="video-progress-track" role="progressbar" :aria-valuenow="transcodeProgress" aria-valuemin="0" aria-valuemax="100">
|
||||
<span class="video-progress-bar" :style="{ width: `${transcodeProgress}%` }"></span>
|
||||
</div>
|
||||
<strong>{{ transcodeProgress }}%</strong>
|
||||
</div>
|
||||
|
||||
<div v-else-if="videoPreviewError" class="video-state-card video-state-error">
|
||||
<h3>视频预览暂时不可用</h3>
|
||||
<p>{{ videoPreviewError }}</p>
|
||||
<p>原始文件仍保存在浏览器本地，你可以继续下载后使用本地播放器打开。</p>
|
||||
</div>
|
||||
|
||||
<video
|
||||
v-else-if="objectUrl"
|
||||
ref="videoElement"
|
||||
class="video-player"
|
||||
controls
|
||||
playsinline
|
||||
:src="objectUrl"
|
||||
@loadedmetadata="handleVideoLoadedMetadata"
|
||||
@timeupdate="handleVideoTimeUpdate"
|
||||
></video>
|
||||
|
||||
<section v-if="objectUrl" class="video-edit-panel">
|
||||
<div class="video-edit-header">
|
||||
<div>
|
||||
<h3>基础编辑</h3>
|
||||
<p>裁剪片段、静音导出，并把结果另存为新的 MP4 文件。</p>
|
||||
</div>
|
||||
|
||||
<div class="video-edit-meta">
|
||||
<span>总时长 {{ formatDuration(videoDurationSeconds) }}</span>
|
||||
<span>当前播放 {{ formatDuration(videoCurrentTime) }}</span>
|
||||
<span>片段时长 {{ formatDuration(selectedClipDuration) }}</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="video-edit-grid">
|
||||
<label class="video-field">
|
||||
<span>开始时间（秒）</span>
|
||||
<input v-model.number="clipStartSeconds" type="number" min="0" :max="videoDurationSeconds || 0" step="0.1" @blur="normalizeVideoEditRange" />
|
||||
</label>
|
||||
|
||||
<label class="video-field">
|
||||
<span>结束时间（秒）</span>
|
||||
<input v-model.number="clipEndSeconds" type="number" min="0" :max="videoDurationSeconds || 0" step="0.1" @blur="normalizeVideoEditRange" />
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div class="video-quick-actions">
|
||||
<button class="ghost-btn" type="button" @click="setClipStartFromCurrentTime" :disabled="!videoDurationSeconds">用当前时间设为开始</button>
|
||||
<button class="ghost-btn" type="button" @click="setClipEndFromCurrentTime" :disabled="!videoDurationSeconds">用当前时间设为结束</button>
|
||||
</div>
|
||||
|
||||
<label class="video-checkbox">
|
||||
<input v-model="muteVideoAudio" type="checkbox" />
|
||||
<span>静音导出（移除音轨）</span>
|
||||
</label>
|
||||
|
||||
<div v-if="videoEditError" class="video-inline-error">{{ videoEditError }}</div>
|
||||
|
||||
<div v-if="isApplyingVideoEdit" class="video-edit-progress-shell">
|
||||
<div class="video-progress-track" role="progressbar" :aria-valuenow="videoEditProgress" aria-valuemin="0" aria-valuemax="100">
|
||||
<span class="video-progress-bar" :style="{ width: `${videoEditProgress}%` }"></span>
|
||||
</div>
|
||||
<strong>{{ videoEditProgress }}%</strong>
|
||||
</div>
|
||||
|
||||
<div class="video-edit-actions">
|
||||
<button class="action-btn primary-btn" type="button" @click="applyVideoEdits" :disabled="isApplyingVideoEdit || !hasPendingVideoEdit">生成编辑预览</button>
|
||||
<button class="action-btn" type="button" @click="downloadEditedVideo" :disabled="!hasEditedVideo">下载编辑结果</button>
|
||||
<button class="action-btn" type="button" @click="restoreOriginalVideo" :disabled="!hasEditedVideo">恢复原片</button>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div v-else-if="isPdf && objectUrl" class="content-preview">
|
||||
<iframe class="pdf-frame" :src="objectUrl" :title="node.name"></iframe>
|
||||
</div>
|
||||
@@ -411,17 +846,39 @@ function downloadFile() {
|
||||
background: var(--github-bg);
|
||||
}
|
||||
|
||||
.file-path-bar,
|
||||
.file-meta-row {
|
||||
.file-header-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 16px;
|
||||
padding: 12px 20px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.file-path-bar {
|
||||
border-bottom: 1px solid var(--github-border);
|
||||
.sidebar-toggle-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 8px;
|
||||
background: var(--github-bg);
|
||||
color: var(--github-text-secondary);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.sidebar-toggle-btn:hover {
|
||||
background: var(--github-hover);
|
||||
}
|
||||
|
||||
.file-header-main {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 14px;
|
||||
min-width: 0;
|
||||
flex: 1;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.file-path {
|
||||
@@ -432,12 +889,6 @@ function downloadFile() {
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.file-path-item {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
.breadcrumb-link {
|
||||
border: none;
|
||||
background: transparent;
|
||||
@@ -458,41 +909,18 @@ function downloadFile() {
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.top-link {
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--github-text-secondary);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.meta-tabs {
|
||||
.file-header-meta {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.meta-tab {
|
||||
height: 32px;
|
||||
padding: 0 12px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 8px;
|
||||
background: var(--github-hover);
|
||||
color: var(--github-text);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.meta-tab[disabled] {
|
||||
opacity: 0.56;
|
||||
}
|
||||
|
||||
.meta-tab.active {
|
||||
background: var(--github-bg);
|
||||
}
|
||||
|
||||
.file-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
flex-wrap: wrap;
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.action-btn {
|
||||
@@ -611,6 +1039,243 @@ function downloadFile() {
|
||||
box-shadow: 0 18px 40px rgba(15, 23, 42, 0.12);
|
||||
}
|
||||
|
||||
.image-surface.image-editor-active {
|
||||
display: block;
|
||||
padding: 0;
|
||||
background: transparent;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.image-surface.image-editor-active img {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.image-inline-error {
|
||||
margin-bottom: 14px;
|
||||
padding: 12px 14px;
|
||||
border: 1px solid color-mix(in srgb, #cf222e 35%, var(--github-border) 65%);
|
||||
border-radius: 10px;
|
||||
background: color-mix(in srgb, #cf222e 8%, var(--github-bg) 92%);
|
||||
color: #cf222e;
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.video-surface {
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
justify-content: flex-start;
|
||||
gap: 20px;
|
||||
min-height: 72vh;
|
||||
padding: 24px;
|
||||
background:
|
||||
radial-gradient(circle at top, rgba(9, 105, 218, 0.08), transparent 38%),
|
||||
linear-gradient(180deg, rgba(13, 17, 23, 0.04), rgba(13, 17, 23, 0.08));
|
||||
}
|
||||
|
||||
.video-player {
|
||||
align-self: center;
|
||||
width: min(1100px, 100%);
|
||||
max-height: 78vh;
|
||||
border-radius: 14px;
|
||||
background: #000;
|
||||
box-shadow: 0 20px 50px rgba(15, 23, 42, 0.22);
|
||||
}
|
||||
|
||||
.video-edit-panel {
|
||||
width: min(1100px, 100%);
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 16px;
|
||||
background: color-mix(in srgb, var(--github-bg) 94%, white 6%);
|
||||
box-shadow: 0 18px 40px rgba(15, 23, 42, 0.08);
|
||||
}
|
||||
|
||||
.video-edit-header {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
justify-content: space-between;
|
||||
gap: 16px;
|
||||
margin-bottom: 18px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.video-edit-header h3 {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.video-edit-header p {
|
||||
margin: 6px 0 0;
|
||||
color: var(--github-text-secondary);
|
||||
line-height: 1.7;
|
||||
}
|
||||
|
||||
.video-edit-meta {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.video-edit-meta span {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
min-height: 28px;
|
||||
padding: 0 10px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 999px;
|
||||
background: var(--github-hover);
|
||||
color: var(--github-text-secondary);
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.video-edit-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 14px;
|
||||
}
|
||||
|
||||
.video-field {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
color: var(--github-text-secondary);
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.video-field input {
|
||||
height: 40px;
|
||||
padding: 0 12px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 10px;
|
||||
background: var(--github-bg);
|
||||
color: var(--github-text);
|
||||
}
|
||||
|
||||
.video-quick-actions,
|
||||
.video-edit-actions {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.video-quick-actions {
|
||||
margin-top: 14px;
|
||||
}
|
||||
|
||||
.ghost-btn {
|
||||
height: 34px;
|
||||
padding: 0 12px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 999px;
|
||||
background: transparent;
|
||||
color: var(--github-text-secondary);
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.ghost-btn:hover:enabled {
|
||||
background: var(--github-hover);
|
||||
color: var(--github-text);
|
||||
}
|
||||
|
||||
.ghost-btn:disabled {
|
||||
opacity: 0.45;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.video-checkbox {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
margin-top: 14px;
|
||||
color: var(--github-text);
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.video-inline-error {
|
||||
margin-top: 14px;
|
||||
padding: 12px 14px;
|
||||
border: 1px solid color-mix(in srgb, #cf222e 35%, var(--github-border) 65%);
|
||||
border-radius: 10px;
|
||||
background: color-mix(in srgb, #cf222e 8%, var(--github-bg) 92%);
|
||||
color: #cf222e;
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.video-edit-progress-shell {
|
||||
margin-top: 16px;
|
||||
}
|
||||
|
||||
.video-edit-progress-shell strong {
|
||||
display: inline-block;
|
||||
margin-top: 8px;
|
||||
}
|
||||
|
||||
.video-edit-actions {
|
||||
margin-top: 18px;
|
||||
}
|
||||
|
||||
.primary-btn {
|
||||
border-color: #0969da;
|
||||
background: #0969da;
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.primary-btn:hover:enabled {
|
||||
background: #0859ba;
|
||||
}
|
||||
|
||||
.video-state-card {
|
||||
width: min(520px, 100%);
|
||||
padding: 24px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 16px;
|
||||
background: color-mix(in srgb, var(--github-bg) 92%, white 8%);
|
||||
box-shadow: 0 18px 40px rgba(15, 23, 42, 0.08);
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.video-state-card h3 {
|
||||
margin: 0 0 10px;
|
||||
}
|
||||
|
||||
.video-state-card p {
|
||||
margin: 0;
|
||||
color: var(--github-text-secondary);
|
||||
line-height: 1.7;
|
||||
}
|
||||
|
||||
.video-state-card strong {
|
||||
display: block;
|
||||
margin-top: 12px;
|
||||
color: var(--github-text);
|
||||
}
|
||||
|
||||
.video-state-error {
|
||||
border-color: color-mix(in srgb, #cf222e 40%, var(--github-border) 60%);
|
||||
}
|
||||
|
||||
.video-progress-track {
|
||||
position: relative;
|
||||
height: 10px;
|
||||
margin-top: 18px;
|
||||
overflow: hidden;
|
||||
border-radius: 999px;
|
||||
background: var(--github-hover);
|
||||
}
|
||||
|
||||
.video-progress-bar {
|
||||
display: block;
|
||||
height: 100%;
|
||||
border-radius: inherit;
|
||||
background: linear-gradient(90deg, #0969da, #2da44e);
|
||||
transition: width 180ms ease;
|
||||
}
|
||||
|
||||
.pdf-frame {
|
||||
width: 100%;
|
||||
min-height: 78vh;
|
||||
@@ -694,6 +1359,7 @@ function downloadFile() {
|
||||
.icon-sql::before { content: 'SQL'; font-size: 8px; font-weight: 700; color: #8250df; }
|
||||
.icon-image::before { content: '🖼'; font-size: 13px; }
|
||||
.icon-pdf::before { content: 'PDF'; font-size: 8px; font-weight: 700; color: #cf222e; }
|
||||
.icon-video::before { content: '▶'; font-size: 11px; font-weight: 700; color: #0969da; }
|
||||
.icon-word::before { content: 'DOC'; font-size: 7px; font-weight: 700; color: #0969da; }
|
||||
.icon-ppt::before { content: 'PPT'; font-size: 7px; font-weight: 700; color: #bc4c00; }
|
||||
.icon-excel::before { content: 'XLS'; font-size: 7px; font-weight: 700; color: #1f883d; }
|
||||
@@ -701,13 +1367,21 @@ function downloadFile() {
|
||||
.icon-text::before { content: 'TXT'; font-size: 8px; font-weight: 700; color: #6e7781; }
|
||||
|
||||
@media (max-width: 960px) {
|
||||
.file-path-bar,
|
||||
.file-meta-row,
|
||||
.file-header-row,
|
||||
.directory-card-header {
|
||||
flex-direction: column;
|
||||
align-items: flex-start;
|
||||
}
|
||||
|
||||
.file-header-main,
|
||||
.file-actions {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.file-actions {
|
||||
margin-left: 0;
|
||||
}
|
||||
|
||||
.directory-row {
|
||||
grid-template-columns: minmax(0, 1fr);
|
||||
padding-top: 10px;
|
||||
|
||||
@@ -354,21 +354,18 @@ function forwardDragOver(event, id) {
|
||||
|
||||
.branch-actions {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
gap: 4px;
|
||||
padding: 2px;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 10px;
|
||||
background: var(--github-bg);
|
||||
}
|
||||
|
||||
.header-icon-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
padding: 0;
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 8px;
|
||||
background: var(--github-bg);
|
||||
color: var(--github-text-secondary);
|
||||
cursor: pointer;
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
border: 1px solid transparent;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.header-icon-btn:hover {
|
||||
@@ -513,21 +510,11 @@ function forwardDragOver(event, id) {
|
||||
|
||||
.chevron,
|
||||
.chevron-placeholder {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.chevron {
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--github-text-secondary);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.icon-folder,
|
||||
.icon-file {
|
||||
display: inline-flex;
|
||||
@@ -592,7 +579,8 @@ function forwardDragOver(event, id) {
|
||||
|
||||
.node-actions {
|
||||
display: flex;
|
||||
gap: 2px;
|
||||
gap: 4px;
|
||||
margin-left: 4px;
|
||||
opacity: 0;
|
||||
transition: opacity 0.15s ease;
|
||||
}
|
||||
@@ -602,25 +590,56 @@ function forwardDragOver(event, id) {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.action-btn {
|
||||
.header-icon-btn,
|
||||
.action-btn,
|
||||
.chevron {
|
||||
appearance: none;
|
||||
-webkit-appearance: none;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 22px;
|
||||
height: 22px;
|
||||
padding: 0;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
background: transparent;
|
||||
color: var(--github-text-secondary);
|
||||
cursor: pointer;
|
||||
flex-shrink: 0;
|
||||
line-height: 0;
|
||||
}
|
||||
|
||||
.action-btn:hover {
|
||||
.chevron {
|
||||
margin-left: -2px;
|
||||
border: 1px solid transparent;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.action-btn {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: 1px solid transparent;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.header-icon-btn svg,
|
||||
.action-btn svg,
|
||||
.chevron svg {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.header-icon-btn:hover,
|
||||
.action-btn:hover,
|
||||
.tree-node:hover .chevron,
|
||||
.tree-node.selected .chevron {
|
||||
background: rgba(9, 105, 218, 0.08);
|
||||
color: var(--github-text);
|
||||
}
|
||||
|
||||
.header-icon-btn:focus-visible,
|
||||
.action-btn:focus-visible,
|
||||
.chevron:focus-visible {
|
||||
outline: none;
|
||||
box-shadow: 0 0 0 2px rgba(9, 105, 218, 0.18);
|
||||
}
|
||||
|
||||
.tree-content::-webkit-scrollbar {
|
||||
width: 8px;
|
||||
}
|
||||
|
||||
317
src/components/ImageEditorComponent.vue
Normal file
317
src/components/ImageEditorComponent.vue
Normal file
@@ -0,0 +1,317 @@
|
||||
<script setup>
|
||||
import { onBeforeUnmount, onMounted, ref, watch } from 'vue'
|
||||
import 'tui-color-picker/dist/tui-color-picker.css'
|
||||
import 'tui-image-editor/dist/tui-image-editor.css'
|
||||
|
||||
const props = defineProps({
|
||||
imageUrl: { type: String, required: true },
|
||||
imageName: { type: String, default: 'image' }
|
||||
})
|
||||
|
||||
const emit = defineEmits(['error'])
|
||||
|
||||
const editorHost = ref(null)
|
||||
const isLoading = ref(false)
|
||||
const isReady = ref(false)
|
||||
|
||||
const editorMenus = ['crop', 'flip', 'rotate', 'draw', 'shape', 'icon', 'text', 'filter']
|
||||
const editorTheme = {
|
||||
'common.bi.image': 'data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw==',
|
||||
'common.backgroundColor': '#08111f',
|
||||
'common.border': '0px',
|
||||
'menu.normalLabel.color': '#9fb0c8',
|
||||
'menu.activeLabel.color': '#f8fafc',
|
||||
'submenu.backgroundColor': 'rgba(8, 17, 31, 0.95)',
|
||||
'submenu.partition.color': 'rgba(159, 176, 200, 0.2)',
|
||||
'submenu.normalLabel.color': '#9fb0c8',
|
||||
'submenu.activeLabel.color': '#f8fafc',
|
||||
'checkbox.backgroundColor': '#f8fafc',
|
||||
'range.pointer.color': '#f8fafc',
|
||||
'range.bar.color': 'rgba(159, 176, 200, 0.3)',
|
||||
'range.subbar.color': '#38bdf8',
|
||||
'range.value.color': '#f8fafc',
|
||||
'range.value.backgroundColor': 'rgba(15, 23, 42, 0.88)',
|
||||
'range.value.border': '1px solid rgba(159, 176, 200, 0.22)',
|
||||
'range.title.color': '#cbd5e1',
|
||||
'colorpicker.button.border': '1px solid rgba(159, 176, 200, 0.2)',
|
||||
'colorpicker.title.color': '#e2e8f0'
|
||||
}
|
||||
|
||||
let editorInstance = null
|
||||
let loadRequestId = 0
|
||||
|
||||
function getErrorMessage(error, fallback) {
|
||||
return error instanceof Error && error.message ? error.message : fallback
|
||||
}
|
||||
|
||||
function dataUrlToBlob(dataUrl) {
|
||||
const [header, body = ''] = String(dataUrl || '').split(',')
|
||||
const mimeMatch = header.match(/data:([^;]+);base64/)
|
||||
const mimeType = mimeMatch ? mimeMatch[1] : 'image/png'
|
||||
const binary = atob(body)
|
||||
const bytes = new Uint8Array(binary.length)
|
||||
|
||||
for (let index = 0; index < binary.length; index += 1) {
|
||||
bytes[index] = binary.charCodeAt(index)
|
||||
}
|
||||
|
||||
return new Blob([bytes], { type: mimeType })
|
||||
}
|
||||
|
||||
async function getImageEditorConstructor() {
|
||||
const module = await import('tui-image-editor')
|
||||
return module.default || module
|
||||
}
|
||||
|
||||
async function initializeEditor() {
|
||||
if (!editorHost.value || !props.imageUrl || editorInstance) return
|
||||
|
||||
isLoading.value = true
|
||||
|
||||
try {
|
||||
const ImageEditor = await getImageEditorConstructor()
|
||||
editorInstance = new ImageEditor(editorHost.value, {
|
||||
includeUI: {
|
||||
loadImage: {
|
||||
path: props.imageUrl,
|
||||
name: props.imageName || 'image'
|
||||
},
|
||||
menu: editorMenus,
|
||||
initMenu: 'draw',
|
||||
menuBarPosition: 'bottom',
|
||||
uiSize: {
|
||||
width: '100%',
|
||||
height: '100%'
|
||||
},
|
||||
theme: editorTheme,
|
||||
usageStatistics: false
|
||||
},
|
||||
cssMaxWidth: 1280,
|
||||
cssMaxHeight: 960,
|
||||
selectionStyle: {
|
||||
cornerSize: 16,
|
||||
rotatingPointOffset: 56
|
||||
},
|
||||
usageStatistics: false
|
||||
})
|
||||
|
||||
isReady.value = true
|
||||
} catch (error) {
|
||||
emit('error', getErrorMessage(error, '图片编辑器加载失败'))
|
||||
} finally {
|
||||
isLoading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function reloadImageFromProps() {
|
||||
if (!editorInstance || !props.imageUrl) return
|
||||
|
||||
const currentRequestId = ++loadRequestId
|
||||
isLoading.value = true
|
||||
|
||||
try {
|
||||
await editorInstance.loadImageFromURL(props.imageUrl, props.imageName || 'image')
|
||||
editorInstance.clearUndoStack()
|
||||
editorInstance.clearRedoStack()
|
||||
} catch (error) {
|
||||
if (currentRequestId === loadRequestId) {
|
||||
emit('error', getErrorMessage(error, '重新载入图片失败'))
|
||||
}
|
||||
} finally {
|
||||
if (currentRequestId === loadRequestId) {
|
||||
isLoading.value = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function exportImageBlob(options = {}) {
|
||||
if (!editorInstance) {
|
||||
throw new Error('图片编辑器尚未就绪')
|
||||
}
|
||||
|
||||
editorInstance.discardSelection()
|
||||
const dataUrl = editorInstance.toDataURL({
|
||||
format: options.format || 'png',
|
||||
quality: typeof options.quality === 'number' ? options.quality : 0.92
|
||||
})
|
||||
|
||||
return dataUrlToBlob(dataUrl)
|
||||
}
|
||||
|
||||
function destroyEditor() {
|
||||
loadRequestId += 1
|
||||
isReady.value = false
|
||||
|
||||
if (!editorInstance) return
|
||||
|
||||
editorInstance.destroy()
|
||||
editorInstance = null
|
||||
}
|
||||
|
||||
watch(
|
||||
() => props.imageUrl,
|
||||
(nextUrl, previousUrl) => {
|
||||
if (!editorInstance || !nextUrl || nextUrl === previousUrl) return
|
||||
reloadImageFromProps()
|
||||
}
|
||||
)
|
||||
|
||||
onMounted(() => {
|
||||
initializeEditor()
|
||||
})
|
||||
|
||||
onBeforeUnmount(() => {
|
||||
destroyEditor()
|
||||
})
|
||||
|
||||
defineExpose({
|
||||
exportImageBlob,
|
||||
reloadImageFromProps,
|
||||
isReady
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="image-editor-shell">
|
||||
<div v-if="isLoading && !isReady" class="editor-placeholder">
|
||||
<div class="editor-placeholder-card">
|
||||
<strong>正在装载图片编辑器</strong>
|
||||
<p>首次打开会初始化 TUI Image Editor，时间会略长一些。</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div
|
||||
ref="editorHost"
|
||||
class="image-editor-host tui-image-editor-container bottom"
|
||||
:class="{ 'is-initializing': isLoading && !isReady }"
|
||||
></div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.image-editor-shell {
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
min-height: 640px;
|
||||
height: min(78vh, 920px);
|
||||
border: 1px solid var(--github-border);
|
||||
border-radius: 18px;
|
||||
background:
|
||||
radial-gradient(circle at top, rgba(56, 189, 248, 0.14), transparent 32%),
|
||||
linear-gradient(180deg, #08111f, #0f172a 58%, #121a2c);
|
||||
box-shadow: 0 28px 60px rgba(15, 23, 42, 0.22);
|
||||
}
|
||||
|
||||
.image-editor-host.tui-image-editor-container {
|
||||
height: 100%;
|
||||
min-height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.image-editor-host.is-initializing {
|
||||
visibility: hidden;
|
||||
}
|
||||
|
||||
.editor-placeholder {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
z-index: 1;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 24px;
|
||||
}
|
||||
|
||||
.editor-placeholder-card {
|
||||
width: min(420px, 100%);
|
||||
padding: 22px 24px;
|
||||
border: 1px solid rgba(159, 176, 200, 0.18);
|
||||
border-radius: 18px;
|
||||
background: rgba(8, 17, 31, 0.82);
|
||||
color: #f8fafc;
|
||||
text-align: center;
|
||||
backdrop-filter: blur(14px);
|
||||
}
|
||||
|
||||
.editor-placeholder-card strong {
|
||||
display: block;
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
.editor-placeholder-card p {
|
||||
margin: 10px 0 0;
|
||||
color: rgba(226, 232, 240, 0.84);
|
||||
line-height: 1.7;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-container) {
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-header) {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-main-container) {
|
||||
top: 0;
|
||||
bottom: 78px;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-main) {
|
||||
top: 0;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-wrap) {
|
||||
top: 0;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-controls) {
|
||||
height: 78px;
|
||||
border-top: 1px solid rgba(159, 176, 200, 0.12);
|
||||
background: rgba(8, 17, 31, 0.9);
|
||||
backdrop-filter: blur(14px);
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-menu > .tui-image-editor-item) {
|
||||
margin: 0 6px;
|
||||
padding: 10px 10px 4px;
|
||||
border-radius: 12px;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-menu > .tui-image-editor-item.active) {
|
||||
background: rgba(248, 250, 252, 0.14);
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-submenu) {
|
||||
height: 166px;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-submenu > div) {
|
||||
padding-bottom: 18px;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-canvas-container) {
|
||||
border-radius: 14px;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
@media (max-width: 960px) {
|
||||
.image-editor-shell {
|
||||
min-height: 560px;
|
||||
height: 72vh;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-main-container) {
|
||||
bottom: 88px;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-controls) {
|
||||
height: 88px;
|
||||
overflow-x: auto;
|
||||
}
|
||||
|
||||
.image-editor-shell :deep(.tui-image-editor-menu) {
|
||||
padding: 0 16px;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
@@ -280,8 +280,9 @@ const createExportName = () => {
|
||||
const buildDocxBlob = async (markdown) => {
|
||||
const { Document, Packer, Paragraph, HeadingLevel } = await import('docx')
|
||||
const children = []
|
||||
const normalizedMarkdown = String(markdown || '').replace(/\r\n?/g, '\n')
|
||||
|
||||
for (const line of markdown.split('\n')) {
|
||||
for (const line of normalizedMarkdown.split('\n')) {
|
||||
if (line.startsWith('# ')) {
|
||||
children.push(new Paragraph({ text: line.slice(2), heading: HeadingLevel.HEADING_1 }))
|
||||
continue
|
||||
@@ -457,11 +458,11 @@ const serializeSelectionToMarkdown = (view, from, to) => {
|
||||
const state = view.state
|
||||
const slice = state.doc.slice(from, to)
|
||||
const doc = state.schema.topNodeType.createAndFill(undefined, slice.content)
|
||||
if (!doc) return state.doc.textBetween(from, to, '\n', '\n')
|
||||
if (!doc) return state.doc.textBetween(from, to, '\n\n', '\n')
|
||||
return crepe?.editor?.action((ctx) => {
|
||||
const serializer = ctx.get(serializerCtx)
|
||||
return serializer(doc)
|
||||
}) || state.doc.textBetween(from, to, '\n', '\n')
|
||||
}) || state.doc.textBetween(from, to, '\n\n', '\n')
|
||||
}
|
||||
|
||||
const selectionIncludesDocBlock = (state) => {
|
||||
@@ -515,13 +516,24 @@ const SKIP_TTS_TYPES = new Set([
|
||||
const extractSelectionText = (view, from, to) => {
|
||||
const { doc } = view.state
|
||||
const parts = []
|
||||
const pushBlockSeparator = () => {
|
||||
if (parts.length === 0) return
|
||||
const lastPart = parts[parts.length - 1]
|
||||
if (typeof lastPart === 'string' && lastPart.endsWith('\n')) return
|
||||
parts.push('\n')
|
||||
}
|
||||
|
||||
doc.nodesBetween(from, to, (node, pos) => {
|
||||
if (SKIP_TTS_TYPES.has(node.type.name)) {
|
||||
return false
|
||||
}
|
||||
|
||||
if (node.isTextblock && pos >= from) {
|
||||
pushBlockSeparator()
|
||||
}
|
||||
|
||||
if (node.type.name === DOC_BLOCK_NODE_TYPE) {
|
||||
pushBlockSeparator()
|
||||
if (node.attrs.content) {
|
||||
parts.push(node.attrs.content)
|
||||
}
|
||||
@@ -544,7 +556,7 @@ const extractSelectionText = (view, from, to) => {
|
||||
return true
|
||||
})
|
||||
|
||||
return parts.join('\n').trim()
|
||||
return parts.join('').trim()
|
||||
}
|
||||
|
||||
const showTTSMenu = (event) => {
|
||||
|
||||
@@ -180,3 +180,164 @@ function forwardDragOver(event, id) {
|
||||
/>
|
||||
</template>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.tree-node {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
height: 30px;
|
||||
padding-right: 8px;
|
||||
font-size: 13px;
|
||||
color: var(--github-text);
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.tree-node:hover {
|
||||
background: var(--github-hover);
|
||||
}
|
||||
|
||||
.tree-node.selected {
|
||||
background: var(--github-selected);
|
||||
color: var(--github-text);
|
||||
}
|
||||
|
||||
.tree-node.clipped {
|
||||
opacity: 0.55;
|
||||
}
|
||||
|
||||
.tree-node-new {
|
||||
margin: 2px 8px;
|
||||
border: 1px dashed var(--github-border);
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.chevron,
|
||||
.chevron-placeholder {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.icon-folder,
|
||||
.icon-file {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.icon-folder::before {
|
||||
content: '📁';
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.icon-file::before {
|
||||
content: '📄';
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.icon-markdown::before { content: 'Ⓜ'; font-size: 12px; font-weight: 700; color: #0969da; }
|
||||
.icon-json::before { content: '{ }'; font-size: 8px; font-weight: 700; color: #8250df; }
|
||||
.icon-javascript::before { content: 'JS'; font-size: 9px; font-weight: 700; color: #9a6700; }
|
||||
.icon-typescript::before { content: 'TS'; font-size: 9px; font-weight: 700; color: #0969da; }
|
||||
.icon-css::before { content: 'CSS'; font-size: 7px; font-weight: 700; color: #1f883d; }
|
||||
.icon-html::before { content: 'HTML'; font-size: 6px; font-weight: 700; color: #bc4c00; }
|
||||
.icon-python::before { content: 'PY'; font-size: 9px; font-weight: 700; color: #0969da; }
|
||||
.icon-vue::before { content: 'Vue'; font-size: 8px; font-weight: 700; color: #1f883d; }
|
||||
.icon-yaml::before { content: 'YML'; font-size: 8px; font-weight: 700; color: #0969da; }
|
||||
.icon-xml::before { content: '</>'; font-size: 8px; font-weight: 700; color: #bc4c00; }
|
||||
.icon-csv::before { content: 'CSV'; font-size: 8px; font-weight: 700; color: #1f883d; }
|
||||
.icon-log::before { content: 'LOG'; font-size: 7px; font-weight: 700; color: #6e7781; }
|
||||
.icon-sql::before { content: 'SQL'; font-size: 8px; font-weight: 700; color: #8250df; }
|
||||
.icon-image::before { content: '🖼'; font-size: 13px; }
|
||||
.icon-pdf::before { content: 'PDF'; font-size: 8px; font-weight: 700; color: #cf222e; }
|
||||
.icon-word::before { content: 'DOC'; font-size: 7px; font-weight: 700; color: #0969da; }
|
||||
.icon-ppt::before { content: 'PPT'; font-size: 7px; font-weight: 700; color: #bc4c00; }
|
||||
.icon-excel::before { content: 'XLS'; font-size: 7px; font-weight: 700; color: #1f883d; }
|
||||
.icon-zip::before { content: 'ZIP'; font-size: 7px; font-weight: 700; color: #6f42c1; }
|
||||
.icon-text::before { content: 'TXT'; font-size: 8px; font-weight: 700; color: #6e7781; }
|
||||
|
||||
.node-name {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.rename-input {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
height: 24px;
|
||||
padding: 0 8px;
|
||||
border: 1px solid #0969da;
|
||||
border-radius: 6px;
|
||||
outline: none;
|
||||
background: var(--github-bg);
|
||||
color: var(--github-text);
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.node-actions {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
margin-left: 4px;
|
||||
opacity: 0;
|
||||
transition: opacity 0.15s ease;
|
||||
}
|
||||
|
||||
.tree-node:hover .node-actions,
|
||||
.tree-node.selected .node-actions {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.action-btn,
|
||||
.chevron {
|
||||
appearance: none;
|
||||
-webkit-appearance: none;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 0;
|
||||
border-radius: 6px;
|
||||
color: var(--github-text-secondary);
|
||||
cursor: pointer;
|
||||
flex-shrink: 0;
|
||||
line-height: 0;
|
||||
}
|
||||
|
||||
.chevron {
|
||||
margin-left: -2px;
|
||||
border: 1px solid transparent;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.action-btn {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: 1px solid transparent;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.action-btn svg,
|
||||
.chevron svg {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.action-btn:hover,
|
||||
.tree-node:hover .chevron,
|
||||
.tree-node.selected .chevron {
|
||||
background: rgba(9, 105, 218, 0.08);
|
||||
color: var(--github-text);
|
||||
}
|
||||
|
||||
.action-btn:focus-visible,
|
||||
.chevron:focus-visible {
|
||||
outline: none;
|
||||
box-shadow: 0 0 0 2px rgba(9, 105, 218, 0.18);
|
||||
}
|
||||
</style>
|
||||
|
||||
@@ -98,8 +98,12 @@ function isBinaryExtension(ext) {
|
||||
function inferMimeType(name, fallback = '') {
|
||||
const ext = getExtension(name)
|
||||
const map = {
|
||||
avi: 'video/x-msvideo',
|
||||
md: 'text/markdown',
|
||||
markdown: 'text/markdown',
|
||||
mkv: 'video/x-matroska',
|
||||
mov: 'video/quicktime',
|
||||
mp4: 'video/mp4',
|
||||
txt: 'text/plain',
|
||||
json: 'application/json',
|
||||
js: 'text/javascript',
|
||||
@@ -126,11 +130,17 @@ function inferMimeType(name, fallback = '') {
|
||||
ps1: 'text/plain',
|
||||
jpg: 'image/jpeg',
|
||||
jpeg: 'image/jpeg',
|
||||
flv: 'video/x-flv',
|
||||
m4v: 'video/x-m4v',
|
||||
png: 'image/png',
|
||||
gif: 'image/gif',
|
||||
webp: 'image/webp',
|
||||
svg: 'image/svg+xml',
|
||||
pdf: 'application/pdf'
|
||||
pdf: 'application/pdf',
|
||||
ogg: 'video/ogg',
|
||||
ogv: 'video/ogg',
|
||||
webm: 'video/webm',
|
||||
wmv: 'video/x-ms-wmv'
|
||||
}
|
||||
return fallback || map[ext] || 'application/octet-stream'
|
||||
}
|
||||
@@ -398,6 +408,43 @@ export function useFileSystem() {
|
||||
return true
|
||||
}
|
||||
|
||||
function updateFile(id, nextValue, options = {}) {
|
||||
const file = records.value.find((item) => item.id === id && item.type === 'file')
|
||||
if (!file) return false
|
||||
|
||||
const nextName = options.name || file.name
|
||||
const isBlobValue = nextValue instanceof Blob
|
||||
const nextContent = isBlobValue ? (options.content ?? '') : String(options.content ?? nextValue ?? '')
|
||||
const nextSize = typeof options.size === 'number'
|
||||
? options.size
|
||||
: isBlobValue
|
||||
? nextValue.size
|
||||
: new Blob([nextContent]).size
|
||||
|
||||
if (nextSize > MAX_FILE_SIZE) {
|
||||
error.value = '单个文件不能超过 1GB'
|
||||
return false
|
||||
}
|
||||
|
||||
file.name = nextName
|
||||
file.updatedAt = Date.now()
|
||||
file.mimeType = inferMimeType(nextName, options.mimeType || (isBlobValue ? nextValue.type : file.mimeType))
|
||||
file.size = nextSize
|
||||
file.storageKind = options.storageKind || (isBlobValue ? 'blob' : 'text')
|
||||
file.content = nextContent
|
||||
file.previewText = options.previewText ?? (file.storageKind === 'text' ? nextContent : '')
|
||||
file.isTruncatedPreview = Boolean(options.isTruncatedPreview)
|
||||
file.blob = isBlobValue ? nextValue : null
|
||||
records.value = [...records.value]
|
||||
error.value = null
|
||||
|
||||
persistRecord(file).catch(() => {
|
||||
error.value = '保存文件失败，可能是浏览器存储空间不足'
|
||||
})
|
||||
touchParent(file.parentId)
|
||||
return true
|
||||
}
|
||||
|
||||
function createFolder(parentId, name) {
|
||||
if (records.value.length >= MAX_NODES) {
|
||||
error.value = `目录项数量不能超过 ${MAX_NODES} 个`
|
||||
@@ -594,8 +641,12 @@ export function useFileSystem() {
|
||||
function getFileIcon(name) {
|
||||
const ext = getExtension(name)
|
||||
const iconMap = {
|
||||
avi: 'video',
|
||||
md: 'markdown',
|
||||
markdown: 'markdown',
|
||||
mkv: 'video',
|
||||
mov: 'video',
|
||||
mp4: 'video',
|
||||
txt: 'text',
|
||||
json: 'json',
|
||||
js: 'javascript',
|
||||
@@ -617,13 +668,19 @@ export function useFileSystem() {
|
||||
jpeg: 'image',
|
||||
png: 'image',
|
||||
gif: 'image',
|
||||
flv: 'video',
|
||||
m4v: 'video',
|
||||
webp: 'image',
|
||||
svg: 'image',
|
||||
ogg: 'video',
|
||||
ogv: 'video',
|
||||
pdf: 'pdf',
|
||||
doc: 'word',
|
||||
docx: 'word',
|
||||
ppt: 'ppt',
|
||||
pptx: 'ppt',
|
||||
webm: 'video',
|
||||
wmv: 'video',
|
||||
xls: 'excel',
|
||||
xlsx: 'excel',
|
||||
zip: 'zip'
|
||||
@@ -681,6 +738,7 @@ export function useFileSystem() {
|
||||
stats,
|
||||
load,
|
||||
createFile,
|
||||
updateFile,
|
||||
createFolder,
|
||||
rename,
|
||||
remove,
|
||||
|
||||
@@ -1,42 +1,71 @@
|
||||
# Plugins 模块指南
|
||||
# Plugins 插件层指引
|
||||
|
||||
## OVERVIEW
|
||||
Milkdown/ProseMirror 插件，AI 补全核心逻辑。
|
||||
本文件适用于 src/plugins/ 下的插件代码。
|
||||
|
||||
## STRUCTURE
|
||||
- copilotPlugin.ts - ProseMirror Mark 系统、ghost text、防抖请求
|
||||
- docBlockPlugin.ts - 文档块插件
|
||||
- mermaidPlugin.ts - Mermaid 图表渲染
|
||||
- index.ts - 插件导出
|
||||
- types.ts - 类型定义
|
||||
## 插件层职责
|
||||
|
||||
## WHERE TO LOOK
|
||||
- 承接 Milkdown/ProseMirror 与业务逻辑之间的粘合层。
|
||||
- 负责 AI 补全 ghost text、文档块节点、Mermaid 预览等编辑器级行为。
|
||||
- 这里的代码状态性很强，误改后通常会出现“看起来还能编译，但交互细节坏掉”的问题。
|
||||
|
||||
| 任务 | 文件 | 说明 |
|
||||
|------|------|------|
|
||||
| AI 补全核心 | copilotPlugin.ts | ProseMirror Mark、ghost text |
|
||||
| 文档块处理 | docBlockPlugin.ts | 文档块解析与渲染 |
|
||||
| Mermaid 图表 | mermaidPlugin.ts | 图表渲染集成 |
|
||||
| 插件导出 | index.ts | 统一导出入口 |
|
||||
| 类型定义 | types.ts | 公共类型 |
|
||||
## 文件分工
|
||||
|
||||
## 关键函数
|
||||
- scheduleFetch - 防抖触发补全请求
|
||||
- insertGhostText - 插入 ghost text
|
||||
- acceptSuggestion - Tab 接受建议
|
||||
- rejectSuggestion - Esc 取消建议
|
||||
- copilotPlugin.ts：AI 补全主逻辑，包含 ghost text、请求调度、取消、语言识别、隐藏上下文注入和大小限制。
|
||||
- docBlockPlugin.ts：文档块节点、remark 适配和视图逻辑。
|
||||
- mermaidPlugin.ts：Mermaid 代码块预览和渲染。
|
||||
- index.ts：导出入口。
|
||||
- types.ts：公共类型。
|
||||
|
||||
## CONVENTIONS
|
||||
- TypeScript 2 空格缩进
|
||||
- 函数：camelCase
|
||||
- 接口/类型：PascalCase
|
||||
## copilotPlugin 的真实职责
|
||||
|
||||
## ANTI-PATTERNS
|
||||
- 空 catch 块
|
||||
- 类型错误使用 as any
|
||||
- 硬编码超时值
|
||||
- 维护 ghost suggestion 的 plugin state。
|
||||
- 用 mark + decoration 表示 ghost text，而不是简单的纯字符串缓存。
|
||||
- 为每个 EditorView 维护 runtime：
|
||||
- enabled
|
||||
- debounceTimer
|
||||
- abortController
|
||||
- ctx
|
||||
- requestSeq
|
||||
- docVersion
|
||||
- 推断当前光标语言，处理 fenced code、latex、mermaid 等上下文。
|
||||
- 提取 OCR 缓存和文档块内容，拼到隐藏上下文里。
|
||||
- 控制请求失效和中止，避免旧请求把新文档状态覆盖掉。
|
||||
- 在结果插入后恢复合理的光标位置和选择状态。
|
||||
|
||||
## 注意事项
|
||||
- 防抖时间：1000ms（可配置）
|
||||
- 文档大小限制：32KB 自动禁用
|
||||
- Tab/Esc 快捷键交互
|
||||
## 先看哪些函数
|
||||
|
||||
- getCursorLanguageId：判断当前上下文语言
|
||||
- extractDocBlocksFromMarkdown：从文档块生成隐藏上下文
|
||||
- doFetchSuggestion：真正发起补全请求
|
||||
- clearRuntimeRequests：中止定时器和请求
|
||||
- insertGhostText / insertPlainText：把建议插回编辑器
|
||||
|
||||
## 编辑规则
|
||||
|
||||
- 不要把隐藏 OCR 上下文或文档块上下文直接变成最终可见文本，除非需求明确要求。
|
||||
- 不要破坏 requestSeq 和 AbortController 这套旧请求失效机制。
|
||||
- 不要随意改 ghost text 的 mark/decorations 表示方式；这会影响显示、接受、拒绝和同步。
|
||||
- parserCtx 和 serializerCtx 是和 Milkdown 集成的关键上下文，改解析插入逻辑时要一起考虑。
|
||||
- 插件行为并不完全自洽，很多配置和启停逻辑是在 components/MilkdownEditor.vue 里接上的；不要只在插件内部脑补整体行为。
|
||||
|
||||
## 容易踩坑的点
|
||||
|
||||
- 大小限制既有插件常量，也有编辑器 UI 层联动。
|
||||
- 图片节点类型不止一种名字，代码已经兼容 image、image-block、imageBlock。
|
||||
- 上下文语言识别会影响 Prompt 行为，改错以后通常不是“报错”，而是补全质量或补全边界变差。
|
||||
- Mermaid、LaTeX、文档块这几类上下文都和“是否允许补全、是否允许上传”相关。
|
||||
- 这是状态机式代码；局部改动后要优先验证交互，而不是只看类型通过。
|
||||
|
||||
## 调试顺序
|
||||
|
||||
- 如果是“为什么没有发请求”，先查 enabled、size limit、cursor context、debounce 和 abort。
|
||||
- 如果是“为什么结果插错位置”，先查 ghost range、mapping、selection 恢复。
|
||||
- 如果是“为什么上下文不对”，先查 OCR 缓存、文档块摘录和语言识别。
|
||||
- 如果是“为什么改了插件没生效”，再回看 MilkdownEditor.vue 里的 use/config/watch 是否也要一起改。
|
||||
|
||||
## 改动建议
|
||||
|
||||
- 先做最小改动。
|
||||
- 先验证单个交互问题，不要一口气同时改 ghost 插入、语言识别和请求时机。
|
||||
- 当改动影响请求 payload 时，再考虑是否需要同步后端 Prompt 测试。
|
||||
- 只有在确认插件本身是控制点时，才在这里改；很多行为实际上是在上层组件里决定的。
|
||||
|
||||
@@ -4,6 +4,7 @@ import { parserCtx, serializerCtx } from '@milkdown/kit/core'
|
||||
import { Node as ProseNode, Slice } from '@milkdown/prose/model'
|
||||
import type { Ctx } from '@milkdown/kit/core'
|
||||
import { Decoration, DecorationSet, type EditorView } from '@milkdown/prose/view'
|
||||
import { extractDocBlockContextFromMarkdown } from '../utils/docBlock.js'
|
||||
import { getOcrCache, OCR_SIZE_LIMIT, extractTextFromOCR } from '../utils/ocrCache'
|
||||
|
||||
const COPILOT_PLUGIN_KEY = new PluginKey('milkdown-copilot')
|
||||
@@ -11,6 +12,8 @@ const DEBOUNCE_MS = 1000
|
||||
const SIZE_LIMIT = OCR_SIZE_LIMIT
|
||||
const DOC_SIZE_LIMIT = 32 * 1024 // 文档块32KB限制
|
||||
const IMAGE_NODE_TYPES = new Set(['image', 'image-block', 'imageBlock'])
|
||||
const FALLBACK_BLOCK_SEPARATOR = '\n\n'
|
||||
const FALLBACK_LEAF_TEXT = '\n'
|
||||
|
||||
interface CopilotState {
|
||||
from: number
|
||||
@@ -181,10 +184,6 @@ function normalizeSuggestionText(raw: string): string {
|
||||
}
|
||||
}
|
||||
|
||||
if (!text.includes('\n') && text.includes('\\n')) {
|
||||
text = text.replace(/\\n/g, '\n')
|
||||
}
|
||||
|
||||
return text
|
||||
}
|
||||
|
||||
@@ -307,7 +306,7 @@ function serializeRangeToMarkdown(
|
||||
const slice = doc.slice(from, to)
|
||||
if (slice.content.size <= 0) return ''
|
||||
const sliceDoc = schema.topNodeType.createAndFill(undefined, slice.content)
|
||||
return sliceDoc ? serializer(sliceDoc) : doc.textBetween(from, to, '\n', '\n')
|
||||
return sliceDoc ? serializer(sliceDoc) : doc.textBetween(from, to, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
|
||||
}
|
||||
|
||||
function buildOcrContextForRequest(doc: ProseNode, cursorPos: number): string {
|
||||
@@ -328,32 +327,7 @@ function buildOcrContextForRequest(doc: ProseNode, cursorPos: number): string {
|
||||
})
|
||||
|
||||
if (lines.length === 0) return ''
|
||||
return `\n\n${lines.join('\n')}`
|
||||
}
|
||||
|
||||
// 从markdown中提取文档块内容用于AI补全上下文
|
||||
function extractDocBlocksFromMarkdown(markdown: string): string {
|
||||
const lines: string[] = []
|
||||
|
||||
// 使用正则表达式匹配文档块
|
||||
// <doc_type="pdf" doc_name="xxx" upload_time="xxx">content</doc_end>
|
||||
const docBlockRegex = /<doc_type="(\w+)"\s+doc_name="([^"]+)"\s+upload_time="([^"]+)">([\s\S]*?)<\/doc_end>/g
|
||||
|
||||
let match
|
||||
while ((match = docBlockRegex.exec(markdown)) !== null) {
|
||||
const docType = match[1]
|
||||
const docName = match[2]
|
||||
const content = match[4].trim()
|
||||
|
||||
if (content) {
|
||||
// 将文档内容格式化为上下文，限制长度
|
||||
const truncatedContent = content.length > 500 ? content.substring(0, 500) + '...' : content
|
||||
lines.push(`<doc_type="${docType}" doc_name="${docName}">\n${truncatedContent}\n</doc_end>`)
|
||||
}
|
||||
}
|
||||
|
||||
if (lines.length === 0) return ''
|
||||
return `\n\n-- 已上传文档内容 --\n${lines.join('\n\n')}`
|
||||
return lines.join('\n')
|
||||
}
|
||||
|
||||
function doFetchSuggestion(
|
||||
@@ -413,26 +387,26 @@ function scheduleFetch(view: EditorView, runtime: CopilotRuntime, pos: number) {
|
||||
try {
|
||||
prefixMarkdown = serializeRangeToMarkdown(doc, 0, pos, schema, serializer)
|
||||
if (!prefixMarkdown) {
|
||||
prefixMarkdown = doc.textBetween(0, pos, '\n', '\n')
|
||||
prefixMarkdown = doc.textBetween(0, pos, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
|
||||
}
|
||||
|
||||
suffixMarkdown = serializeRangeToMarkdown(doc, pos, doc.content.size, schema, serializer)
|
||||
if (!suffixMarkdown) {
|
||||
suffixMarkdown = doc.textBetween(pos, doc.content.size, '\n', '\n')
|
||||
suffixMarkdown = doc.textBetween(pos, doc.content.size, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
|
||||
}
|
||||
} catch {
|
||||
prefixMarkdown = doc.textBetween(0, pos, '\n', '\n')
|
||||
suffixMarkdown = doc.textBetween(pos, doc.content.size, '\n', '\n')
|
||||
prefixMarkdown = doc.textBetween(0, pos, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
|
||||
suffixMarkdown = doc.textBetween(pos, doc.content.size, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
|
||||
}
|
||||
|
||||
// 构建上下文：OCR内容 + 上传文档内容
|
||||
const ocrContext = buildOcrContextForRequest(doc, pos)
|
||||
|
||||
// 从markdown中提取文档块内容用于AI补全上下文
|
||||
const docContext = extractDocBlocksFromMarkdown(prefixMarkdown + suffixMarkdown)
|
||||
const docContext = extractDocBlockContextFromMarkdown(prefixMarkdown + suffixMarkdown, 500)
|
||||
|
||||
// 组合所有上下文到prefix前面
|
||||
const fullPrefixWithContext = `${ocrContext}${docContext}\n\n${prefixMarkdown}`
|
||||
const fullPrefixWithContext = [ocrContext, docContext, prefixMarkdown].filter(Boolean).join('\n\n')
|
||||
|
||||
const totalTextLen = (prefixMarkdown + suffixMarkdown).length
|
||||
const contextLen = fullPrefixWithContext.length - prefixMarkdown.length
|
||||
|
||||
@@ -16,6 +16,9 @@ import {
|
||||
stripDocBlockMarkdown,
|
||||
} from '../utils/docBlock.js'
|
||||
|
||||
const FALLBACK_BLOCK_SEPARATOR = '\n\n'
|
||||
const FALLBACK_LEAF_TEXT = '\n'
|
||||
|
||||
function serializeRangeToMarkdown(
|
||||
doc: ProseNode,
|
||||
from: number,
|
||||
@@ -27,7 +30,7 @@ function serializeRangeToMarkdown(
|
||||
const slice = doc.slice(from, to)
|
||||
if (slice.content.size <= 0) return ''
|
||||
const sliceDoc = schema.topNodeType.createAndFill(undefined, slice.content)
|
||||
return sliceDoc ? serializer(sliceDoc) : doc.textBetween(from, to, '\n', '\n')
|
||||
return sliceDoc ? serializer(sliceDoc) : doc.textBetween(from, to, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
|
||||
}
|
||||
|
||||
function buildDocContext(doc: ProseNode, excludePos?: number) {
|
||||
|
||||
@@ -5,6 +5,17 @@ export const DOC_CONTEXT_LIMIT = 32 * 1024
|
||||
const IMAGE_MD_RE = /!\[[^\]]*]\([^)]+\)/g
|
||||
const IMAGE_HTML_RE = /<img\b[^>]*>/gi
|
||||
const HEADER_SEPARATOR = '\n---\n'
|
||||
const FENCED_DOC_BLOCK_RE = /(^|\n)(`{3,})llm-file[^\n]*\n([\s\S]*?)\n\2(?=\n|$)/g
|
||||
const LEGACY_DOC_BLOCK_RE = /<doc_type="[^"]+"\s+doc_name="[^"]+"\s+upload_time="[^"]+"(?:\s+collapsed="[^"]+")?>[\s\S]*?<\/doc_end>/g
|
||||
|
||||
function normalizeMarkdownText(markdown = '') {
|
||||
return String(markdown || '').replace(/\r\n?/g, '\n')
|
||||
}
|
||||
|
||||
function clipDocContext(content = '', limit = 0) {
|
||||
if (!limit || content.length <= limit) return content
|
||||
return `${content.slice(0, limit)}...`
|
||||
}
|
||||
|
||||
export function normalizeDocType(value = '') {
|
||||
const lower = String(value || '').trim().toLowerCase()
|
||||
@@ -54,12 +65,9 @@ export function isSupportedDocFile(file) {
|
||||
}
|
||||
|
||||
export function sanitizeDocContent(markdown = '') {
|
||||
return String(markdown || '')
|
||||
.replace(/\r\n?/g, '\n')
|
||||
return normalizeMarkdownText(markdown)
|
||||
.replace(IMAGE_MD_RE, '')
|
||||
.replace(IMAGE_HTML_RE, '')
|
||||
.replace(/\n{3,}/g, '\n\n')
|
||||
.trim()
|
||||
}
|
||||
|
||||
function quoteMeta(value = '') {
|
||||
@@ -102,7 +110,7 @@ export function buildDocBlockValue(attrs = {}) {
|
||||
}
|
||||
|
||||
export function parseDocBlockValue(raw = '') {
|
||||
const normalized = String(raw || '').replace(/\r\n?/g, '\n')
|
||||
const normalized = normalizeMarkdownText(raw)
|
||||
const separatorIndex = normalized.indexOf(HEADER_SEPARATOR)
|
||||
const headerText = separatorIndex >= 0 ? normalized.slice(0, separatorIndex) : ''
|
||||
const bodyText = separatorIndex >= 0 ? normalized.slice(separatorIndex + HEADER_SEPARATOR.length) : normalized
|
||||
@@ -150,7 +158,8 @@ export function buildLegacyDocBlock(attrs = {}) {
|
||||
}
|
||||
|
||||
export function parseLegacyDocBlock(raw = '') {
|
||||
const match = String(raw || '').match(/^<doc_type="([^"]+)"\s+doc_name="([^"]+)"\s+upload_time="([^"]+)"(?:\s+collapsed="([^"]+)")?>\n?([\s\S]*?)\n?<\/doc_end>$/)
|
||||
const normalized = normalizeMarkdownText(raw)
|
||||
const match = normalized.match(/^<doc_type="([^"]+)"\s+doc_name="([^"]+)"\s+upload_time="([^"]+)"(?:\s+collapsed="([^"]+)")?>\n?([\s\S]*?)\n?<\/doc_end>$/)
|
||||
if (!match) return null
|
||||
return {
|
||||
docType: normalizeDocType(match[1]),
|
||||
@@ -161,33 +170,57 @@ export function parseLegacyDocBlock(raw = '') {
|
||||
}
|
||||
}
|
||||
|
||||
export function extractDocBlockContextFromMarkdown(markdown = '', contentLimit = 0) {
|
||||
const normalized = normalizeMarkdownText(markdown)
|
||||
const contexts = []
|
||||
const appendContext = (attrs) => {
|
||||
if (!attrs) return
|
||||
const rawContent = String(attrs.content || '')
|
||||
if (!rawContent.trim()) return
|
||||
contexts.push(buildDocContextFence({
|
||||
docType: attrs.docType,
|
||||
content: clipDocContext(rawContent, contentLimit),
|
||||
}))
|
||||
}
|
||||
|
||||
normalized.replace(FENCED_DOC_BLOCK_RE, (_full, _prefix, _fence, value) => {
|
||||
appendContext(parseDocBlockValue(value))
|
||||
return _full
|
||||
})
|
||||
|
||||
normalized.replace(LEGACY_DOC_BLOCK_RE, (full) => {
|
||||
appendContext(parseLegacyDocBlock(full))
|
||||
return full
|
||||
})
|
||||
|
||||
return contexts.join('\n\n')
|
||||
}
|
||||
|
||||
export function transformDocBlockMarkdownForClipboard(markdown = '') {
|
||||
const pattern = /(^|\n)(`{3,})llm-file[^\n]*\n([\s\S]*?)\n\2(?=\n|$)/g
|
||||
const replacedFence = String(markdown || '').replace(pattern, (full, prefix, _fence, value) => {
|
||||
const normalized = normalizeMarkdownText(markdown)
|
||||
const replacedFence = normalized.replace(FENCED_DOC_BLOCK_RE, (full, prefix, _fence, value) => {
|
||||
const attrs = parseDocBlockValue(value)
|
||||
return `${prefix}${buildDocContextFence(attrs)}`
|
||||
})
|
||||
return replacedFence.replace(/<doc_type="[^"]+"\s+doc_name="[^"]+"\s+upload_time="[^"]+"(?:\s+collapsed="[^"]+")?>[\s\S]*?<\/doc_end>/g, (full) => {
|
||||
return replacedFence.replace(LEGACY_DOC_BLOCK_RE, (full) => {
|
||||
const attrs = parseLegacyDocBlock(full)
|
||||
return attrs ? buildDocContextFence(attrs) : full
|
||||
})
|
||||
}
|
||||
|
||||
export function stripDocBlockMarkdown(markdown = '') {
|
||||
const pattern = /(^|\n)(`{3,})llm-file[^\n]*\n[\s\S]*?\n\2(?=\n|$)/g
|
||||
return String(markdown || '').replace(pattern, '$1').replace(/\n{3,}/g, '\n\n').trim()
|
||||
return normalizeMarkdownText(markdown).replace(FENCED_DOC_BLOCK_RE, '$1')
|
||||
}
|
||||
|
||||
export function transformLegacyDocBlocksForExport(markdown = '') {
|
||||
return String(markdown || '').replace(/<doc_type="[^"]+"\s+doc_name="[^"]+"\s+upload_time="[^"]+"(?:\s+collapsed="[^"]+")?>[\s\S]*?<\/doc_end>/g, (full) => {
|
||||
return normalizeMarkdownText(markdown).replace(LEGACY_DOC_BLOCK_RE, (full) => {
|
||||
const attrs = parseLegacyDocBlock(full)
|
||||
return attrs ? buildDocBlockMarkdown(attrs) : full
|
||||
})
|
||||
}
|
||||
|
||||
export function transformSpecialDocBlocksToLegacy(markdown = '') {
|
||||
const pattern = /(^|\n)(`{3,})llm-file[^\n]*\n([\s\S]*?)\n\2(?=\n|$)/g
|
||||
return String(markdown || '').replace(pattern, (full, prefix, _fence, value) => {
|
||||
return normalizeMarkdownText(markdown).replace(FENCED_DOC_BLOCK_RE, (full, prefix, _fence, value) => {
|
||||
const attrs = parseDocBlockValue(value)
|
||||
return `${prefix}${buildLegacyDocBlock(attrs)}`
|
||||
})
|
||||
|
||||
230
src/utils/ffmpeg.js
Normal file
230
src/utils/ffmpeg.js
Normal file
@@ -0,0 +1,230 @@
|
||||
import coreURL from '@ffmpeg/core?url'
|
||||
import wasmURL from '@ffmpeg/core/wasm?url'
|
||||
import classWorkerURL from '@ffmpeg/ffmpeg/worker?url'
|
||||
|
||||
const VIDEO_EXTENSIONS = new Set([
|
||||
'mp4',
|
||||
'webm',
|
||||
'ogv',
|
||||
'ogg',
|
||||
'mov',
|
||||
'm4v',
|
||||
'avi',
|
||||
'mkv',
|
||||
'flv',
|
||||
'wmv',
|
||||
'3gp',
|
||||
'm2ts',
|
||||
'mts',
|
||||
'ts'
|
||||
])
|
||||
|
||||
const FORCED_TRANSCODE_EXTENSIONS = new Set([
|
||||
'3gp',
|
||||
'avi',
|
||||
'flv',
|
||||
'm2ts',
|
||||
'mkv',
|
||||
'mts',
|
||||
'ts',
|
||||
'wmv'
|
||||
])
|
||||
|
||||
const VIDEO_MIME_BY_EXTENSION = {
|
||||
m4v: 'video/x-m4v',
|
||||
mov: 'video/quicktime',
|
||||
mp4: 'video/mp4',
|
||||
ogg: 'video/ogg',
|
||||
ogv: 'video/ogg',
|
||||
webm: 'video/webm'
|
||||
}
|
||||
|
||||
let ffmpegInstancePromise = null
|
||||
let progressHandler = null
|
||||
let transcodeQueue = Promise.resolve()
|
||||
|
||||
function getFileExtension(name = '') {
|
||||
const parts = String(name).split('.')
|
||||
return parts.length > 1 ? parts.pop().toLowerCase() : ''
|
||||
}
|
||||
|
||||
function getNodeMimeType(node) {
|
||||
return String(node?.mimeType || '').toLowerCase()
|
||||
}
|
||||
|
||||
function formatFfmpegTime(seconds = 0) {
|
||||
const totalSeconds = Math.max(0, Number(seconds) || 0)
|
||||
return totalSeconds.toFixed(3)
|
||||
}
|
||||
|
||||
function createVideoProbe() {
|
||||
if (typeof document === 'undefined') return null
|
||||
return document.createElement('video')
|
||||
}
|
||||
|
||||
function toAbsoluteURL(url) {
|
||||
if (!url || typeof location === 'undefined') {
|
||||
return url
|
||||
}
|
||||
|
||||
return new URL(url, location.href).href
|
||||
}
|
||||
|
||||
function getLoadConfigs() {
|
||||
return [{
|
||||
classWorkerURL: toAbsoluteURL(classWorkerURL),
|
||||
coreURL: toAbsoluteURL(coreURL),
|
||||
wasmURL: toAbsoluteURL(wasmURL)
|
||||
}]
|
||||
}
|
||||
|
||||
async function loadFFmpegRuntime(FFmpeg, fetchFile) {
|
||||
let lastError = null
|
||||
|
||||
for (const loadConfig of getLoadConfigs()) {
|
||||
const ffmpeg = new FFmpeg()
|
||||
ffmpeg.on('progress', ({ progress }) => {
|
||||
if (typeof progressHandler === 'function') {
|
||||
progressHandler(Math.max(0, Math.min(1, progress || 0)))
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
await ffmpeg.load(loadConfig)
|
||||
return { ffmpeg, fetchFile }
|
||||
} catch (error) {
|
||||
lastError = error
|
||||
}
|
||||
}
|
||||
|
||||
throw lastError || new Error('FFmpeg 初始化失败')
|
||||
}
|
||||
|
||||
async function getFFmpegInstance() {
|
||||
if (!ffmpegInstancePromise) {
|
||||
ffmpegInstancePromise = Promise.all([
|
||||
import('@ffmpeg/ffmpeg'),
|
||||
import('@ffmpeg/util')
|
||||
]).then(([{ FFmpeg }, { fetchFile }]) => loadFFmpegRuntime(FFmpeg, fetchFile))
|
||||
}
|
||||
|
||||
return ffmpegInstancePromise
|
||||
}
|
||||
|
||||
function buildMp4Command(inputName, outputName, options = {}) {
|
||||
const startTime = Math.max(0, Number(options.startTime) || 0)
|
||||
const endTime = Number.isFinite(options.endTime) ? Number(options.endTime) : null
|
||||
const duration = endTime === null ? null : Math.max(0.05, endTime - startTime)
|
||||
const args = []
|
||||
|
||||
if (startTime > 0) {
|
||||
args.push('-ss', formatFfmpegTime(startTime))
|
||||
}
|
||||
|
||||
args.push('-i', inputName)
|
||||
|
||||
if (duration !== null) {
|
||||
args.push('-t', formatFfmpegTime(duration))
|
||||
}
|
||||
|
||||
if (options.preferEncoding !== false) {
|
||||
args.push('-c:v', 'libx264', '-preset', 'ultrafast', '-pix_fmt', 'yuv420p')
|
||||
if (options.muteAudio) {
|
||||
args.push('-an')
|
||||
} else {
|
||||
args.push('-c:a', 'aac')
|
||||
}
|
||||
} else if (options.muteAudio) {
|
||||
args.push('-an')
|
||||
}
|
||||
|
||||
args.push('-movflags', '+faststart', '-y', outputName)
|
||||
return args
|
||||
}
|
||||
|
||||
async function executePreferredMp4Command(ffmpeg, inputName, outputName, options = {}) {
|
||||
try {
|
||||
await ffmpeg.exec(buildMp4Command(inputName, outputName, {
|
||||
...options,
|
||||
preferEncoding: true
|
||||
}))
|
||||
} catch {
|
||||
await ffmpeg.exec(buildMp4Command(inputName, outputName, {
|
||||
...options,
|
||||
preferEncoding: false
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
async function runMp4Job(blob, fileName, options = {}) {
|
||||
const { ffmpeg, fetchFile } = await getFFmpegInstance()
|
||||
const ext = getFileExtension(fileName) || 'bin'
|
||||
const inputName = `input-${Date.now()}.${ext}`
|
||||
const outputName = `output-${Date.now()}.mp4`
|
||||
progressHandler = typeof options.onProgress === 'function' ? options.onProgress : null
|
||||
|
||||
try {
|
||||
await ffmpeg.writeFile(inputName, await fetchFile(blob))
|
||||
await executePreferredMp4Command(ffmpeg, inputName, outputName, options)
|
||||
const data = await ffmpeg.readFile(outputName)
|
||||
return new Blob([data.buffer], { type: 'video/mp4' })
|
||||
} finally {
|
||||
progressHandler = null
|
||||
await ffmpeg.deleteFile?.(inputName)
|
||||
await ffmpeg.deleteFile?.(outputName)
|
||||
}
|
||||
}
|
||||
|
||||
export function isVideoFile(node) {
|
||||
const ext = getFileExtension(node?.name)
|
||||
const mime = getNodeMimeType(node)
|
||||
return mime.startsWith('video/') || VIDEO_EXTENSIONS.has(ext)
|
||||
}
|
||||
|
||||
export function canPlayVideoNatively(node) {
|
||||
if (!isVideoFile(node)) return false
|
||||
|
||||
const ext = getFileExtension(node?.name)
|
||||
if (FORCED_TRANSCODE_EXTENSIONS.has(ext)) return false
|
||||
|
||||
const mime = getNodeMimeType(node) || VIDEO_MIME_BY_EXTENSION[ext] || ''
|
||||
if (!mime) return ['mp4', 'ogv', 'ogg', 'webm'].includes(ext)
|
||||
|
||||
const video = createVideoProbe()
|
||||
if (!video) return ['mp4', 'ogv', 'ogg', 'webm'].includes(ext)
|
||||
return video.canPlayType(mime) !== ''
|
||||
}
|
||||
|
||||
export async function transcodeVideoToMp4(blob, fileName, options = {}) {
|
||||
const run = async () => {
|
||||
try {
|
||||
return await runMp4Job(blob, fileName, options)
|
||||
} catch (error) {
|
||||
const message = error instanceof Error && error.message
|
||||
? error.message
|
||||
: '浏览器内视频转换失败'
|
||||
throw new Error(message)
|
||||
}
|
||||
}
|
||||
|
||||
const result = transcodeQueue.then(run, run)
|
||||
transcodeQueue = result.then(() => undefined, () => undefined)
|
||||
return result
|
||||
}
|
||||
|
||||
export async function editVideoBlob(blob, fileName, options = {}) {
|
||||
const run = async () => {
|
||||
try {
|
||||
return await runMp4Job(blob, fileName, options)
|
||||
} catch (error) {
|
||||
const message = error instanceof Error && error.message
|
||||
? error.message
|
||||
: '浏览器内视频编辑失败'
|
||||
throw new Error(message)
|
||||
}
|
||||
}
|
||||
|
||||
const result = transcodeQueue.then(run, run)
|
||||
transcodeQueue = result.then(() => undefined, () => undefined)
|
||||
return result
|
||||
}
|
||||
@@ -120,7 +120,7 @@ function closeConfirm() {
|
||||
</aside>
|
||||
|
||||
<section class="docs-main">
|
||||
<header class="docs-toolbar">
|
||||
<header v-if="selectedNode?.type !== 'file'" class="docs-toolbar">
|
||||
<div class="toolbar-left">
|
||||
<button
|
||||
class="sidebar-toggle"
|
||||
@@ -166,7 +166,10 @@ function closeConfirm() {
|
||||
:root-nodes="fs.tree.value"
|
||||
:get-file-icon="fs.getFileIcon"
|
||||
:get-file-blob="fs.getFileBlob"
|
||||
:update-file="fs.updateFile"
|
||||
:show-sidebar-toggle="selectedNode?.type === 'file'"
|
||||
@navigate="fs.select"
|
||||
@toggle-sidebar="sidebarCollapsed = !sidebarCollapsed"
|
||||
/>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
import asyncio
|
||||
from backend.tts_asr import _tts_sync_with_retry, _load_asr_pipeline_with_retry, _asr_pipeline
|
||||
import base64
|
||||
|
||||
async def main():
|
||||
text = "早上好"
|
||||
print(f"Testing TTS with text: {text}")
|
||||
audio_bytes, sr = await _tts_sync_with_retry(text, rate=1.0)
|
||||
print(f"TTS generated {len(audio_bytes)} bytes of audio.")
|
||||
|
||||
print("Testing ASR...")
|
||||
await _load_asr_pipeline_with_retry()
|
||||
asr = _asr_pipeline
|
||||
|
||||
# Needs to process audio_bytes. ASR expects float32 numpy array or bytes?
|
||||
# the pipeline takes bytes or dict with raw array
|
||||
result = asr(audio_bytes)
|
||||
print("ASR output:", result)
|
||||
|
||||
asyncio.run(main())
|
||||
@@ -1,11 +1,17 @@
|
||||
import { defineConfig } from 'vite'
|
||||
import vue from '@vitejs/plugin-vue'
|
||||
|
||||
const crossOriginIsolationHeaders = {
|
||||
'Cross-Origin-Embedder-Policy': 'require-corp',
|
||||
'Cross-Origin-Opener-Policy': 'same-origin'
|
||||
}
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [vue()],
|
||||
server: {
|
||||
host: true,
|
||||
port: 5173,
|
||||
headers: crossOriginIsolationHeaders,
|
||||
proxy: {
|
||||
'/v1': {
|
||||
target: 'https://api.imageteach.tech:8002',
|
||||
@@ -13,6 +19,9 @@ export default defineConfig({
|
||||
}
|
||||
}
|
||||
},
|
||||
preview: {
|
||||
headers: crossOriginIsolationHeaders
|
||||
},
|
||||
build: {
|
||||
rollupOptions: {
|
||||
output: {
|
||||
|
||||
Reference in New Issue
Block a user