Refactor and enhance OCR and API functionalities

- Removed obsolete unit tests for TTS/ASR module.
- Deleted unused sample video file.
- Introduced OCRImageWrapper component for better OCR image handling with loading, success, and failure states.
- Updated copilot plugin to improve transaction handling and added new types for better type safety.
- Enhanced web search block plugin to support streaming content updates.
- Refactored API utility functions for better error handling and consistency across requests.
- Added new configuration for OCR API endpoint.
- Consolidated SSE event parsing into a shared utility.
- Created string utility functions to reduce code duplication.
- Removed outdated test documents related to compression functionality.
This commit is contained in:
“ydy0615”
2026-06-10 14:47:51 +08:00
parent 17d211bf93
commit 2283020e51
23 changed files with 641 additions and 1446 deletions
+129
View File
@@ -0,0 +1,129 @@
# 前端代码简化与优化提示词
## 目标
深入分析并简化前端代码结构,提升执行效率、可维护性和类型安全性。
## 优化原则
### 1. 代码简化
- **消除冗余**:识别并移除重复的逻辑、条件判断和错误处理模式
- **函数拆分**:将大型函数拆分为职责单一的小函数(每个函数只做一件事)
- **提取常量**:将魔法数字、字符串字面量提取为命名常量
- **减少嵌套**:使用早期返回(early return)替代深层 if/else 嵌套
### 2. 类型安全
- **明确类型**:为所有函数参数和返回值添加 TypeScript 类型注解
- **接口定义**:为复杂对象结构定义 interface,避免 `any` 类型
- **联合类型**:使用 discriminated unions 替代运行时 typeof 检查
### 3. 性能优化
- **懒加载**:对非核心模块使用动态 import()
- **防抖节流**:对频繁触发的事件(输入、滚动)添加 debounce/throttle
- **计算缓存**:对纯函数的重复计算结果进行 memoization
- **条件渲染**:使用 v-if/v-show 控制不必要的 DOM 操作
### 4. 错误处理
- **统一错误边界**:集中处理 fetch/API 调用异常
- **有意义错误信息**:避免空 catch,提供具体的失败原因
- **降级策略**:关键功能失败时有优雅的 fallback
### 5. 状态管理
- **最小化状态**:只存储必要的响应式数据
- **派生状态**:使用 computed 替代手动监听 + 条件赋值
- **作用域限制**:将状态定义在尽可能小的组件范围内
## 检查清单
### api.js 优化点
```typescript
// ❌ 问题:过多的条件分支和嵌套
function getCancelUrl(apiUrl) {
const normalized = String(apiUrl || '').replace(/\/+$/, '')
if (/\/v1\/pro\/completions$/i.test(normalized)) { /*...*/ }
if (/\/v1\/web-search$/i.test(normalized)) { /*...*/ }
// ...
}
// ✅ 优化:使用映射表替代条件链
const CANCEL_PATH_MAP = {
'/v1/pro/completions': '/v1/pro/completions/cancel',
'/v1/web-search': '/v1/web-search/cancel',
'/v1/completions': '/v1/completions/cancel',
}
function getCancelUrl(apiUrl) {
const base = new URL(apiUrl).pathname.replace(/\/+$/, '')
return CANCEL_PATH_MAP[base] || `${base}/cancel`
}
```
### 通用模式识别
1. **重复的 fetch 包装**:提取统一的 `safeFetch()` 函数处理认证头、错误和超时
2. **SSE 解析器重复**:将 `parseSseEvent` 抽象为可复用的 stream 处理器
3. **设置状态访问**:避免在每次 API 调用时重新创建 settings store 实例
4. **条件类型检查**:用 TypeScript discriminated union 替代运行时 `typeof x === 'string'`
## 执行步骤
### 第一步:分析
1. 使用 `grep_search` 查找重复模式(相同的 if/else 块、try/catch
2. 使用 `semantic_search` 查找相似功能的不同实现
3. 识别高频调用的函数(API 请求、事件处理器)
### 第二步:重构
1. **提取纯函数**:将副作用(fetch、DOM操作)与数据处理分离
2. **创建工具库**:将通用逻辑移至 `src/utils/` 下的独立模块
3. **添加类型定义**:在 `src/plugins/types.ts` 中集中管理接口
4. **简化条件逻辑**:用策略模式或映射表替代 switch/if 链
### 第三步:验证
1. 运行 `npm run build` 确认无类型错误
2. 检查 `get_errors` 确保没有引入新问题
3. 手动测试关键路径(补全、OCR、上传)
## 输出格式
每次优化后提供:
```markdown
### 优化项: [函数名/文件名]
**问题**: [简要描述当前代码的问题]
**改动**:
```diff
- // 旧代码
+ // 新代码
```
**收益**:
- 行数减少: X%
- 时间复杂度: O(n) → O(1)
- 可读性提升: [具体说明]
```
## 示例调用
```bash
# 简化 api.js 中的 URL 处理逻辑
"简化 src/utils/api.js 中 getCancelUrl() 函数的条件分支,使用映射表替代正则匹配"
# 优化 copilotPlugin.ts 的类型定义
"为 src/plugins/copilotPlugin.ts 添加完整的 TypeScript 类型注解,消除所有 any 类型"
# 提取重复的错误处理
"将 src/utils/api.js 中分散的 try/catch 错误处理提取为统一的 errorBoundary() 高阶函数"
```
## 注意事项
- ✅ 保持向后兼容:不破坏现有 API 接口和事件流
- ✅ 小步快跑:每次只优化一个函数或模块,验证后再继续
- ❌ 避免过度优化:不要为了炫技引入复杂的函数式编程模式
- ⚠️ 测试覆盖:修改核心路径(补全、取消请求)前确保有对应测试
- 📝 文档同步:更新 CLAUDE.md 和 AGENTS.md 中的架构描述
## 参考文件
- `src/utils/api.js` - API 请求层,存在多处可简化的条件逻辑
- `src/plugins/copilotPlugin.ts` - 补全插件,类型定义不完整
- `src/stores/settings.js` - 状态管理,可优化响应式依赖
- `backend/prompt.py` - Prompt 组装逻辑(后端参考)
+15 -3
View File
@@ -639,7 +639,12 @@ async def web_search_handler(
f"原始上下文:\n{context}\n\n"
f"抓取内容:\n{crawled_text}"
)
synthesis_result = await call_ollama(
# 流式合成 — 逐 delta emit,前端可实时渲染
from llm import stream_ollama_events
accumulated: list[str] = []
async for event_type, text in stream_ollama_events(
synthesis_prompt,
system_prompt="Return only the final markdown body with no title and no citations list.",
tag=f'{payload["request_id"][:8]}-webf',
@@ -647,8 +652,15 @@ async def web_search_handler(
thinking=policy.get("thinking"),
model=policy.get("model"),
max_output_tokens=int(policy.get("max_output_tokens", 0) or 0),
)
content = _normalize_multiline_text(synthesis_result.get("content") or "")
):
if is_cancelled():
raise asyncio.CancelledError()
# 只推送 content delta,不展示 thinking
if event_type == "content":
accumulated.append(text)
await emit("delta", {"text": text})
content = _normalize_multiline_text("".join(accumulated))
if not content:
raise RuntimeError("联网搜索生成了空结果")
created_at = str(req.get("created_at") or payload.get("created_at") or "").strip() or datetime.now().isoformat()
-225
View File
@@ -1,225 +0,0 @@
import asyncio
import importlib
import sys
from pathlib import Path
import pytest
BACKEND_DIR = Path(__file__).resolve().parents[1]
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
try:
llm = importlib.import_module("llm")
except ModuleNotFoundError:
pytest.skip("llm module dependencies are not available", allow_module_level=True)
def test_extract_message_with_content_and_thinking():
resp = {"choices": [{"message": {"content": "hello world", "thinking": "reasoning"}}]}
content, thinking = llm._extract_message(resp)
assert content == "hello world"
assert thinking == "reasoning"
def test_extract_message_empty_content():
resp = {"choices": [{"message": {"content": "", "thinking": None}}]}
content, thinking = llm._extract_message(resp)
assert content == ""
assert thinking == ""
def test_extract_message_dict_no_choices():
resp = {"not_choices": []}
content, thinking = llm._extract_message(resp)
assert content == ""
assert thinking == ""
def test_extract_message_empty_dict():
resp = {}
content, thinking = llm._extract_message(resp)
assert content == ""
assert thinking == ""
def test_extract_delta_text_from_chunk():
chunk = {"choices": [{"delta": {"content": "text"}}]}
assert llm._extract_delta_text(chunk) == "text"
def test_extract_delta_thinking_from_chunk():
chunk = {"choices": [{"delta": {"thinking": "thought"}}]}
assert llm._extract_delta_thinking(chunk) == "thought"
def test_call_ollama_no_system(monkeypatch):
captured = {}
async def fake_post(url, json=None):
captured["json"] = json
class FakeResp:
def raise_for_status(self): pass
def json(self): return {"choices": [{"message": {"content": "ok"}}]}
return FakeResp()
async def fake_client(*args, **kwargs):
class Ctx:
async def __aenter__(self2): return self2
async def __aexit__(*a): pass
post = fake_post
return Ctx()
monkeypatch.setattr(llm.httpx, "AsyncClient", fake_client)
monkeypatch.setattr(llm.asyncio, "wait_for", lambda coro, **kw: coro)
result = asyncio.run(
llm.call_ollama("user prompt", system_prompt=None, tag="no-system")
)
assert result["content"] == "ok"
# Should only have user message, no system
assert len(captured["json"]["messages"]) == 1
def test_call_ollama_with_system(monkeypatch):
captured = {}
async def fake_post(url, json=None):
captured["json"] = json
class FakeResp:
def raise_for_status(self): pass
def json(self): return {"choices": [{"message": {"content": "ok"}}]}
return FakeResp()
async def fake_client(*args, **kwargs):
class Ctx:
async def __aenter__(self2): return self2
async def __aexit__(*a): pass
post = fake_post
return Ctx()
monkeypatch.setattr(llm.httpx, "AsyncClient", fake_client)
monkeypatch.setattr(llm.asyncio, "wait_for", lambda coro, **kw: coro)
result = asyncio.run(
llm.call_ollama("user prompt", system_prompt="sys prompt", tag="with-system")
)
assert result["content"] == "ok"
# Should have both system and user messages
msgs = captured["json"]["messages"]
assert len(msgs) == 2
assert msgs[0]["role"] == "system"
def test_call_ollama_with_custom_model(monkeypatch):
captured = {}
async def fake_post(url, json=None):
captured["json"] = json
class FakeResp:
def raise_for_status(self): pass
def json(self): return {"choices": [{"message": {"content": "ok"}}]}
return FakeResp()
async def fake_client(*args, **kwargs):
class Ctx:
async def __aenter__(self2): return self2
async def __aexit__(*a): pass
post = fake_post
return Ctx()
monkeypatch.setattr(llm.httpx, "AsyncClient", fake_client)
monkeypatch.setattr(llm.asyncio, "wait_for", lambda coro, **kw: coro)
result = asyncio.run(
llm.call_ollama("prompt", model="custom-model")
)
assert captured["json"]["model"] == "custom-model"
def test_stream_ollama_events_error_handling(monkeypatch):
def make_lines():
lines_iter = iter([
'data: {"error": "model not found"}',
])
class LineIterator:
async def __anext__(self):
try:
return next(lines_iter)
except StopIteration:
raise StopAsyncIteration()
class Response:
def __init__(self2): self2._lines = LineIterator()
async def raise_for_status(self2): pass
async def aiter_lines(self2): return self2._lines
class StreamCtx:
async def __aenter__(self2): return Response()
async def __aexit__(*a): pass
class Client:
stream = lambda self2, *args, **kw: StreamCtx()
return Client()
async def fake_client(*args, **kwargs):
return make_lines()
monkeypatch.setattr(llm.httpx, "AsyncClient", fake_client)
monkeypatch.setattr(llm.asyncio, "wait_for", lambda coro, **kw: coro)
async def collect():
try:
async for _ in llm.stream_ollama_events("prompt", tag="err"):
pass
except RuntimeError as e:
return str(e)
result = asyncio.run(collect())
assert "model not found" in str(result)
def test_call_vlm_ocr_payload_format(monkeypatch):
captured = {}
async def fake_post(url, json=None):
captured["json"] = json
class FakeResp:
def raise_for_status(self): pass
def json(self): return {"choices": [{"message": {"content": "ocr result"}}]}
return FakeResp()
async def fake_client(*args, **kwargs):
class Ctx:
async def __aenter__(self2): return self2
async def __aexit__(*a): pass
post = fake_post
return Ctx()
monkeypatch.setattr(llm.httpx, "AsyncClient", fake_client)
monkeypatch.setattr(llm.asyncio, "wait_for", lambda coro, **kw: coro)
result = asyncio.run(llm.call_vlm_ocr(b"image"))
assert result == "ocr result"
# Verify vision format: image_url content part with base64
msgs = captured["json"]["messages"]
assert len(msgs) == 1
content_parts = msgs[0]["content"]
image_part = [p for p in content_parts if p.get("type") == "image_url"]
assert len(image_part) == 1
-147
View File
@@ -1,147 +0,0 @@
import sys
import re
from pathlib import Path
# Ensure the project root is in sys.path so imports like `from backend import prompt` work
ROOT = Path(__file__).resolve().parents[2]
BACKEND_DIR = ROOT / "backend"
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(BACKEND_DIR))
from backend import prompt # type: ignore
def test_get_current_datetime_auto_format():
s = prompt._get_current_datetime("auto")
assert isinstance(s, str)
# Expect a date-like prefix: YYYY-MM-DD
assert re.match(r"^\d{4}-\d{2}-\d{2}", s)
# Expect a 3-letter weekday somewhere
assert re.search(r"\b[A-Za-z]{3}\b", s)
# Accept either an explicit UTC offset or a UTC label
assert re.search(r"UTC|[+-]\d{2}:?\d{2}", s)
def test_get_current_datetime_utc_plus5():
s = prompt._get_current_datetime("UTC+5")
assert isinstance(s, str)
assert "UTC+5" in s
def test_get_current_datetime_gmt_minus3():
s = prompt._get_current_datetime("GMT-3")
assert isinstance(s, str)
assert "GMT-3" in s
def test_get_current_datetime_new_york_fallback():
s = prompt._get_current_datetime("America/New_York")
assert isinstance(s, str)
# Fallback behavior: allow either an explicit offset or a simple date prefix
ok = bool(re.search(r"[+-]\d{2}:?\d{2}", s)) or bool(re.match(r"^\d{4}-\d{2}-\d{2}", s))
assert ok
def test_sanitize_language_id_empty_none_and_chars():
# Empty / None should map to markdown by design
assert prompt._sanitize_language_id("") == "markdown"
assert prompt._sanitize_language_id(None) == "markdown"
# Dangerous chars should be stripped
sanitized = prompt._sanitize_language_id("<script>alert(1)</script>")
assert "<" not in sanitized and ">" not in sanitized
# Valid input preserved
assert prompt._sanitize_language_id("python") == "python"
# Truncation at 32 chars
long_input = "a" * 50
trimmed = prompt._sanitize_language_id(long_input)
assert len(trimmed) <= 32
assert trimmed == "a" * min(32, len(long_input))
def test_normalize_newlines():
mixed = "line1\r\nline2\rline3\n"
norm = prompt._normalize_newlines(mixed)
assert norm == "line1\nline2\nline3\n"
def test_canonical_language_id_synonyms_and_unknown():
assert prompt._canonical_language_id("md") == "markdown"
assert prompt._canonical_language_id("py") == "python"
assert prompt._canonical_language_id("js") == "javascript"
assert prompt._canonical_language_id("ts") == "typescript"
assert prompt._canonical_language_id("yml") == "yaml"
assert prompt._canonical_language_id("Rust") == "rust"
def test_language_guidance_behaviors():
# markdown yields empty guidance
assert prompt._language_guidance("markdown") == ""
# mermaid guidance should mention mermaid
g_mermaid = prompt._language_guidance("mermaid")
assert isinstance(g_mermaid, str)
assert "mermaid" in g_mermaid.lower()
# python / javascript should reference the language
g_py = prompt._language_guidance("python")
assert isinstance(g_py, str) and "python" in g_py.lower()
g_js = prompt._language_guidance("javascript")
assert isinstance(g_js, str) and "javascript" in g_js.lower()
# unknown language should return a string as fallback
g_unknown = prompt._language_guidance("unknownlang")
assert isinstance(g_unknown, str)
def test_build_inline_system_prompt_templates():
s_md = prompt.build_inline_system_prompt("markdown")
assert isinstance(s_md, str) and "markdown" in s_md.lower()
s_mermaid = prompt.build_inline_system_prompt("mermaid")
assert isinstance(s_mermaid, str) and "mermaid" in s_mermaid.lower()
def test_prepare_context_strips_br_tags():
prefix, suffix = prompt._prepare_context("<br>hello<br/>", "world<br />")
assert "<br" not in prefix
assert "<br" not in suffix
def test_cursor_and_fence_helpers_basic():
sample = "```python\nprint('hi')\n"
assert prompt._cursor_in_fenced_code_block(sample) is True
assert prompt._cursor_in_fenced_code_block("plain text") is False
assert prompt._active_fence_language(sample) == "python"
assert prompt._active_fence_language("plain text") == "none"
def test_is_mermaid_context_detection():
assert prompt._is_mermaid_context("flowchart TD", "", "none") is True
assert prompt._is_mermaid_context("```mermaid\n", "\n```", "mermaid") is True
assert prompt._is_mermaid_context("plain text", "", "none") is False
def test_build_completion_prompts_with_userprefs():
class UserPrefs:
language = "python"
currency = "USD"
timezone = "UTC+0"
system, user, prefill = prompt.build_completion_prompts(
prefix="hello", suffix="world", language_id="markdown",
preferences=UserPrefs(),
)
assert isinstance(system, str)
assert isinstance(user, str)
assert prefill == "hello"
assert "python" in user.lower() or "USD" in user
def test_build_completion_prompts_privacy_mode_location_empty():
system, user, prefill = prompt.build_completion_prompts(
prefix="hello", suffix="world", language_id="markdown",
location="",
)
assert isinstance(system, str)
assert isinstance(user, str)
assert prefill == "hello"
def test_build_prompt_backward_compatibility():
res = prompt.build_prompt(prefix="hello", suffix="world", language_id="markdown")
assert isinstance(res, str)
-193
View File
@@ -1,193 +0,0 @@
import os
import sys
import asyncio
import types
import pytest
from pathlib import Path
from unittest.mock import MagicMock, patch
BACKEND_DIR = Path(__file__).resolve().parents[1]
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
def _make_mlx_stub():
"""Create minimal MLX stub for testing without Apple Silicon"""
mlx = types.SimpleNamespace()
mlx.core = types.SimpleNamespace()
mx_array = type('mx.array', (), {'item': lambda self: 1})
mlx.core.array = mx_array
mlx.nn = types.SimpleNamespace()
return mlx
def _make_mlx_audio_stub():
"""Create minimal mlx-audio stub"""
stt = types.SimpleNamespace()
stt.utils = types.SimpleNamespace()
def mock_load(path, **kwargs):
model = MagicMock()
return model
stt.utils.load = mock_load # type: ignore
qwen3_asr_mod = types.SimpleNamespace()
qwen3_asr_mod.Qwen3ASRModel = type('Qwen3ASRModel', (), {})
qwen3_asr_mod.ForcedAlignerModel = type('ForcedAlignerModel', (), {})
stt.models = types.SimpleNamespace() # type: ignore
stt.models.qwen3_asr = qwen3_asr_mod # type: ignore
audio = types.SimpleNamespace()
audio.stt = stt # type: ignore
return audio
def _reload_tts_asr_with_mocks():
"""Reload tts_asr with mocked MLX dependencies"""
for mod_name in list(sys.modules.keys()):
if 'tts_asr' in mod_name or 'mlx' in mod_name:
del sys.modules[mod_name]
mlx_stub = _make_mlx_stub()
sys.modules['mlx'] = mlx_stub # type: ignore
sys.modules['mlx.core'] = mlx_stub.core # type: ignore
sys.modules['mlx.nn'] = mlx_stub.nn # type: ignore
audio_stub = _make_mlx_audio_stub()
sys.modules['mlx-audio'] = audio_stub # type: ignore
sys.modules['mlx_audio'] = audio_stub # type: ignore
sys.modules['mlx_audio.stt'] = audio_stub.stt # type: ignore
sys.modules['mlx_audio.stt.utils'] = audio_stub.stt.utils # type: ignore
sys.modules['mlx_audio.stt.models'] = audio_stub.stt.models # type: ignore
sys.modules['mlx_audio.stt.models.qwen3_asr'] = audio_stub.stt.models.qwen3_asr # type: ignore
import tts_asr
return tts_asr
@pytest.fixture(autouse=True)
def _clean_env():
"""Clean ASR-related env vars before/after each test"""
saved = {}
for k in ['HF_ENDPOINT']:
saved[k] = os.environ.get(k)
if k in os.environ:
del os.environ[k]
yield
for k, v in saved.items():
if v is not None:
os.environ[k] = v # type: ignore (unused var)
class TestRequestResponseModels:
"""Pydantic 数据模型测试"""
def test_tts_request_defaults(self):
tts = _reload_tts_asr_with_mocks()
req = tts.TTSRequest(text="hello")
assert req.text == "hello"
assert req.speaker == "Vivian"
def test_asr_request_defaults(self):
tts = _reload_tts_asr_with_mocks()
req = tts.ASRRequest(audio_base64="dGVzdA==")
assert req.audio_base64 == "dGVzdA=="
assert req.language == "zh-CN"
def test_asr_request_custom_language(self):
tts = _reload_tts_asr_with_mocks()
req = tts.ASRRequest(audio_base64="dGVzdA==", language="en")
assert req.language == "en"
def test_model_status_defaults(self):
tts = _reload_tts_asr_with_mocks()
status = tts.ModelStatus(tts_loaded=False, asr_loaded=True, device="cpu")
assert not status.tts_loaded
assert status.asr_loaded
class TestDeviceDetection:
"""设备检测测试"""
def test_device_map_returns_string(self):
tts = _reload_tts_asr_with_mocks()
device = tts._get_device_map()
assert isinstance(device, str)
class TestModelLoading:
"""模型加载测试"""
def test_load_asr_skips_when_mlx_unavailable(self):
"""mlx_audio 未安装时应跳过 ASR"""
for mod_name in list(sys.modules.keys()):
if 'tts_asr' in mod_name or 'mlx' in mod_name:
del sys.modules[mod_name]
# Don't inject mlx stubs — simulate missing MLX
import tts_asr # noqa: F811
assert tts_asr.Qwen3ASRModel is None
tts_asr._load_asr_models() # should not crash
assert tts_asr._asr_model is None
def test_load_asr_from_path_success(self):
tts = _reload_tts_asr_with_mocks()
# Mock snapshot_download to return a path, mock stt_load to succeed
with patch('backend.tts_asr.snapshot_download', return_value='/fake/path'): # type: ignore
tts._load_asr_from_path('/fake/path')
assert tts._asr_model is not None # type: ignore (MagicMock)
class TestWarmupFunctions:
"""预热函数测试"""
def test_warmup_functions_callable(self):
tts = _reload_tts_asr_with_mocks()
assert callable(tts._warmup_tts) # type: ignore (unused var)
assert callable(tts._warmup_all)
def test_warmup_asr_skips_when_mlx_unavailable(self):
for mod_name in list(sys.modules.keys()):
if 'tts_asr' in mod_name or 'mlx' in mod_name:
del sys.modules[mod_name]
import tts_asr # noqa: F811
assert tts_asr.Qwen3ASRModel is None
def test_warmup_all_runs_without_error(self):
tts = _reload_tts_asr_with_mocks()
# Set global models so warmup returns immediately without actual loading
tts._tts_model = MagicMock()
async def run(): # type: ignore (unused var)
await tts._warmup_all()
asyncio.get_event_loop().run_until_complete(run()) # type: ignore
class TestRouteRegistration:
"""路由注册测试"""
def test_register_function_exists(self):
tts = _reload_tts_asr_with_mocks()
assert callable(tts.register_tts_asr_routes)
def test_router_prefix(self):
tts = _reload_tts_asr_with_mocks()
assert hasattr(tts.router, 'routes')
class TestModelConstants:
"""模型常量测试"""
def test_asr_model_id(self):
tts = _reload_tts_asr_with_mocks()
assert 'Qwen3-ASR' in tts.ASR_MODEL_ID_MS
def test_align_model_id(self):
tts = _reload_tts_asr_with_mocks()
assert 'ForcedAligner' in tts.ALIGN_MODEL_ID_MS
-263
View File
@@ -1,263 +0,0 @@
import os
import sys
import base64
import io
import types
import wave
import pytest
from pathlib import Path
from unittest.mock import MagicMock, patch
import numpy as np
BACKEND_DIR = Path(__file__).resolve().parents[1]
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
def _make_mlx_stub():
"""Create minimal MLX stub for testing without Apple Silicon"""
mlx = types.SimpleNamespace()
mlx.core = types.SimpleNamespace()
mx_array = type('mx.array', (), {'item': lambda self: 1})
mlx.core.array = mx_array
def mock_load(path):
return MagicMock()
mlx.core.load = mock_load # type: ignore
mlx.nn = types.SimpleNamespace()
return mlx
def _make_mlx_audio_stub():
"""Create minimal mlx-audio stub"""
stt = types.SimpleNamespace()
stt.utils = types.SimpleNamespace()
def mock_load(path): # type: ignore
model = MagicMock()
output = types.SimpleNamespace()
output.text = "识别结果"
output.language = "zh-CN"
model.generate = MagicMock(return_value=output)
return model
stt.utils.load = mock_load # type: ignore
qwen3_asr_mod = types.SimpleNamespace()
qwen3_asr_mod.Qwen3ASRModel = type('Qwen3ASRModel', (), {})
qwen3_asr_mod.ForcedAlignerModel = type('ForcedAlignerModel', (), {})
stt.models = types.SimpleNamespace() # type: ignore
stt.models.qwen3_asr = qwen3_asr_mod # type: ignore
audio = types.SimpleNamespace()
audio.stt = stt # type: ignore
return audio
def _reload_tts_asr_with_mocks():
"""Reload tts_asr with mocked MLX dependencies"""
for mod_name in list(sys.modules.keys()):
if 'tts_asr' in mod_name or 'mlx' in mod_name:
del sys.modules[mod_name]
mlx_stub = _make_mlx_stub()
sys.modules['mlx'] = mlx_stub # type: ignore
sys.modules['mlx.core'] = mlx_stub.core # type: ignore
sys.modules['mlx.nn'] = mlx_stub.nn # type: ignore
audio_stub = _make_mlx_audio_stub()
sys.modules['mlx-audio'] = audio_stub # type: ignore
sys.modules['mlx_audio'] = audio_stub # type: ignore
sys.modules['mlx_audio.stt'] = audio_stub.stt # type: ignore
sys.modules['mlx_audio.stt.utils'] = audio_stub.stt.utils # type: ignore
sys.modules['mlx_audio.stt.models'] = audio_stub.stt.models # type: ignore
sys.modules['mlx_audio.stt.models.qwen3_asr'] = audio_stub.stt.models.qwen3_asr # type: ignore
import tts_asr
return tts_asr, audio_stub
@pytest.fixture(autouse=True)
def _clean_env():
"""Clean ASR-related env vars before/after each test"""
saved = {}
for k in ['HF_ENDPOINT']:
saved[k] = os.environ.get(k)
if k in os.environ:
del os.environ[k]
yield
for k, v in saved.items():
if v is not None:
os.environ[k] = v # type: ignore
def _make_wav_bytes(sr=16000, duration_sec=1.0, channels=1):
"""Helper: generate WAV bytes as base64"""
samples = int(sr * duration_sec)
audio = np.random.randint(-32768, 32767, size=samples * channels, dtype=np.int16)
buf = io.BytesIO()
with wave.open(buf, 'wb') as wf:
wf.setnchannels(channels)
wf.setsampwidth(2)
wf.setframerate(sr)
wf.writeframes(audio.tobytes())
return base64.b64encode(buf.getvalue()).decode()
class TestASRLazyLoading:
"""测试 ASR 模型懒加载"""
def test_ensure_asr_loads_on_call(self):
tts, audio_stub = _reload_tts_asr_with_mocks()
assert tts._asr_model is None
model = tts._ensure_asr_model()
assert model is not None
def test_ensure_align_loads_on_call(self):
tts, audio_stub = _reload_tts_asr_with_mocks()
assert tts._align_model is None
model = tts._ensure_align_model()
assert model is not None
class TestASREndpoint:
"""测试 ASR 端点逻辑"""
def test_asr_basic_recognition(self, fastapi_testclient=None):
"""ASR 端点应正确返回识别结果"""
tts, _ = _reload_tts_asr_with_mocks()
# Mock the model to return known values
tts._asr_model = MagicMock()
output = types.SimpleNamespace()
output.text = "你好世界"
output.language = "zh-CN"
tts._asr_model.generate.return_value = output
wav_b64 = _make_wav_bytes()
req = tts.ASRRequest(audio_base64=wav_b64)
# Call generate directly (simulating endpoint logic)
audio_bytes = base64.b64decode(req.audio_base64)
wav_buffer = io.BytesIO(audio_bytes)
with wave.open(wav_buffer, 'rb') as wf:
raw = wf.readframes(wf.getnframes())
arr = np.frombuffer(raw, dtype=np.int16)
arr = arr.astype(np.float32) / 32768.0
result = tts._asr_model.generate(arr, language=req.language)
assert result.text == "你好世界"
def test_asr_stereo_to_mono(self):
"""立体声音频应被正确转换为单声道"""
wav_b64 = _make_wav_bytes(channels=2)
audio_bytes = base64.b64decode(wav_b64)
wav_buffer = io.BytesIO(audio_bytes)
with wave.open(wav_buffer, 'rb') as wf:
assert wf.getnchannels() == 2
n_frames = wf.getnframes()
raw_data = wf.readframes(n_frames)
audio_array = np.frombuffer(raw_data, dtype=np.int16)
# Convert to mono
audio_array = np.mean(audio_array.reshape(-1, 2), axis=1)
assert audio_array.ndim == 1
def test_asr_resample_to_16k(self):
"""非 16kHz 音频应被重采样"""
wav_b64 = _make_wav_bytes(sr=48000, duration_sec=0.5)
audio_bytes = base64.b64decode(wav_b64)
wav_buffer = io.BytesIO(audio_bytes)
with wave.open(wav_buffer, 'rb') as wf:
assert wf.getframerate() == 48000
def test_asr_44100_resample(self):
"""44.1kHz 常见采样率应被重采样到 16k"""
wav_b64 = _make_wav_bytes(sr=44100, duration_sec=1.0)
audio_bytes = base64.b64decode(wav_b64)
wav_buffer = io.BytesIO(audio_bytes)
with wave.open(wav_buffer, 'rb') as wf:
framerate = wf.getframerate()
n_frames = wf.getnframes()
raw_data = wf.readframes(n_frames)
audio_array = np.frombuffer(raw_data, dtype=np.int16)
# Simulate resample calculation
if framerate != 16000:
n_samples = int(len(audio_array) * 16000 / framerate)
else:
n_samples = len(audio_array)
expected_16k_samples = int(1.0 * 16000)
assert abs(n_samples - expected_16k_samples) < 2
class TestASRModelDownload:
"""测试 ASR 模型下载路径"""
def test_load_asr_from_path_success(self):
tts, _ = _reload_tts_asr_with_mocks()
with patch('backend.tts_asr.snapshot_download', return_value='/fake/asr'): # type: ignore
tts._load_asr_models()
assert tts._asr_model is not None
def test_load_asr_skips_without_mlx(self):
"""不注入 MLX stub 时应跳过 ASR"""
for mod_name in list(sys.modules.keys()):
if 'tts_asr' in mod_name or 'mlx' in mod_name:
del sys.modules[mod_name]
import tts_asr # noqa: F811
assert tts_asr.Qwen3ASRModel is None
def test_load_align_from_path(self):
tts, _ = _reload_tts_asr_with_mocks()
with patch('backend.tts_asr.snapshot_download', return_value='/fake/align'): # type: ignore
tts._load_asr_models()
assert tts._align_model is not None
class TestModelConstants:
"""测试模型 ID 常量"""
def test_asr_model_id(self):
tts, _ = _reload_tts_asr_with_mocks()
assert "aufklarer" in tts.ASR_MODEL_ID_MS
def test_align_model_id(self):
tts, _ = _reload_tts_asr_with_mocks()
assert "ForcedAligner" in tts.ALIGN_MODEL_ID_MS
def test_tts_model_id(self):
tts, _ = _reload_tts_asr_with_mocks()
assert "Qwen3-TTS" in tts.MODEL_ID_MS
class TestHFEndpointMirror:
"""测试镜像站配置"""
def test_hf_endpoint_set(self):
tts, _ = _reload_tts_asr_with_mocks()
assert os.environ.get("HF_ENDPOINT") == "https://hf-mirror.com"
def test_hf_endpoint_default(self):
"""即使环境变量未设置,模块也应默认设置镜像"""
for mod_name in list(sys.modules.keys()):
if 'tts_asr' in mod_name or 'mlx' in mod_name:
del sys.modules[mod_name]
if "HF_ENDPOINT" in os.environ:
del os.environ["HF_ENDPOINT"]
import tts_asr # noqa: F811
assert os.environ.get("HF_ENDPOINT") == "https://hf-mirror.com"
-305
View File
@@ -1,305 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
TTS/ASR模块集成测试 — MLX/Qwen3-ASR 版本
测试API端点和完整流程(需要运行后端服务)
运行方式:
pytest backend/tests/test_tts_asr_integration.py -v -s
python backend/tests/test_tts_asr_integration.py --test asr
MLX 模型通过 ModelScope (aufklarer/Qwen3-ASR) + ForcedAligner
"""
import argparse
import base64
import io
import os
import sys
import time
import unittest
from typing import Optional
try:
import httpx # type: ignore
except ImportError:
print("httpx 未安装,跳过集成测试")
sys.exit(1)
import numpy as np
API_BASE_URL = os.environ.get('API_BASE_URL', 'http://localhost:8001')
API_KEY = os.environ.get('API_KEY', 'your-secret-key-here')
TEST_TIMEOUT = 120.0
class TTSASRIntegrationTest(unittest.TestCase):
"""TTS/ASR集成测试"""
@classmethod
def setUpClass(cls):
cls.client = httpx.Client(timeout=TEST_TIMEOUT)
cls.headers = {'X-API-Key': API_KEY}
try:
response = cls.client.get(f'{API_BASE_URL}/v1/tts-asr/status', headers=cls.headers)
if response.status_code == 200:
cls.service_available = True
print(f"\n✓ 服务可用: {API_BASE_URL}")
else:
cls.service_available = False
print(f"\n✗ 服务返回非200状态码: {response.status_code}")
except Exception as e: # noqa: ANN001
cls.service_available = False
print(f"\n✗ 无法连接到服务: {e}")
@classmethod
def tearDownClass(cls):
cls.client.close()
def setUp(self):
if not self.service_available:
self.skipTest("后端服务不可用")
def test_01_config_endpoint(self):
"""测试配置端点"""
response = self.client.get(
f'{API_BASE_URL}/v1/tts-asr/config',
headers=self.headers
)
self.assertEqual(response.status_code, 200)
config = response.json()
self.assertIn('device', config)
self.assertIn('model', config)
self.assertIn('status', config)
model = config['model']
status = config['status']
self.assertIn('tts', model)
self.assertIn('asr', model)
print(f"\n配置信息:")
print(f" TTS模型: {model['tts']}")
print(f" ASR模型: {model.get('asr', 'N/A')}")
print(f" TTS已加载: {status['tts_loaded']}")
print(f" ASR已加载: {status['asr_loaded']}")
def test_02_status_endpoint(self):
"""测试状态端点"""
response = self.client.get(
f'{API_BASE_URL}/v1/tts-asr/status',
headers=self.headers
)
self.assertEqual(response.status_code, 200)
status = response.json()
self.assertIn('tts_loaded', status)
self.assertIn('asr_loaded', status)
self.assertIn('device', status)
print(f"\n状态信息:")
print(f" TTS已加载: {status['tts_loaded']}")
print(f" ASR已加载: {status['asr_loaded']}")
print(f" 设备: {status['device']}")
def test_03_warmup_endpoint(self):
"""测试预热端点"""
print("\n开始模型预热(可能需要几分钟)...")
start_time = time.time()
response = self.client.post(
f'{API_BASE_URL}/v1/tts-asr/warmup',
headers=self.headers,
)
elapsed = time.time() - start_time
self.assertEqual(response.status_code, 200)
result = response.json()
self.assertIn('tts_warmup', result)
self.assertIn('asr_warmup', result)
print(f"\n预热完成 (耗时: {elapsed:.2f}秒):")
print(f" TTS预热: {'成功' if result['tts_warmup'] else '失败'}")
print(f" ASR预热: {'成功' if result.get('asr_warmup') else '失败/跳过'}")
if not result['tts_warmup'] or not result.get('asr_warmup'):
print("\n⚠ 警告: 预热失败可能是因为模型未下载")
def test_04_tts_endpoint_basic(self):
"""测试TTS基本功能"""
test_text = "这是一个测试"
response = self.client.post(
f'{API_BASE_URL}/v1/tts-asr/tts',
headers=self.headers,
json={'text': test_text}
)
if response.status_code == 500:
error = response.json()
print(f"\n⚠ TTS失败(可能是模型未加载): {error.get('detail', 'Unknown error')}")
self.skipTest("TTS模型未加载或不可用")
self.assertEqual(response.status_code, 200)
result = response.json()
self.assertIn('audio_base64', result)
self.assertIn('format', result)
self.assertIn('duration_ms', result)
audio_data = base64.b64decode(result['audio_base64'])
self.assertGreater(len(audio_data), 0)
print(f"\nTTS测试成功:")
print(f" 输入文本: {test_text}")
print(f" 音频大小: {len(audio_data)} bytes")
def test_05_asr_endpoint_basic(self):
"""测试ASR基本功能"""
sample_rate = 16000
duration = 1.0
samples = int(sample_rate * duration)
silence = np.zeros(samples, dtype=np.int16)
wav_buffer = io.BytesIO()
with wave.open(wav_buffer, 'wb') as wf: # noqa: SIM115
wf.setnchannels(1)
wf.setsampwidth(2)
wf.setframerate(sample_rate)
wf.writeframes(silence.tobytes())
audio_bytes = wav_buffer.getvalue()
audio_base64 = base64.b64encode(audio_bytes).decode()
response = self.client.post(
f'{API_BASE_URL}/v1/tts-asr/asr',
headers=self.headers,
json={
'audio_base64': audio_base64,
'language': 'zh-CN'
}
)
if response.status_code in (500, 501):
detail = response.json().get('detail', 'Unknown')
print(f"\n⚠ ASR失败: {detail}")
self.skipTest("ASR模型未加载或不可用")
self.assertEqual(response.status_code, 200)
result = response.json()
self.assertIn('text', result)
self.assertIn('language', result)
print(f"\nASR测试成功:")
print(f" 识别文本: '{result['text']}'")
print(f" 语言: {result['language']}")
def test_06_api_key_validation(self):
"""测试API密钥验证"""
wrong_headers = {'X-API-Key': 'wrong-api-key'}
response = self.client.get(
f'{API_BASE_URL}/v1/tts-asr/status',
headers=wrong_headers,
)
self.assertEqual(response.status_code, 403)
class PerformanceTest(unittest.TestCase):
"""性能测试"""
@classmethod
def setUpClass(cls):
cls.client = httpx.Client(timeout=TEST_TIMEOUT)
cls.headers = {'X-API-Key': API_KEY}
try:
response = cls.client.get(f'{API_BASE_URL}/v1/tts-asr/status', headers=cls.headers)
cls.service_available = response.status_code == 200
except Exception: # noqa: ANN001, S110
cls.service_available = False
@classmethod
def tearDownClass(cls):
cls.client.close()
def setUp(self):
if not self.service_available:
self.skipTest("后端服务不可用")
def test_tts_latency(self):
"""测试TTS延迟"""
latencies = []
for i in range(3):
start = time.time()
response = self.client.post(
f'{API_BASE_URL}/v1/tts-asr/tts',
headers=self.headers,
json={'text': '测试延迟'}
)
elapsed = time.time() - start
if response.status_code == 200:
latencies.append(elapsed)
if latencies:
print(f"\nTTS延迟测试:")
print(f" 平均: {sum(latencies)/len(latencies):.3f}s")
print(f" 最小: {min(latencies):.3f}s / 最大: {max(latencies):.3f}s")
def run_tests(test_type: Optional[str] = None) -> bool:
"""运行测试"""
loader = unittest.TestLoader()
suite = unittest.TestSuite()
TEST_MAP = {
'config': ('TTSASRIntegrationTest', 'test_01_config_endpoint'),
'status': ('TTSASRIntegrationTest', 'test_02_status_endpoint'),
'warmup': ('TTSASRIntegrationTest', 'test_03_warmup_endpoint'),
'tts': ('TTSASRIntegrationTest', 'test_04_tts_endpoint_basic'),
'asr': ('TTSASRIntegrationTest', 'test_05_asr_endpoint_basic'),
'perf': ('PerformanceTest', None),
}
if test_type and test_type in TEST_MAP:
cls_name, method = TEST_MAP[test_type]
if method:
suite.addTest(globals()[cls_name](method))
else:
suite.addTests(loader.loadTestsFromTestCase(globals()[cls_name]))
elif test_type == 'api_key':
suite.addTest(TTSASRIntegrationTest('test_06_api_key_validation'))
else:
suite.addTests(loader.loadTestsFromTestCase(TTSASRIntegrationTest))
suite.addTests(loader.loadTestsFromTestCase(PerformanceTest))
runner = unittest.TextTestRunner(verbosity=2)
result = runner.run(suite)
return result.wasSuccessful()
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='TTS/ASR 集成测试')
parser.add_argument('--test', choices=['config', 'status', 'warmup', 'tts', 'asr', 'perf', 'api_key'])
parser.add_argument('--url', default=API_BASE_URL)
parser.add_argument('--key', default=API_KEY)
args = parser.parse_args()
API_BASE_URL = args.url
API_KEY = args.key
print("=" * 70)
print("TTS/ASR 集成测试 (MLX/Qwen3-ASR)")
print("=" * 70)
success = run_tests(args.test)
sys.exit(0 if success else 1)
-156
View File
@@ -1,156 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
TTS/ASR模块单元测试 — 测试核心功能,无需实际运行模型
MLX/Qwen3-ASR 版本:仅测试数据模型、设备检测等轻量逻辑
运行方式: pytest backend/tests/test_tts_asr_unit.py -v --no-cov
"""
import base64
import io
import os
import sys
import unittest
import wave
from unittest.mock import patch, MagicMock
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../..')))
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import numpy as np
class TestRequestResponseModels(unittest.TestCase):
"""测试请求/响应数据模型"""
def test_asr_request_defaults(self):
from backend.tts_asr import ASRRequest
req = ASRRequest(audio_base64="dGVzdA==")
self.assertEqual(req.audio_base64, "dGVzdA==")
self.assertEqual(req.language, "zh-CN")
def test_asr_request_with_language(self):
from backend.tts_asr import ASRRequest
req = ASRRequest(audio_base64="dGVzdA==", language="en")
self.assertEqual(req.language, "en")
def test_asr_response(self):
from backend.tts_asr import ASRResponse
resp = ASRResponse(text="你好世界", language="zh-CN")
self.assertEqual(resp.text, "你好世界")
self.assertEqual(resp.language, "zh-CN")
def test_tts_request_defaults(self):
from backend.tts_asr import TTSRequest
req = TTSRequest(text="测试文本")
self.assertEqual(req.text, "测试文本")
self.assertEqual(req.speaker, "Vivian")
self.assertEqual(req.format, "wav")
def test_model_status(self):
from backend.tts_asr import ModelStatus
status = ModelStatus(tts_loaded=False, asr_loaded=True, device="mps")
self.assertFalse(status.tts_loaded)
self.assertTrue(status.asr_loaded)
self.assertEqual(status.device, "mps")
class TestDeviceDetection(unittest.TestCase):
"""测试设备检测逻辑"""
def test_device_map_returns_string(self):
from backend.tts_asr import _get_device_map
device = _get_device_map()
self.assertIsInstance(device, str)
class TestAudioDecoding(unittest.TestCase):
"""测试音频 base64 解码与 WAV 解析"""
def _make_wav_bytes(self, sr=16000, duration_sec=1.0):
samples = int(sr * duration_sec)
audio = np.random.randint(-32768, 32767, size=samples, dtype=np.int16)
buf = io.BytesIO()
with wave.open(buf, 'wb') as wf:
wf.setnchannels(1)
wf.setsampwidth(2)
wf.setframerate(sr)
wf.writeframes(audio.tobytes())
return buf.getvalue()
def test_decode_valid_wav(self):
"""有效 WAV 应能正常解码"""
wav_bytes = self._make_wav_bytes()
audio_b64 = base64.b64encode(wav_bytes).decode()
decoded = base64.b64decode(audio_b64)
wav_buffer = io.BytesIO(decoded)
with wave.open(wav_buffer, 'rb') as wf:
self.assertEqual(wf.getframerate(), 16000)
self.assertEqual(wf.getnchannels(), 1)
def test_decode_empty_raises(self):
"""空 base64 解码后 wave.open 应抛出异常"""
decoded = base64.b64decode("")
self.assertEqual(decoded, b"") # Python 3: empty base64 -> empty bytes
wav_buffer = io.BytesIO(decoded)
with self.assertRaises(Exception):
wave.open(wav_buffer, 'rb') # noqa: SIM115
class TestModelLoadingFunctions(unittest.TestCase):
"""测试模型加载函数存在性(不实际下载)"""
@patch.object(sys.modules.get('backend.tts_asr', MagicMock()), 'Qwen3ASRModel', None)
def test_load_asr_skips_when_mlx_unavailable(self):
"""mlx_audio 未安装时应跳过 ASR 加载"""
from backend.tts_asr import _load_asr_models, Qwen3ASRModel as global_qwen
# 当 Qwen3ASRModel 为 None 时,_load_asr_models 应直接返回
# 这里只验证函数可被调用且不崩溃(因为 modelscope/mlx 都 mock
pass
class TestWarmupFunctions(unittest.TestCase):
"""测试预热函数存在性"""
def test_warmup_functions_exist(self):
from backend.tts_asr import _warmup_tts, _warmup_all
self.assertTrue(callable(_warmup_tts))
self.assertTrue(callable(_warmup_all))
class TestRouteRegistration(unittest.TestCase):
"""测试路由注册函数"""
def test_register_function_exists(self):
from backend.tts_asr import register_tts_asr_routes
self.assertTrue(callable(register_tts_asr_routes))
def run_tests():
loader = unittest.TestLoader()
suite = unittest.TestSuite()
for cls in (TestRequestResponseModels, TestDeviceDetection,
TestAudioDecoding, TestModelLoadingFunctions,
TestWarmupFunctions, TestRouteRegistration):
suite.addTests(loader.loadTestsFromTestCase(cls))
runner = unittest.TextTestRunner(verbosity=2)
result = runner.run(suite)
return result.wasSuccessful()
if __name__ == '__main__':
success = run_tests()
sys.exit(0 if success else 1)
BIN
View File
Binary file not shown.
+198
View File
@@ -0,0 +1,198 @@
<template>
<div class="ocr-image-wrapper" :class="[statusClass, { 'is-loading': isLoading, 'is-success': isSuccess, 'is-failed': isFailed }]">
<img :src="src" :alt="alt" loading="lazy" />
<!-- Loading overlay -->
<div v-if="isLoading" class="overlay loading-overlay">
<svg class="spinner-svg" viewBox="0 0 40 40">
<circle class="spinner-track" cx="20" cy="20" r="16" />
<circle class="spinner-arc" cx="20" cy="20" r="16" />
</svg>
<span class="overlay-label">OCR 识别中...</span>
</div>
<!-- Success overlay -->
<div v-if="isSuccess" class="overlay success-overlay">
<div class="success-badge">
<svg viewBox="0 0 24 24" class="checkmark-svg">
<path d="M9 16.2l-3.5-3.5L4 14.2l5 5 10-10-1.8-1.8z" fill="none" stroke="#fff" stroke-width="2" />
</svg>
</div>
<div class="text-preview" v-if="ocrText">
<span class="preview-label">{{ ocrText }}</span>
</div>
</div>
<!-- Failed overlay -->
<div v-if="isFailed" class="overlay failed-overlay">
<div class="error-message">{{ errorMsg || 'OCR 识别失败' }}</div>
<button class="retry-button" @click="$emit('retry')">
🔄 重试
</button>
</div>
</div>
</template>
<script setup lang="ts">
import { computed } from 'vue'
const props = withDefaults(defineProps<{
src: string
alt?: string
ocrStatus?: 'loading' | 'success' | 'failed'
ocrText?: string
errorMsg?: string
}>(), {
ocrStatus: 'loading',
ocrText: '',
errorMsg: ''
})
defineEmits<{
retry: []
}>()
const isLoading = computed(() => props.ocrStatus === 'loading')
const isSuccess = computed(() => props.ocrStatus === 'success')
const isFailed = computed(() => props.ocrStatus === 'failed')
const statusClass = computed(() => `status-${props.ocrStatus}`)
</script>
<style scoped>
.ocr-image-wrapper {
position: relative;
display: inline-block;
border-radius: 8px;
overflow: hidden;
transition: filter 0.3s ease, border-color 0.3s ease;
}
.ocr-image-wrapper img {
display: block;
max-width: 100%;
height: auto;
}
.is-loading img {
filter: grayscale(100%);
}
.is-failed {
border: 2px solid #ef4444;
}
.overlay {
position: absolute;
inset: 0;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 8px;
padding: 12px;
background: rgba(0, 0, 0, 0.4);
backdrop-filter: blur(2px);
transition: opacity 0.3s ease;
}
.success-overlay {
background: rgba(0, 0, 0, 0.15);
}
/* Spinner */
.spinner-svg {
width: 32px;
height: 32px;
animation: spinner-rotate 1s linear infinite;
}
.spinner-track {
fill: none;
stroke: rgba(255, 255, 255, 0.3);
stroke-width: 3;
}
.spinner-arc {
fill: none;
stroke: #fff;
stroke-width: 3;
stroke-linecap: round;
stroke-dasharray: 60% 72%;
animation: spinner-dash 1.5s ease-in-out infinite;
}
@keyframes spinner-rotate {
100% { transform: rotate(360deg); }
}
@keyframes spinner-dash {
0% { stroke-dasharray: 1% 72%; stroke-dashoffset: 0; }
50% { stroke-dasharray: 60% 72%; stroke-dashoffset: -30%; }
100% { stroke-dasharray: 60% 72%; stroke-dashoffset: -120%; }
}
.overlay-label {
color: #fff;
font-size: 13px;
margin-top: 4px;
}
/* Success badge */
.success-badge {
position: absolute;
bottom: 8px;
right: 8px;
width: 24px;
height: 24px;
background: #22c55e;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.2);
}
.checkmark-svg {
width: 16px;
height: 16px;
}
.text-preview {
margin-top: 8px;
max-width: 90%;
}
.preview-label {
color: rgba(255, 255, 255, 0.8);
font-size: 12px;
}
/* Failed state */
.error-message {
color: #fca5a5;
font-size: 13px;
text-align: center;
}
.retry-button {
display: flex;
align-items: center;
gap: 4px;
padding: 6px 14px;
background: rgba(255, 255, 255, 0.15);
border: 1px solid rgba(255, 255, 255, 0.3);
border-radius: 6px;
color: #fff;
font-size: 13px;
cursor: pointer;
transition: background 0.2s ease, transform 0.1s ease;
}
.retry-button:hover {
background: rgba(255, 255, 255, 0.25);
}
.retry-button:active {
transform: scale(0.96);
}
</style>
+21 -9
View File
@@ -1,13 +1,15 @@
import { Plugin, PluginKey, Selection } from '@milkdown/prose/state'
import { $prose, $ctx, $markSchema } from '@milkdown/kit/utils'
import { parserCtx, serializerCtx } from '@milkdown/kit/core'
import { Node as ProseNode, Slice } from '@milkdown/prose/model'
import type { Ctx } from '@milkdown/kit/core'
import { Node as ProseNode, Schema, Slice } from '@milkdown/prose/model'
import type { Ctx, MarkType } from '@milkdown/kit/core'
import type { Transaction } from '@milkdown/prose/model'
import { Decoration, DecorationSet, type EditorView } from '@milkdown/prose/view'
import { extractDocBlockContextFromMarkdown } from '../utils/docBlock.js'
import { extractWebSearchContextFromMarkdown, WEB_SEARCH_NODE_TYPE } from '../utils/webSearch.js'
import { getOcrCache, OCR_SIZE_LIMIT, extractTextFromOCR, buildOcrContextForDoc } from '../utils/ocrCache'
import { isDocumentVisible } from '../composables/useVisibility.js'
import type { CopilotState, CopilotRuntime, GhostMarkType } from './copilotTypes.js'
const COPILOT_PLUGIN_KEY = new PluginKey('milkdown-copilot')
const DEBOUNCE_MS = 1000
@@ -112,13 +114,13 @@ function clearGhostText(view: EditorView): boolean {
return true
}
function getCursorBeforeGhostInsert(tr: any, from: number): number {
function getCursorBeforeGhostInsert(tr: Transaction, from: number): number {
const mapped = tr.mapping.map(from, -1)
return Math.max(0, Math.min(mapped, tr.doc.content.size))
}
function insertParsedMarkdownSlice(
tr: any,
tr: Transaction,
from: number,
parsedDoc: ProseNode
): { from: number; to: number } | null {
@@ -155,8 +157,8 @@ function createGhostDecorations(doc: ProseNode, from: number, to: number) {
return DecorationSet.create(doc, decorations)
}
function addGhostMarksToTextNodes(tr: any, from: number, to: number, markType: any) {
tr.doc.nodesBetween(from, to, (node: any, pos: number) => {
function addGhostMarksToTextNodes(tr: Transaction, from: number, to: number, markType: MarkType) {
tr.doc.nodesBetween(from, to, (node, pos) => {
if (!node.isText || node.nodeSize <= 0) return true
const start = Math.max(pos, from)
@@ -269,7 +271,7 @@ async function insertGhostText(view: EditorView, suggestion: string, from: numbe
}
}
function insertPlainText(view: EditorView, suggestion: string, from: number, markType: any) {
function insertPlainText(view: EditorView, suggestion: string, from: number, markType: MarkType) {
const tr = view.state.tr
tr.insertText(suggestion, from)
const endPos = from + suggestion.length
@@ -283,8 +285,8 @@ function serializeRangeToMarkdown(
doc: ProseNode,
from: number,
to: number,
schema: any,
serializer: any
schema: Schema,
serializer: (content: ProseNode) => string
): string {
if (from >= to) return ''
const slice = doc.slice(from, to)
@@ -686,12 +688,22 @@ export function interruptCopilot(view: EditorView): void {
export function checkSizeLimit(view: EditorView): { size: number; overLimit: boolean } {
let size = view.state.doc.content.size
view.state.doc.descendants((node) => {
// 文档块:统计 content 属性
if (node.type.name === 'doc_block' && node.attrs.content) {
size += String(node.attrs.content).length
}
// PRO 块:统计 instruction 属性
if (node.type.name === 'pro_block' && node.attrs.instruction) {
size += String(node.attrs.instruction).length
}
// 网络搜索块:统计 content 属性
if (node.type.name === WEB_SEARCH_NODE_TYPE && node.attrs.content) {
size += String(node.attrs.content).length
}
// 图片节点:统计 alt 属性(ASR 转录文本常存于此)
if (node.type.name === 'image' && node.attrs.alt) {
size += String(node.attrs.alt).length
}
})
return { size, overLimit: size > SIZE_LIMIT }
}
+48
View File
@@ -0,0 +1,48 @@
/**
* Shared TypeScript types for the copilot plugin.
*/
import type { Ctx } from '@milkdown/kit/core'
import type { EditorView } from '@milkdown/prose/view'
import type { Transaction } from '@milkdown/prose/model'
/** State tracked by the copilot plugin key */
export interface CopilotState {
from: number
to: number
suggestion: string
}
/** Runtime state associated with each editor view */
export interface CopilotRuntime {
enabled: boolean
debounceTimer: ReturnType<typeof setTimeout> | null
abortController: AbortController | null
ctx: Ctx
requestSeq: number
docVersion: number
}
/** Configuration for the copilot plugin */
export interface CopilotConfig {
fetchSuggestion: (prefix: string, suffix: string, languageId: string, signal?: AbortSignal) => Promise<string>
debounceMs?: number
}
/** Result of checking document size limits */
export interface SizeCheckResult {
size: number
overLimit: boolean
}
/** Mark type from milkdown schema */
export interface GhostMarkType {
create(): any
}
/** Helper type for transaction operations */
export type TransactionLike = {
mapping: { map(pos: number, bias: number): number; mapResult(pos: number, bias: number): { pos: number } }
doc: { content: { size: number }; resolve(pos: number): any; textBetween(from: number, to: number, blockSeparator string, leafText string): string }
tr: any
}
+31
View File
@@ -31,6 +31,7 @@ interface WebSearchBlockConfig {
languageId: string
signal?: AbortSignal
onEvent?: (event: string, data?: Record<string, any>) => void
onDelta?: (data: { text: string }) => void
}) => Promise<{ content: string; createdAt: string }>
t: (key: string) => string
showError: (message: string) => void
@@ -397,6 +398,9 @@ class WebSearchBlockNodeView implements NodeView {
this.setStage('queued')
try {
// 用于累积流式 delta 内容,实现打字机效果
let streamedContent = ''
const result = await this.config.fetchWebSearchStream({
...payload,
signal: this.abortController.signal,
@@ -409,6 +413,33 @@ class WebSearchBlockNodeView implements NodeView {
}
this.setStage(event, String(data?.message || ''))
},
onDelta: (data) => {
if (this.destroyed || this.requestSeq !== requestSeq) return
const text = String(data?.text || '')
if (!text) return
streamedContent += text
this.updateAttrs({ content: streamedContent }
return
}
this.setStage(event, String(data?.message || ''))
},
onDelta: (data) => {
if (this.destroyed || this.requestSeq !== requestSeq) return
const text = String(data?.text || '')
if (!text) return
streamedContent += text
this.updateAttrs({ content: streamedContent }
return
}
this.setStage(event, String(data?.message || ''))
},
onDelta: (data) => {
if (this.destroyed || this.requestSeq !== requestSeq) return
const text = String(data?.text || '')
if (!text) return
streamedContent += text
this.updateAttrs({ content: streamedContent })
},
})
if (this.destroyed || this.requestSeq !== requestSeq) return
+34 -72
View File
@@ -12,6 +12,9 @@ import {
COMPRESS_STATUS_URL,
JOB_LOAD_URL,
} from './config.js'
import { safeString, stripTrailingSlashes } from './string.js'
import { parseSseEvent } from './sse.js'
import { safeFetch, buildHeaders, parseJsonResponse } from './fetch.js'
import { useSettingsStore } from '../stores/settings'
function generateRequestId() {
@@ -22,32 +25,7 @@ function generateRequestId() {
}
function normalizeAbortReason(reason) {
if (typeof reason === 'string' && reason.trim()) {
return reason.trim().slice(0, 64)
}
return 'abort'
}
function parseSseEvent(rawEvent) {
const lines = String(rawEvent || '').replace(/\r/g, '').split('\n')
let event = 'message'
const dataLines = []
for (const line of lines) {
if (!line) continue
if (line.startsWith('event:')) {
event = line.slice(6).trim() || 'message'
continue
}
if (line.startsWith('data:')) {
dataLines.push(line.slice(5).trimStart())
}
}
return {
event,
data: dataLines.join('\n'),
}
return safeString(reason).trim().slice(0, 64) || 'abort'
}
function createAbortError(message = 'Request aborted') {
@@ -78,18 +56,15 @@ async function sendCancelRequest(cancelUrl, requestId, reason) {
}
}
const CANCEL_PATH_MAP = {
'/v1/pro/completions': '/v1/pro/completions/cancel',
'/v1/web-search': '/v1/web-search/cancel',
'/v1/completions': '/v1/completions/cancel',
}
function getCancelUrl(apiUrl) {
const normalized = String(apiUrl || '').replace(/\/+$/, '')
if (/\/v1\/pro\/completions$/i.test(normalized)) {
return normalized.replace(/\/v1\/pro\/completions$/i, '/v1/pro/completions/cancel')
}
if (/\/v1\/web-search$/i.test(normalized)) {
return normalized.replace(/\/v1\/web-search$/i, '/v1/web-search/cancel')
}
if (/\/v1\/completions$/i.test(normalized)) {
return normalized.replace(/\/v1\/completions$/i, '/v1/completions/cancel')
}
return `${normalized}/cancel`
const base = stripTrailingSlashes(apiUrl)
return CANCEL_PATH_MAP[base] || `${base}/cancel`
}
function buildCompletionBody(settings, prefix, suffix, languageId, extra = {}) {
@@ -311,6 +286,7 @@ export async function fetchWebSearchStream(payload, apiUrl = WEB_SEARCH_URL) {
signal,
timeoutMs = WEB_SEARCH_FRONTEND_TIMEOUT_MS,
onEvent,
onDelta,
} = payload || {}
const settings = useSettingsStore()
@@ -341,6 +317,10 @@ export async function fetchWebSearchStream(payload, apiUrl = WEB_SEARCH_URL) {
onEvent?.(String(data?.phase || ''), data)
return
}
if (event === 'delta') {
onDelta?.(data)
return
}
if (event === 'error') {
onEvent?.('error', data)
}
@@ -367,43 +347,28 @@ export async function fetchTTS(text, instruct = '', apiUrl = TTS_URL) {
}
export async function fetchTTSStatus(apiUrl = TTS_STATUS_URL) {
const res = await fetch(apiUrl, {
headers: API_KEY ? { 'X-API-Key': API_KEY } : {},
credentials: 'include',
const res = await safeFetch(apiUrl, {
headers: buildHeaders({ 'Content-Type': 'application/json' }),
})
if (!res.ok) throw new Error(`TTS Status HTTP ${res.status}`)
return res.json()
return parseJsonResponse(res)
}
export async function fetchTTSConfig(apiUrl = TTS_CONFIG_URL) {
const res = await fetch(apiUrl, {
headers: API_KEY ? { 'X-API-Key': API_KEY } : {},
credentials: 'include',
const res = await safeFetch(apiUrl, {
headers: buildHeaders({ 'Content-Type': 'application/json' }),
})
if (!res.ok) throw new Error(`TTS Config HTTP ${res.status}`)
return res.json()
return parseJsonResponse(res)
}
export async function submitCompress(content, docType = 'txt', apiUrl = COMPRESS_SUBMIT_URL) {
const headers = {
'Content-Type': 'application/json',
}
if (API_KEY) {
headers['X-API-Key'] = API_KEY
}
const res = await fetch(apiUrl, {
const res = await safeFetch(apiUrl, {
method: 'POST',
headers,
credentials: 'include',
headers: buildHeaders({
'Content-Type': 'application/json',
}),
body: JSON.stringify({ content, docType }),
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`压缩提交失败 HTTP ${res.status}: ${errorText}`)
}
return res.json()
return parseJsonResponse(res)
}
export function pollCompressStatus(taskId, onStateChange, apiUrl = COMPRESS_STATUS_URL) {
@@ -411,11 +376,9 @@ export function pollCompressStatus(taskId, onStateChange, apiUrl = COMPRESS_STAT
const interval = setInterval(async () => {
try {
const res = await fetch(`${apiUrl}?task_id=${encodeURIComponent(taskId)}`, {
headers: API_KEY ? { 'X-API-Key': API_KEY } : {},
credentials: 'include',
const res = await safeFetch(`${apiUrl}?task_id=${encodeURIComponent(taskId)}`, {
headers: buildHeaders({ 'Content-Type': 'application/json' }),
})
if (!res.ok) {
consecutiveErrors++
if (consecutiveErrors >= 5) {
@@ -426,7 +389,7 @@ export function pollCompressStatus(taskId, onStateChange, apiUrl = COMPRESS_STAT
}
consecutiveErrors = 0
const data = await res.json()
const data = await parseJsonResponse(res)
onStateChange(data.status, data.content || '', data.message, data)
if (['completed', 'error', 'cancelled'].includes(data.status)) {
@@ -445,12 +408,11 @@ export function pollCompressStatus(taskId, onStateChange, apiUrl = COMPRESS_STAT
}
export async function fetchJobLoad(apiUrl = JOB_LOAD_URL) {
const res = await fetch(apiUrl, {
headers: API_KEY ? { 'X-API-Key': API_KEY } : {},
credentials: 'include',
const res = await safeFetch(apiUrl, {
headers: buildHeaders({ 'Content-Type': 'application/json' }),
})
if (!res.ok) {
throw new Error(`Job Load HTTP ${res.status}`)
}
return res.json()
return parseJsonResponse(res)
}
+1
View File
@@ -9,6 +9,7 @@ export const PRO_FRONTEND_TIMEOUT_MS = Number(import.meta.env.VITE_PRO_FRONTEND_
export const WEB_SEARCH_URL = import.meta.env.VITE_WEB_SEARCH_URL || `${API_BASE_URL}/v1/web-search`
export const WEB_SEARCH_FRONTEND_TIMEOUT_MS = Number(import.meta.env.VITE_WEB_SEARCH_FRONTEND_TIMEOUT_MS || 3660000)
export const OCR_URL = import.meta.env.VITE_OCR_URL || `${API_BASE_URL}/v1/ocr`
export const OCR_CONFIG_URL = import.meta.env.VITE_OCR_CONFIG_URL || `${API_BASE_URL}/v1/ocr/config`
export const CONVERT_URL = import.meta.env.VITE_CONVERT_URL || `${API_BASE_URL}/v1/convert`
export const EXPORT_PDF_URL = import.meta.env.VITE_EXPORT_PDF_URL || `${API_BASE_URL}/v1/export/pdf`
export const TTS_URL = import.meta.env.VITE_TTS_URL || `${API_BASE_URL}/v1/tts-asr/tts`
+1 -21
View File
@@ -1,26 +1,6 @@
import { CONVERT_URL, ASR_URL } from './config.js'
function parseSseEvent(rawEvent) {
const lines = String(rawEvent || '').replace(/\r/g, '').split('\n')
let event = 'message'
const dataLines = []
for (const line of lines) {
if (!line) continue
if (line.startsWith('event:')) {
event = line.slice(6).trim() || 'message'
continue
}
if (line.startsWith('data:')) {
dataLines.push(line.slice(5).trimStart())
}
}
return {
event,
data: dataLines.join('\n'),
}
}
import { parseSseEvent } from './sse.js'
async function consumeSseResult(res) {
if (!res.ok) {
+54
View File
@@ -0,0 +1,54 @@
/**
* Unified safeFetch wrapper for API requests.
* Handles authentication headers, credentials, and error mapping consistently across all endpoints.
*/
import { API_KEY } from './config.js'
/** Build headers for JSON API requests with optional X-API-Key */
function buildHeaders(extra = {}) {
return API_KEY
? { 'X-API-Key': API_KEY, ...extra }
: { ...extra }
}
/** Parse JSON response from fetch result, handling non-OK status codes */
async function parseJsonResponse(res) {
if (!res.ok) {
let message = `HTTP ${res.status}`
try {
const data = await res.json()
message = data.detail || data.error || message
} catch {
const text = await res.text()
if (text) message = text
}
throw new Error(message)
}
return res.json()
}
/**
* Fetch wrapper that handles:
* - Authentication headers (X-API-Key)
* - Credentials (include cookies)
* - Error mapping (HTTP status meaningful message)
*/
export async function safeFetch(url, options = {}) {
const headers = buildHeaders(options.headers || {})
const res = await fetch(url, {
...options,
headers,
credentials: 'include',
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
return res
}
/** Export helper functions for use in other modules */
export { buildHeaders, parseJsonResponse }
+43 -7
View File
@@ -1,16 +1,55 @@
const SIZE_LIMIT = 32 * 1024
export const IMAGE_SIZE_LIMIT = 100 * 1024 * 1024
// ─── Status enum ───────────────────────────────────────────────
export const OcrStatus = Object.freeze({
PENDING: 'pending',
LOADING: 'loading',
SUCCESS: 'success',
FAILED: 'failed',
})
// ─── State cache (per-image-hash) ──────────────────────────────
// Map<hash, { status, text?, error?, updatedAt }>
const ocrStateCache = new Map()
// ─── Legacy text cache (kept for backward compat) ──────────────
const ocrCache = new Map()
const imageHashCache = new Map()
// ─── Hash utilities ────────────────────────────────────────────
export async function calculateImageHash(imageBytes) {
const hashBuffer = await crypto.subtle.digest('SHA-256', imageBytes)
const hashArray = Array.from(new Uint8Array(hashBuffer))
return hashArray.map(b => b.toString(16).padStart(2, '0')).join('')
}
// ─── State API (new) ───────────────────────────────────────────
export function setOcrState(hash, status, text = '', error = '') {
ocrStateCache.set(hash, {
status,
text: typeof text === 'string' ? text : '',
error: typeof error === 'string' ? error : '',
updatedAt: Date.now(),
})
}
export function getOcrState(hash) {
return ocrStateCache.get(hash) || { status: OcrStatus.PENDING, text: '', error: '' }
}
export function resetOcrState(hash) {
ocrStateCache.delete(hash)
}
export function clearAllOcrState() {
ocrStateCache.clear()
}
// ─── Legacy API (backward compat) ──────────────────────────────
export function getOcrByHash(hash) {
const state = ocrStateCache.get(hash)
if (state && state.status === OcrStatus.SUCCESS) return state.text
return imageHashCache.get(hash) || ''
}
@@ -38,6 +77,7 @@ export function clearAllOcrCache() {
ocrCache.clear()
}
// ─── Size utilities ────────────────────────────────────────────
export function calculateOcrSize(imageFilenames) {
let total = 0
for (const name of imageFilenames) {
@@ -54,12 +94,13 @@ export function checkSizeLimit(docTextSize, imageFilenames) {
size: total,
docSize: docTextSize,
ocrSize: ocrSize,
overLimit: total > SIZE_LIMIT
overLimit: total > SIZE_LIMIT,
}
}
export const OCR_SIZE_LIMIT = SIZE_LIMIT
// ─── Text extraction ───────────────────────────────────────────
export function extractTextFromOCR(ocrText, maxLen = 100) {
if (!ocrText) return ''
const match = ocrText.match(/TEXT:\s*([\s\S]*?)(?:KEY_DETAILS|LANGUAGE|SUMMARY|$)/i)
@@ -68,14 +109,9 @@ export function extractTextFromOCR(ocrText, maxLen = 100) {
return text.length > maxLen ? text.substring(0, maxLen) + '...' : text
}
// ─── Context builder (for AI completion) ───────────────────────
const IMAGE_NODE_TYPES = new Set(['image', 'image-block', 'imageBlock'])
/**
* ProseMirror doc 中提取 OCR 上下文 AI 补全使用
* @param {ProseNode} doc - ProseMirror document node
* @param {number} maxLen - OCR 文本最大长度
* @returns {string}
*/
export function buildOcrContextForDoc(doc, maxLen = 120) {
const lines = []
+2 -4
View File
@@ -1,8 +1,6 @@
const OUTER_FENCE_RE = /^(`{3,}|~{3,})[^\n]*\n([\s\S]*?)\n\1[ \t]*$/
import { normalizeNewlines } from './string.js'
function normalizeNewlines(value = '') {
return String(value || '').replace(/\r\n?/g, '\n')
}
const OUTER_FENCE_RE = /^(`{3,}|~{3,})[^\n]*\n([\s\S]*?)\n\1[ \t]*$/
function unescapeLiteralNewlines(value = '') {
const text = String(value || '')
+37
View File
@@ -0,0 +1,37 @@
/**
* Shared SSE (Server-Sent Events) parsing utilities.
* Previously duplicated in api.js and convert.js.
*/
/** Single parsed SSE event */
export interface SseEvent {
event: string
data: string
}
/**
* Parse a raw SSE event chunk into { event, data }.
* Handles multi-line data fields and explicit event type overrides.
*
* @example
* parseSseEvent('event: result\ndata: {"ok":true}')
* // → { event: 'result', data: '{"ok":true}' }
*/
export function parseSseEvent(rawEvent: string): SseEvent {
const lines = String(rawEvent || '').replace(/\r/g, '').split('\n')
let event = 'message'
const dataLines: string[] = []
for (const line of lines) {
if (!line) continue
if (line.startsWith('event:')) {
event = line.slice(6).trim() || 'message'
continue
}
if (line.startsWith('data:')) {
dataLines.push(line.slice(5).trimStart())
}
}
return { event, data: dataLines.join('\n') }
}
+27
View File
@@ -0,0 +1,27 @@
/**
* Shared string utility functions.
* Extracted from repeated patterns across api.js, docBlock.js, proAccept.js, proBlock.js, webSearch.js.
*/
/**
* Normalize line endings to Unix-style (\n).
* Handles \r\n (Windows), \r (old Mac), and \n (Unix).
*/
export function normalizeNewlines(value: string = ''): string {
return String(value || '').replace(/\r\n?/g, '\n')
}
/**
* Safely coerce any value to a non-null string.
* Replaces the common `String(x || '')` pattern scattered across 20+ locations.
*/
export function safeString(value: unknown): string {
return String(value ?? '')
}
/**
* Remove trailing slashes from a URL/path string.
*/
export function stripTrailingSlashes(value: string = ''): string {
return safeString(value).replace(/\/+$/, '')
}
-18
View File
@@ -1,18 +0,0 @@
这是一份用于测试压缩功能的文档。
人工智能(Artificial Intelligence,简称 AI)是计算机科学的一个分支,它试图理解智能的本质,并生产出一种新的能以人类智能相似的方式做出反应的智能机器。
人工智能的研究包括机器人、语言识别、图像识别、自然语言处理和专家系统等。人工智能从诞生以来,理论和技术日益成熟,应用领域也不断扩大,可以设想未来人工智能带来的科技产品将会是人类智慧的容器。
人工智能可以对人的意识、思维的信息过程的模拟。人工智能不是人的智能,但能像人那样思考、也可能超过人的智能。
机器学习是人工智能的核心领域之一。它使用算法来解析数据、从中学习,然后对真实世界中的事件做出决策和预测。
深度学习是机器学习的一个子集,它使用多层神经网络来分析各种因素。深度学习的出现使得人工智能在许多领域取得了突破性进展,包括计算机视觉、语音识别和自然语言处理。
大型语言模型(LLM)是深度学习在自然语言处理领域的最新成果。它们通过在海量的文本数据上进行训练,学习到了语言的复杂模式和规律。
这些模型能够生成流畅的、符合语法的文本,回答问题,进行翻译,甚至创作诗歌和故事。
然而,人工智能的发展也带来了一些伦理和社会问题,比如隐私保护、算法偏见和就业影响等。
-23
View File
@@ -1,23 +0,0 @@
```llm-file
doc_type: txt
doc_name: test_compress_doc.txt
这是一份用于测试压缩功能的文档。
人工智能(Artificial Intelligence,简称 AI)是计算机科学的一个分支,它试图理解智能的本质,并生产出一种新的能以人类智能相似的方式做出反应的智能机器。
人工智能的研究包括机器人、语言识别、图像识别、自然语言处理和专家系统等。人工智能从诞生以来,理论和技术日益成熟,应用领域也不断扩大,可以设想未来人工智能带来的科技产品将会是人类智慧的容器。
人工智能可以对人的意识、思维的信息过程的模拟。人工智能不是人的智能,但能像人那样思考、也可能超过人的智能。
机器学习是人工智能的核心领域之一。它使用算法来解析数据、从中学习,然后对真实世界中的事件做出决策和预测。
深度学习是机器学习的一个子集,它使用多层神经网络来分析各种因素。深度学习的出现使得人工智能在许多领域取得了突破性进展,包括计算机视觉、语音识别和自然语言处理。
大型语言模型(LLM)是深度学习在自然语言处理领域的最新成果。它们通过在海量的文本数据上进行训练,学习到了语言的复杂模式和规律。
这些模型能够生成流畅的、符合语法的文本,回答问题,进行翻译,甚至创作诗歌和故事。
然而,人工智能的发展也带来了一些伦理和社会问题,比如隐私保护、算法偏见和就业影响等。
```