Migrate backend jobs to Redis Streams

This commit is contained in:
“ydy0615”
2026-06-06 15:44:00 +08:00
parent 2c7a02f587
commit 81f711ef0b
69 changed files with 2709 additions and 6291 deletions
+10
View File
@@ -2,3 +2,13 @@ VITE_API_BASE_URL=
VITE_API_URL=
VITE_OCR_URL=
VITE_CONVERT_URL=
VITE_PRO_URL=
VITE_TTS_URL=
VITE_TTS_STATUS_URL=
VITE_TTS_CONFIG_URL=
VITE_ASR_URL=
VITE_JOB_LOAD_URL=
VITE_PRO_FRONTEND_TIMEOUT_MS=3660000
# Document block compression context limit (characters)
VITE_DOC_COMPRESS_CONTEXT_LIMIT=128000
+23 -1
View File
@@ -7,6 +7,22 @@
- 这是一个智能 Markdown 编辑器,前端负责编辑器 UI、上传导出、补全交互和设置状态,后端负责 LLM、OCR、文件转换和 TTS 接口。
- 前端技术栈:Vue 3 + Vite + Milkdown/Crepe + Pinia + Vue Router。
- 后端技术栈:FastAPI + Python + Ollama。
## 功能块系统(核心概念)
- **统一命名**:文档块、PRO 块、上传块统称为"功能块"。
- **禁止嵌套**:所有功能块的 schema 均设 `atom: true, isolating: true`ProseMirror 层面强制禁止互相嵌套。DocBlockCrepe.vue 的嵌套 Crepe 编辑器不注册任何功能块插件,从架构上杜绝深层嵌套。
- **无数量限制**:一篇文档可包含任意数量的功能块,彼此独立存在。
- **导入自动解析**:从 Markdown 文件导入后,各功能块语法必须通过对应的 Remark/parseMarkdown 解析器自动识别并还原为交互卡片。
- **导出可复原**:通过 toMarkdown/leafText 序列化器将功能块还原为 Markdown 语法,确保导出后再次导入能完整复原。doc_block toMarkdown 输出 legacy HTML tag`getExportMarkdown()` 中通过 `transformLegacyDocBlocksForExport()` 转换为 fenced code block。
| 类型 | Node Type | Markdown 语法 | Plugin 文件 | Utility 文件 |
|------|-----------|---------------|-------------|--------------|
| 文档块 | `doc_block` | \`\`\`llm-file fenced code / `<doc_type=...>` legacy HTML tag | `plugins/docBlockPlugin.ts` | `utils/docBlock.js` |
| PRO 块 | `pro_block` | `[PRO]` / `[PRO]{指令}` | `plugins/proBlockPlugin.ts` | `utils/proBlock.js` |
| 上传块 | `upload_block` | `{{{}}}` / `{{{upload file type:...}}}` | `plugins/uploadBlockPlugin.ts` | `utils/uploadBlock.js` |
- **已验证**:当前代码完全符合"禁止嵌套、自动解析复原"的要求。修改功能块相关逻辑时需验证:1) schema 的 atom/isolating 属性不被移除;2) Remark/parseMarkdown/toMarkdown 解析链路完整。
- 当前代码中可以确认的主功能是:AI 补全、OCR、文档转 Markdown、TTS、Markdown/DOCX/PDF 导入导出。
- 历史文档中有一部分 TTS/ASR、Apple Silicon、Whisper、离线模式说明已经落后于当前代码;出现冲突时以实际代码和测试为准。
@@ -28,7 +44,13 @@
## 稳定事实
- 补全接口当前不是 SSE;前端用普通 POST 请求拿 JSON 响应
- **功能块禁止嵌套**`doc_block``pro_block``upload_block` 的 schema 均设 `atom: true, isolating: true`ProseMirror 层面强制禁止互相嵌套。DocBlockCrepe.vue 的嵌套 Crepe 编辑器仅注册 copilotPlugin + hiddenText*,不注册任何功能块插件。修改时不得移除 atom/isolating 属性或在嵌套编辑器中引入功能块插件
- **功能块导入解析**:各功能块的 Remark 插件(`docBlockRemark`, `proBlockRemark`, `uploadBlockRemark`)负责从 Markdown AST 识别对应语法并转换为节点。doc_block Remark 同时支持 fenced code (`llm-file`) 和 legacy HTML tag (`<doc_type=...>`)。解析链路必须保持完整,否则导入后无法自动复原。
- **功能块导出序列化**doc_block 的 `toMarkdown` runner 输出 legacy HTML tag,但 `getExportMarkdown()` 中通过 `transformLegacyDocBlocksForExport()` 转换为 fenced code block。pro_block/upload_block 的 toMarkdown/leafText 直接输出标准语法。修改时需验证导出后再次导入能完整复原。
- **Store 同步格式**`scheduleMarkdownSync()` emit markdown 不做转换(doc_block 为 legacy HTML tag),store 中始终是 legacy format。`syncInitialMarkdown()` / `getExportMarkdown()` 负责格式转换。
- **上传块生命周期**upload_block 是临时占位符,文件上传后被替换为 image node(图片)或 doc_block(文档),不会与 pro_block 共存。
- **PRO block escape/unescape**`escapeProBlockContent()` / `unescapeProBlockSyntax()` 处理 `\`, `]`, `}`, newline,确保指令 round-trip 正确。
- **补全接口当前不是 SSE**;前端用普通 POST 请求拿 JSON 响应。
- 前端会生成 X-Request-Id,并在请求被中止时额外调用 /v1/completions/cancel。
- 文档超过 32 KB 时,AI 补全会在前端和插件层被禁用。
- OCR 文本和文档块内容会被注入补全上下文,但这些内容属于隐藏上下文,不应被直接当作用户可见文本重复输出。
+21 -5
View File
@@ -16,10 +16,18 @@
- 流式响应,低延迟体验
- 多种交互方式:Tab接受、Esc拒绝、点击接受
### 文档处理
- OCR 图片识别:上传图片自动识别文字
- 文档转换:PDF、DOCX、PPTX、TXT 转 Markdown
- 文档块嵌入:可折叠的文档预览块
### 功能块系统
编辑器提供三种**功能块**,统一为顶层原子节点(`atom: true, isolating: true`),通过 ProseMirror schema 强制禁止互相嵌套,无数量限制。导入 Markdown 后自动解析还原为交互卡片,导出后可完整复原:
| 功能块 | Markdown 语法 | 作用 |
|--------|---------------|------|
| **文档块** (`doc_block`) | \`\`\`llm-file fenced code / `<doc_type=...>` legacy HTML tag | 上传的 PDF/DOCX/PPTX/TXT 等文件以可折叠卡片嵌入编辑器,支持内联编辑和 AI 补全 |
| **PRO 块** (`pro_block`) | `[PRO]` / `[PRO]{指令}` | 基于全文上下文进行深度 AI 思考并流式生成 Markdown`Ctrl+Shift+P` 快速插入 |
| **上传块** (`upload_block`) | `{{{}}}` / `{{{upload file type:pdf,docx}}}` | 文件上传占位符,支持按类型过滤(PDF/DOCX/PPTX/TXT/JSON/YAML/图片等) |
### 文档处理(历史名称,已整合入功能块系统)
- OCR 图片识别:上传图片自动识别文字(OCR 结果注入 AI 补全和 PRO 块上下文)
- 智能大小限制:32KB自动禁用AI
### 设置面板
@@ -37,9 +45,17 @@
## 技术架构
前端: Vue3 + Vite + Milkdown + ProseMirror
前端: Vue3 + Vite + Milkdown/Crepe + ProseMirror
后端: FastAPI + PythonOpenAI 兼容端点)
### 功能块架构
三种功能块统一为顶层原子节点(`atom: true, isolating: true`),通过 ProseMirror schema 强制禁止嵌套:
- **文档块** (`doc_block`) — `src/plugins/docBlockPlugin.ts`Markdown 语法:\`\`\`llm-file fenced code block
- **PRO 块** (`pro_block`) — `src/plugins/proBlockPlugin.ts`Markdown 语法:`[PRO]` / `[PRO]{指令}`
- **上传块** (`upload_block`) — `src/plugins/uploadBlockPlugin.ts`Markdown 语法:`{{{}}}` / `{{{upload file type:...}}}`
每个功能块配备独立的 Remark 解析器和序列化器,确保 Markdown 导入导出时自动识别和还原。
## 快速开始
环境: Node.js 18+、Python 3.8+
+35 -5
View File
@@ -15,12 +15,42 @@ VLM_MODEL=qwen3-vl:30b
# API key for the FastAPI app (change in production)
API_KEY=your-secret-key-here
# PRO completion timeout (seconds)
PRO_COMPLETION_TIMEOUT=1200
# Job backend
JOB_BACKEND=redis
REDIS_URL=redis://localhost:6379/0
JOB_REDIS_PREFIX=llmtext:jobs
JOB_CONSUMER_NAME=
JOB_SHARED_TEMP_DIR=/tmp/llm-in-text-jobs
JOB_STATE_TTL_SECONDS=600
JOB_EVENT_TTL_SECONDS=600
JOB_EVENT_STREAM_MAXLEN=512
JOB_CANCEL_POLL_SECONDS=0.5
JOB_BUSY_NORMAL_THRESHOLD=0.25
JOB_BUSY_HIGH_THRESHOLD=0.75
JOB_BUSY_FULL_THRESHOLD=1.0
# Concurrency limits
STANDARD_CONCURRENCY_LIMIT=5
PRO_CONCURRENCY_LIMIT=20
# Per-queue concurrency and capacity
JOB_COMPLETION_CONCURRENCY=2
JOB_COMPLETION_MAX_QUEUE=16
JOB_PRO_COMPLETION_CONCURRENCY=1
JOB_PRO_COMPLETION_MAX_QUEUE=8
JOB_COMPRESS_CONCURRENCY=1
JOB_COMPRESS_MAX_QUEUE=8
JOB_OCR_CONCURRENCY=1
JOB_OCR_MAX_QUEUE=8
JOB_CONVERT_CONCURRENCY=1
JOB_CONVERT_MAX_QUEUE=8
JOB_TTS_CONCURRENCY=1
JOB_TTS_MAX_QUEUE=4
JOB_ASR_CONCURRENCY=1
JOB_ASR_MAX_QUEUE=4
# Timeouts (seconds)
LLM_COMPLETION_TIMEOUT=600
LLM_OCR_TIMEOUT=600
# Compression limit
DOC_COMPRESS_CONTEXT_LIMIT=128000
# Legacy fallback: if LLM_BASE_URL is not set, OLLAMA_HOST will be auto-converted to /v1/ path
#OLLAMA_HOST=http://localhost:11434
+256
View File
@@ -0,0 +1,256 @@
import asyncio
import os
import re
from contextlib import suppress
from typing import Any, Callable, Awaitable
import markitdown
from llm import call_ollama, call_vlm_ocr, stream_ollama_events
from prompt import (
build_completion_prompts,
build_pro_completion_prompts,
prepare_prompt_context,
)
try: # pragma: no cover - optional heavy dependency path
from tts_asr import generate_asr_response, generate_tts_response
except Exception: # pragma: no cover
generate_tts_response = None
generate_asr_response = None
IMAGE_MARKDOWN_RE = re.compile(r"!\[[^\]]*]\([^)]+\)")
IMAGE_HTML_RE = re.compile(r"<img\b[^>]*>", re.IGNORECASE)
ALLOWED_CONVERT_EXTENSIONS = {".txt", ".docx", ".pptx", ".pdf"}
_markitdown_instance = None
def _get_markitdown():
global _markitdown_instance
if _markitdown_instance is None:
_markitdown_instance = markitdown.MarkItDown()
return _markitdown_instance
def _safe_unlink(path: str | None) -> None:
if not path:
return
with suppress(FileNotFoundError):
os.unlink(path)
def _sanitize_converted_markdown(text: str) -> str:
value = (text or "").replace("\r\n", "\n").replace("\r", "\n")
value = IMAGE_MARKDOWN_RE.sub("", value)
value = IMAGE_HTML_RE.sub("", value)
value = re.sub(r"\n{3,}", "\n\n", value)
return value.strip()
def sanitize_inline_completion_content(text: str, prefill: str = "") -> str:
value = (text or "").strip()
if not value:
return ""
fim_middle = value.rfind("<|fim_middle|>")
if fim_middle >= 0:
value = value[fim_middle + len("<|fim_middle|>") :]
end_index = value.find("<|end|>")
if end_index >= 0:
value = value[:end_index]
quoted = re.findall(r'"([^"]+)"', value)
if quoted:
value = quoted[-1]
marker_index = max(value.rfind("|fim_middle|>"), value.rfind("<|start|>assistant"))
if marker_index >= 0:
tail = value.split(">")[-1]
if tail:
value = tail
value = value.strip()
if prefill and value.startswith(prefill):
value = value[len(prefill) :]
return value.strip()
async def completion_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
req = payload["request"]
system_prompt, user_prompt, prefill = build_completion_prompts(
req["prefix"],
req["suffix"],
req.get("languageId", "markdown"),
location=payload.get("location", ""),
thinking_level=req.get("model_thinking", "low"),
preferences=req.get("user_preferences"),
)
result = await call_ollama(
user_prompt,
system_prompt=system_prompt,
tag=f'{payload["request_id"][:8]}-completion',
temperature=float(req.get("temperature", 0.7)),
thinking=req.get("model_thinking") if req.get("model_thinking") != "none" else None,
model=req.get("model"),
prefill=prefill or None,
)
content = sanitize_inline_completion_content(result.get("content") or "", prefill=prefill or "")
if is_cancelled():
raise asyncio.CancelledError()
await emit("result", {"content": content})
return {"content": content, "request_id": payload["request_id"]}
async def pro_completion_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
req = payload["request"]
system_prompt, user_prompt = build_pro_completion_prompts(
prefix=req["prefix"],
suffix=req["suffix"],
instruction=req.get("instruction", ""),
language_id=req.get("languageId", "markdown"),
location=payload.get("location", ""),
pro_thinking_level=req.get("pro_thinking", "medium"),
preferences=req.get("user_preferences"),
)
chunks: list[str] = []
async for event_type, delta in stream_ollama_events(
user_prompt,
system_prompt=system_prompt,
tag=f'{payload["request_id"][:8]}-pro',
temperature=0.7,
thinking=req.get("pro_thinking", "medium"),
use_pro_model=True,
enable_thinking=True,
):
if is_cancelled():
raise asyncio.CancelledError()
if event_type == "thinking":
await emit("progress", {"phase": "thinking"})
continue
if delta:
chunks.append(delta)
await emit("result", {"delta": delta})
content = "".join(chunks)
return {"content": content, "request_id": payload["request_id"]}
async def compress_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
content = payload["content"]
doc_type = payload.get("docType", "txt")
system_prompt = (
f"你是一个专业的文档摘要助手。请将以下 {doc_type} 类型文档内容进行精简压缩,"
"保留核心信息和关键要点,去除冗余和啰嗦的表述。"
"请直接输出压缩后的内容,不要添加任何解释性文字。"
)
result = await call_ollama(
content,
system_prompt=system_prompt,
tag=f'{payload["request_id"][:8]}-compress',
)
if is_cancelled():
raise asyncio.CancelledError()
compressed = result.get("content") or ""
await emit("result", {"content": compressed})
return {"content": compressed, "request_id": payload["request_id"]}
async def ocr_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
path = payload["input_path"]
try:
with open(path, "rb") as handle:
image_bytes = handle.read()
text = await call_vlm_ocr(image_bytes, payload.get("language", "auto"))
if is_cancelled():
raise asyncio.CancelledError()
await emit("result", {"text": text})
return {"text": text, "filename": payload.get("filename", "image.jpg")}
finally:
_safe_unlink(path)
async def convert_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
path = payload["input_path"]
filename = payload.get("filename", "document")
ext = os.path.splitext(filename)[1].lower()
if ext not in ALLOWED_CONVERT_EXTENSIONS:
_safe_unlink(path)
raise ValueError("仅支持 txt、docx、pptx、pdf 格式")
try:
if ext == ".txt":
with open(path, "rb") as handle:
markdown = _sanitize_converted_markdown(handle.read().decode("utf-8", errors="ignore"))
else:
md = _get_markitdown()
result = await asyncio.to_thread(md.convert, path)
markdown = _sanitize_converted_markdown(result.text_content)
if is_cancelled():
raise asyncio.CancelledError()
await emit("result", {"markdown": markdown})
return {"markdown": markdown, "filename": filename}
finally:
_safe_unlink(path)
async def tts_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
if generate_tts_response is None:
raise RuntimeError("TTS 功能当前不可用")
response = await generate_tts_response(
text=payload["text"],
instruct=payload.get("instruct", ""),
speaker=payload.get("speaker", "Vivian"),
output_format=payload.get("format", "wav"),
)
if is_cancelled():
raise asyncio.CancelledError()
result = response.dict()
await emit("result", result)
return result
async def asr_handler(
payload: dict[str, Any],
emit: Callable[[str, dict[str, Any]], Awaitable[None]],
is_cancelled: Callable[[], bool],
) -> dict[str, Any]:
if generate_asr_response is None:
raise RuntimeError("ASR 功能当前不可用")
path = payload["input_path"]
try:
with open(path, "rb") as handle:
audio_bytes = handle.read()
response = await generate_asr_response(audio_bytes, payload.get("language", "zh-CN"))
if is_cancelled():
raise asyncio.CancelledError()
result = response.dict()
await emit("result", result)
return result
finally:
_safe_unlink(path)
+704
View File
@@ -0,0 +1,704 @@
import asyncio
import inspect
import json
import logging
import os
import tempfile
import time
import uuid
from contextlib import suppress
from dataclasses import dataclass
from pathlib import Path
from typing import Any, AsyncIterator, Awaitable, Callable, Optional
logger = logging.getLogger("job_system")
try: # pragma: no cover - optional dependency in tests
from redis import asyncio as redis_asyncio # type: ignore
except Exception: # pragma: no cover - optional dependency in tests
redis_asyncio = None
TERMINAL_STATUSES = {"completed", "failed", "cancelled"}
TERMINAL_EVENTS = {"done", "error", "cancelled"}
JOB_TYPES = (
"completion",
"pro_completion",
"compress",
"ocr",
"convert",
"tts",
"asr",
)
DEFAULT_CONCURRENCY = {
"completion": 2,
"pro_completion": 1,
"compress": 1,
"ocr": 1,
"convert": 1,
"tts": 1,
"asr": 1,
}
DEFAULT_QUEUE_SIZE = {
"completion": 16,
"pro_completion": 8,
"compress": 8,
"ocr": 8,
"convert": 8,
"tts": 4,
"asr": 4,
}
class JobSystemError(RuntimeError):
pass
class QueueFullError(JobSystemError):
pass
@dataclass(frozen=True)
class QueueConfig:
job_type: str
concurrency: int
max_queue: int
Handler = Callable[[dict[str, Any], Callable[[str, dict[str, Any]], Awaitable[None]], Callable[[], bool]], Awaitable[dict[str, Any]]]
def _bool_env(name: str, default: bool) -> bool:
value = os.getenv(name)
if value is None:
return default
return value.strip().lower() in {"1", "true", "yes", "on"}
def _int_env(name: str, default: int) -> int:
try:
return max(1, int(os.getenv(name, str(default))))
except (TypeError, ValueError):
return default
def _float_env(name: str, default: float) -> float:
try:
return float(os.getenv(name, str(default)))
except (TypeError, ValueError):
return default
def _now_ms() -> int:
return int(time.time() * 1000)
def _busy_level(ratio: float) -> str:
if ratio >= _float_env("JOB_BUSY_FULL_THRESHOLD", 1.0):
return "full"
if ratio >= _float_env("JOB_BUSY_HIGH_THRESHOLD", 0.75):
return "busy"
if ratio >= _float_env("JOB_BUSY_NORMAL_THRESHOLD", 0.25):
return "normal"
return "idle"
def _json_dumps(value: Any) -> str:
return json.dumps(value, ensure_ascii=False)
def _json_loads(value: str | bytes | None, default: Any = None) -> Any:
if value is None:
return default
if isinstance(value, bytes):
value = value.decode("utf-8")
if not value:
return default
return json.loads(value)
def _queue_config(job_type: str) -> QueueConfig:
upper = job_type.upper()
concurrency = _int_env(f"JOB_{upper}_CONCURRENCY", DEFAULT_CONCURRENCY[job_type])
max_queue = _int_env(f"JOB_{upper}_MAX_QUEUE", DEFAULT_QUEUE_SIZE[job_type])
return QueueConfig(job_type=job_type, concurrency=concurrency, max_queue=max_queue)
def get_job_backend_name() -> str:
value = (os.getenv("JOB_BACKEND") or "").strip().lower()
if value:
return value
if redis_asyncio is not None:
return "redis"
return "memory"
def _shared_temp_dir() -> Path:
path = Path(os.getenv("JOB_SHARED_TEMP_DIR", tempfile.gettempdir()) or tempfile.gettempdir())
path.mkdir(parents=True, exist_ok=True)
return path
def persist_temp_input(raw_bytes: bytes, suffix: str) -> str:
directory = _shared_temp_dir()
fd, path = tempfile.mkstemp(prefix="job-input-", suffix=suffix, dir=directory)
os.close(fd)
with open(path, "wb") as handle:
handle.write(raw_bytes)
return path
async def _maybe_await(value: Any) -> Any:
if inspect.isawaitable(value):
return await value
return value
class BaseJobManager:
def __init__(self) -> None:
self.handlers: dict[str, Handler] = {}
def register_handler(self, job_type: str, handler: Handler) -> None:
self.handlers[job_type] = handler
async def submit(self, job_type: str, payload: dict[str, Any], request_id: str | None = None) -> str:
raise NotImplementedError
async def cancel(self, job_id: str, reason: str = "abort") -> dict[str, Any]:
raise NotImplementedError
async def get_status(self, job_id: str) -> dict[str, Any] | None:
raise NotImplementedError
async def stream_events(self, job_id: str) -> AsyncIterator[dict[str, Any]]:
raise NotImplementedError
async def close(self) -> None:
return None
class InMemoryJobManager(BaseJobManager):
def __init__(self) -> None:
super().__init__()
self.jobs: dict[str, dict[str, Any]] = {}
self.event_history: dict[str, list[dict[str, Any]]] = {}
self.subscribers: dict[str, list[asyncio.Queue]] = {}
self.queues = {job_type: asyncio.Queue() for job_type in JOB_TYPES}
self.semaphores = {job_type: asyncio.Semaphore(_queue_config(job_type).concurrency) for job_type in JOB_TYPES}
self.queue_counts = {job_type: 0 for job_type in JOB_TYPES}
self.running_counts = {job_type: 0 for job_type in JOB_TYPES}
self.running_tasks: dict[str, asyncio.Task] = {}
self.worker_tasks: list[asyncio.Task] = []
self.started = False
self.lock = asyncio.Lock()
async def _ensure_started(self) -> None:
if self.started:
return
self.started = True
for job_type in JOB_TYPES:
self.worker_tasks.append(asyncio.create_task(self._worker_loop(job_type)))
def _metrics(self, job_type: str) -> dict[str, Any]:
config = _queue_config(job_type)
queued = self.queue_counts[job_type]
running = self.running_counts[job_type]
capacity = max(config.max_queue + config.concurrency, 1)
ratio = min((queued + running) / capacity, 1.0)
return {
"queue_position": queued if queued > 0 else 0,
"queued_count": queued,
"running_count": running,
"concurrency_limit": config.concurrency,
"max_queue": config.max_queue,
"busy_ratio": round(ratio, 4),
"busy_level": _busy_level(ratio),
}
async def _publish(self, job_id: str, event: str, data: dict[str, Any]) -> None:
event_payload = {"event": event, **data}
self.event_history.setdefault(job_id, []).append(event_payload)
for queue in self.subscribers.get(job_id, []):
await queue.put(event_payload)
async def submit(self, job_type: str, payload: dict[str, Any], request_id: str | None = None) -> str:
await self._ensure_started()
if job_type not in self.handlers:
raise JobSystemError(f"missing handler for job type: {job_type}")
async with self.lock:
config = _queue_config(job_type)
if self.queue_counts[job_type] >= config.max_queue:
raise QueueFullError(f"{job_type} queue is full")
job_id = request_id or str(uuid.uuid4())
self.jobs[job_id] = {
"job_id": job_id,
"request_id": job_id,
"job_type": job_type,
"status": "queued",
"payload": payload,
"result": None,
"error": "",
"cancel_requested": False,
"created_at": _now_ms(),
"updated_at": _now_ms(),
}
self.event_history[job_id] = []
self.queue_counts[job_type] += 1
metrics = self._metrics(job_type)
await self._publish(job_id, "queued", {"job_id": job_id, "type": job_type, "status": "queued", **metrics})
await self.queues[job_type].put(job_id)
return job_id
async def cancel(self, job_id: str, reason: str = "abort") -> dict[str, Any]:
async with self.lock:
job = self.jobs.get(job_id)
if not job:
return {"cancelled": False, "status": "not_found"}
if job["status"] in TERMINAL_STATUSES:
return {"cancelled": False, "status": job["status"]}
job["cancel_requested"] = True
job["updated_at"] = _now_ms()
task = self.running_tasks.get(job_id)
if task and not task.done():
task.cancel()
if job["status"] == "queued":
job["status"] = "cancelled"
self.queue_counts[job["job_type"]] = max(0, self.queue_counts[job["job_type"]] - 1)
metrics = self._metrics(job["job_type"])
else:
job["status"] = "cancelled"
metrics = self._metrics(job["job_type"])
await self._publish(job_id, "cancelled", {"job_id": job_id, "type": job["job_type"], "status": "cancelled", "reason": reason, **metrics})
return {"cancelled": True, "status": "ok"}
async def get_status(self, job_id: str) -> dict[str, Any] | None:
job = self.jobs.get(job_id)
if not job:
return None
metrics = self._metrics(job["job_type"])
return {
"job_id": job_id,
"request_id": job["request_id"],
"type": job["job_type"],
"status": job["status"],
"result": job["result"],
"error": job["error"],
**metrics,
}
async def stream_events(self, job_id: str) -> AsyncIterator[dict[str, Any]]:
queue: asyncio.Queue = asyncio.Queue()
history = list(self.event_history.get(job_id, []))
for item in history:
yield item
self.subscribers.setdefault(job_id, []).append(queue)
try:
while True:
event = await queue.get()
yield event
if event["event"] in TERMINAL_EVENTS:
break
finally:
with suppress(ValueError):
self.subscribers.get(job_id, []).remove(queue)
async def _worker_loop(self, job_type: str) -> None:
queue = self.queues[job_type]
sem = self.semaphores[job_type]
while True:
job_id = await queue.get()
async with self.lock:
job = self.jobs.get(job_id)
if not job or job["status"] == "cancelled":
continue
await sem.acquire()
task = asyncio.create_task(self._run_job(job_id))
self.running_tasks[job_id] = task
async def _run_job(self, job_id: str) -> None:
job = self.jobs[job_id]
job_type = job["job_type"]
try:
async with self.lock:
if job["status"] == "cancelled":
return
self.queue_counts[job_type] = max(0, self.queue_counts[job_type] - 1)
self.running_counts[job_type] += 1
job["status"] = "running"
job["updated_at"] = _now_ms()
metrics = self._metrics(job_type)
await self._publish(job_id, "started", {"job_id": job_id, "type": job_type, "status": "running", **metrics})
async def emit(event: str, data: dict[str, Any]) -> None:
metrics_now = self._metrics(job_type)
await self._publish(job_id, event, {"job_id": job_id, "type": job_type, "status": job["status"], **metrics_now, **data})
def is_cancelled() -> bool:
return bool(job.get("cancel_requested"))
result = await self.handlers[job_type](job["payload"], emit, is_cancelled)
async with self.lock:
if job["cancel_requested"]:
job["status"] = "cancelled"
metrics = self._metrics(job_type)
await emit("cancelled", {"reason": "abort"})
return
job["status"] = "completed"
job["result"] = result
job["updated_at"] = _now_ms()
metrics = self._metrics(job_type)
await self._publish(job_id, "done", {"job_id": job_id, "type": job_type, "status": "completed", "result": result, **metrics})
except asyncio.CancelledError:
async with self.lock:
job["status"] = "cancelled"
job["cancel_requested"] = True
metrics = self._metrics(job_type)
await self._publish(job_id, "cancelled", {"job_id": job_id, "type": job_type, "status": "cancelled", **metrics})
raise
except Exception as exc:
logger.exception("in-memory job failed job_id=%s type=%s", job_id, job_type)
async with self.lock:
job["status"] = "failed"
job["error"] = str(exc)
job["updated_at"] = _now_ms()
metrics = self._metrics(job_type)
await self._publish(job_id, "error", {"job_id": job_id, "type": job_type, "status": "failed", "error": str(exc), **metrics})
finally:
async with self.lock:
self.running_counts[job_type] = max(0, self.running_counts[job_type] - 1)
self.running_tasks.pop(job_id, None)
self.semaphores[job_type].release()
async def close(self) -> None:
for task in self.worker_tasks:
task.cancel()
for task in self.running_tasks.values():
task.cancel()
for task in self.worker_tasks:
with suppress(asyncio.CancelledError):
await task
self.worker_tasks.clear()
self.running_tasks.clear()
self.started = False
class RedisJobManager(BaseJobManager):
def __init__(self) -> None:
super().__init__()
if redis_asyncio is None:
raise JobSystemError("redis package is not installed")
self.redis = redis_asyncio.from_url(
os.getenv("REDIS_URL", "redis://localhost:6379/0"),
encoding="utf-8",
decode_responses=True,
)
self.prefix = (os.getenv("JOB_REDIS_PREFIX") or "llmtext:jobs").strip() or "llmtext:jobs"
self.state_ttl = _int_env("JOB_STATE_TTL_SECONDS", 600)
self.event_ttl = _int_env("JOB_EVENT_TTL_SECONDS", 600)
def _queue_key(self, job_type: str) -> str:
return f"{self.prefix}:queue:{job_type}"
def _event_key(self, job_id: str) -> str:
return f"{self.prefix}:events:{job_id}"
def _state_key(self, job_id: str) -> str:
return f"{self.prefix}:state:{job_id}"
def _metrics_key(self, job_type: str) -> str:
return f"{self.prefix}:metrics:{job_type}"
def _group_name(self, job_type: str) -> str:
return f"{self.prefix}:group:{job_type}"
async def ensure_groups(self) -> None:
for job_type in JOB_TYPES:
stream = self._queue_key(job_type)
group = self._group_name(job_type)
try:
await self.redis.xgroup_create(stream, group, id="0-0", mkstream=True)
except Exception as exc: # pragma: no cover - redis-specific
if "BUSYGROUP" not in str(exc):
raise
async def close(self) -> None:
await self.redis.aclose()
async def _metrics(self, job_type: str) -> dict[str, Any]:
raw = await self.redis.hgetall(self._metrics_key(job_type))
queued = int(raw.get("queued_count", "0") or 0)
running = int(raw.get("running_count", "0") or 0)
config = _queue_config(job_type)
capacity = max(config.max_queue + config.concurrency, 1)
ratio = min((queued + running) / capacity, 1.0)
return {
"queued_count": queued,
"running_count": running,
"concurrency_limit": config.concurrency,
"max_queue": config.max_queue,
"busy_ratio": round(ratio, 4),
"busy_level": _busy_level(ratio),
}
async def _emit_event(self, job_id: str, event: str, data: dict[str, Any]) -> None:
key = self._event_key(job_id)
payload = {k: _json_dumps(v) if not isinstance(v, str) else v for k, v in data.items()}
payload["event"] = event
await self.redis.xadd(key, payload, maxlen=_int_env("JOB_EVENT_STREAM_MAXLEN", 512), approximate=True)
await self.redis.expire(key, self.event_ttl)
async def _set_state(self, job_id: str, state: dict[str, Any]) -> None:
serializable = {k: _json_dumps(v) if isinstance(v, (dict, list)) else str(v) for k, v in state.items()}
await self.redis.hset(self._state_key(job_id), mapping=serializable)
await self.redis.expire(self._state_key(job_id), self.state_ttl)
async def submit(self, job_type: str, payload: dict[str, Any], request_id: str | None = None) -> str:
config = _queue_config(job_type)
metrics = await self._metrics(job_type)
if metrics["queued_count"] >= config.max_queue:
raise QueueFullError(f"{job_type} queue is full")
job_id = request_id or str(uuid.uuid4())
created_at = _now_ms()
state = {
"job_id": job_id,
"request_id": job_id,
"type": job_type,
"status": "queued",
"error": "",
"created_at": created_at,
"updated_at": created_at,
"cancel_requested": "0",
}
await self._set_state(job_id, state)
await self.redis.hincrby(self._metrics_key(job_type), "queued_count", 1)
await self.redis.expire(self._metrics_key(job_type), self.state_ttl)
metrics = await self._metrics(job_type)
await self._emit_event(job_id, "queued", {"job_id": job_id, "type": job_type, "status": "queued", **metrics})
await self.redis.xadd(self._queue_key(job_type), {"job_id": job_id, "payload": _json_dumps(payload), "request_id": job_id})
return job_id
async def cancel(self, job_id: str, reason: str = "abort") -> dict[str, Any]:
state = await self.get_status(job_id)
if not state:
return {"cancelled": False, "status": "not_found"}
if state["status"] in TERMINAL_STATUSES:
return {"cancelled": False, "status": state["status"]}
await self.redis.hset(self._state_key(job_id), mapping={"cancel_requested": "1", "status": "cancelled", "updated_at": _now_ms(), "cancel_reason": reason})
metrics = await self._metrics(state["type"])
await self._emit_event(job_id, "cancelled", {"job_id": job_id, "type": state["type"], "status": "cancelled", "reason": reason, **metrics})
return {"cancelled": True, "status": "ok"}
async def get_status(self, job_id: str) -> dict[str, Any] | None:
state = await self.redis.hgetall(self._state_key(job_id))
if not state:
return None
job_type = state.get("type", "")
metrics = await self._metrics(job_type) if job_type else {}
result = state.get("result")
error = state.get("error", "")
return {
"job_id": state.get("job_id", job_id),
"request_id": state.get("request_id", job_id),
"type": job_type,
"status": state.get("status", "queued"),
"error": error,
"result": _json_loads(result, result),
"cancel_requested": state.get("cancel_requested") == "1",
**metrics,
}
async def stream_events(self, job_id: str) -> AsyncIterator[dict[str, Any]]:
stream = self._event_key(job_id)
last_id = "0-0"
while True:
events = await self.redis.xread({stream: last_id}, block=1000, count=20)
if not events:
state = await self.get_status(job_id)
if state and state["status"] in TERMINAL_STATUSES:
break
continue
for _, entries in events:
for entry_id, fields in entries:
last_id = entry_id
event_payload: dict[str, Any] = {}
for key, value in fields.items():
if key == "event":
event_payload[key] = value
continue
try:
event_payload[key] = json.loads(value)
except Exception:
event_payload[key] = value
yield event_payload
if event_payload.get("event") in TERMINAL_EVENTS:
return
class RedisWorker:
def __init__(self, manager: RedisJobManager) -> None:
self.manager = manager
self.running_tasks: dict[str, asyncio.Task] = {}
self.queue_semaphores = {job_type: asyncio.Semaphore(_queue_config(job_type).concurrency) for job_type in JOB_TYPES}
self.poll_interval = _float_env("JOB_CANCEL_POLL_SECONDS", 0.5)
self.consumer_name = (os.getenv("JOB_CONSUMER_NAME") or f"worker-{uuid.uuid4().hex[:8]}").strip()
async def run_forever(self) -> None:
await self.manager.ensure_groups()
cancel_task = asyncio.create_task(self._cancel_watch_loop())
consumers = [asyncio.create_task(self._consume_loop(job_type)) for job_type in JOB_TYPES]
try:
await asyncio.gather(*consumers)
finally:
cancel_task.cancel()
with suppress(asyncio.CancelledError):
await cancel_task
async def _cancel_watch_loop(self) -> None:
while True:
await asyncio.sleep(self.poll_interval)
for job_id, task in list(self.running_tasks.items()):
state = await self.manager.get_status(job_id)
if state and state.get("cancel_requested") and not task.done():
task.cancel()
async def _consume_loop(self, job_type: str) -> None:
queue_key = self.manager._queue_key(job_type)
group = self.manager._group_name(job_type)
semaphore = self.queue_semaphores[job_type]
while True:
streams = await self.manager.redis.xreadgroup(group, self.consumer_name, {queue_key: ">"}, count=1, block=1000)
if not streams:
continue
for _, messages in streams:
for message_id, fields in messages:
await semaphore.acquire()
task = asyncio.create_task(self._run_message(job_type, queue_key, group, message_id, fields, semaphore))
self.running_tasks[fields["job_id"]] = task
async def _run_message(
self,
job_type: str,
queue_key: str,
group: str,
message_id: str,
fields: dict[str, str],
semaphore: asyncio.Semaphore,
) -> None:
job_id = fields["job_id"]
try:
state = await self.manager.get_status(job_id)
if not state or state["status"] == "cancelled":
await self.manager.redis.xack(queue_key, group, message_id)
return
await self.manager.redis.hincrby(self.manager._metrics_key(job_type), "queued_count", -1)
await self.manager.redis.hincrby(self.manager._metrics_key(job_type), "running_count", 1)
await self.manager._set_state(job_id, {
"job_id": job_id,
"request_id": state["request_id"],
"type": job_type,
"status": "running",
"updated_at": _now_ms(),
"created_at": state.get("created_at", _now_ms()),
"cancel_requested": "1" if state.get("cancel_requested") else "0",
"error": "",
})
metrics = await self.manager._metrics(job_type)
await self.manager._emit_event(job_id, "started", {"job_id": job_id, "type": job_type, "status": "running", **metrics})
payload = _json_loads(fields["payload"], {})
async def emit(event: str, data: dict[str, Any]) -> None:
live_state = await self.manager.get_status(job_id) or {"status": "running"}
live_metrics = await self.manager._metrics(job_type)
await self.manager._emit_event(job_id, event, {"job_id": job_id, "type": job_type, "status": live_state["status"], **live_metrics, **data})
def is_cancelled() -> bool:
task = self.running_tasks.get(job_id)
return bool(task and task.cancelled())
result = await self.manager.handlers[job_type](payload, emit, is_cancelled)
current = await self.manager.get_status(job_id)
if current and current["status"] == "cancelled":
return
await self.manager._set_state(job_id, {
"job_id": job_id,
"request_id": state["request_id"],
"type": job_type,
"status": "completed",
"updated_at": _now_ms(),
"created_at": state.get("created_at", _now_ms()),
"cancel_requested": "0",
"error": "",
"result": _json_dumps(result),
})
metrics = await self.manager._metrics(job_type)
await self.manager._emit_event(job_id, "done", {"job_id": job_id, "type": job_type, "status": "completed", "result": result, **metrics})
await self.manager.redis.xack(queue_key, group, message_id)
except asyncio.CancelledError:
await self.manager.redis.hset(self.manager._state_key(job_id), mapping={"status": "cancelled", "cancel_requested": "1", "updated_at": _now_ms()})
metrics = await self.manager._metrics(job_type)
await self.manager._emit_event(job_id, "cancelled", {"job_id": job_id, "type": job_type, "status": "cancelled", **metrics})
await self.manager.redis.xack(queue_key, group, message_id)
raise
except Exception as exc:
logger.exception("redis worker failed job_id=%s type=%s", job_id, job_type)
state = await self.manager.get_status(job_id)
request_id = state["request_id"] if state else job_id
await self.manager._set_state(job_id, {
"job_id": job_id,
"request_id": request_id,
"type": job_type,
"status": "failed",
"updated_at": _now_ms(),
"created_at": state.get("created_at", _now_ms()) if state else _now_ms(),
"cancel_requested": "0",
"error": str(exc),
})
metrics = await self.manager._metrics(job_type)
await self.manager._emit_event(job_id, "error", {"job_id": job_id, "type": job_type, "status": "failed", "error": str(exc), **metrics})
await self.manager.redis.xack(queue_key, group, message_id)
finally:
self.running_tasks.pop(job_id, None)
await self.manager.redis.hincrby(self.manager._metrics_key(job_type), "running_count", -1)
semaphore.release()
_job_manager: BaseJobManager | None = None
def get_job_manager() -> BaseJobManager:
global _job_manager
if _job_manager is None:
backend = get_job_backend_name()
if backend == "redis":
_job_manager = RedisJobManager()
else:
_job_manager = InMemoryJobManager()
return _job_manager
def reset_job_manager() -> None:
global _job_manager
manager = _job_manager
if manager is not None:
close = getattr(manager, "close", None)
if close is not None:
try:
loop = asyncio.get_running_loop()
except RuntimeError:
try:
asyncio.run(close())
except Exception:
pass
else:
loop.create_task(close())
_job_manager = None
+10 -8
View File
@@ -87,16 +87,17 @@ def _build_chat_payload(
if prefill:
messages.append({'role': 'assistant', 'content': prefill})
options = {'temperature': temperature}
if thinking:
options['think'] = thinking
payload = {
'model': _resolve_model_name(model, use_pro_model=use_pro_model),
'messages': messages,
'stream': False,
'options': options,
}
options = {'temperature': temperature}
if thinking:
payload['options'] = {'temperature': temperature, 'think': thinking}
return payload
@@ -119,16 +120,17 @@ def _build_chat_stream_payload(
if prefill:
messages.append({'role': 'assistant', 'content': prefill})
options = {'temperature': temperature}
if thinking:
options['think'] = thinking
payload = {
'model': _resolve_model_name(model, use_pro_model=use_pro_model),
'messages': messages,
'stream': True,
'options': options,
}
options = {'temperature': temperature}
if thinking:
payload['options'] = {'temperature': temperature, 'think': thinking}
return payload
+326 -339
View File
@@ -1,12 +1,7 @@
import asyncio
import base64
import json
import logging
import os
import re
import shutil
import subprocess
import tempfile
import uuid
from typing import Optional
@@ -17,10 +12,28 @@ from fastapi.security import APIKeyHeader
from pydantic import BaseModel
from geoip import get_ip_location_text
from llm import call_ollama, call_vlm_ocr, stream_ollama
from job_handlers import (
_sanitize_converted_markdown,
sanitize_inline_completion_content,
ALLOWED_CONVERT_EXTENSIONS,
asr_handler,
completion_handler,
compress_handler,
convert_handler,
ocr_handler,
pro_completion_handler,
tts_handler,
)
from job_system import (
InMemoryJobManager,
JobSystemError,
JOB_TYPES,
QueueFullError,
RedisJobManager,
get_job_manager,
persist_temp_input,
)
from models import UserPreferences
from prompt import build_completion_prompts, prepare_prompt_context
import markitdown
logging.basicConfig(
level=logging.INFO,
@@ -28,22 +41,7 @@ logging.basicConfig(
)
logger = logging.getLogger("api")
_markitdown_instance = None
def _get_markitdown(): # pragma: no cover
global _markitdown_instance
if _markitdown_instance is None:
_markitdown_instance = markitdown.MarkItDown()
return _markitdown_instance
app = FastAPI()
# Startup event disabled — TTS model loads lazily on first request
# to avoid blocking startup and OOM crashes.
ACTIVE_COMPLETIONS: dict[str, asyncio.Task] = {}
ACTIVE_COMPLETIONS_LOCK = asyncio.Lock()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
@@ -53,16 +51,9 @@ app.add_middleware(
)
API_KEY = os.getenv("API_KEY", "your-secret-key-here")
DOC_COMPRESS_CONTEXT_LIMIT = int(os.getenv("DOC_COMPRESS_CONTEXT_LIMIT", "128000"))
api_key_header = APIKeyHeader(name="X-API-Key")
async def get_api_key(api_key: str = Security(api_key_header)): # pragma: no cover
if api_key != API_KEY:
raise HTTPException(
status_code=403,
detail="Could not validate credentials",
)
return api_key
_handlers_registered = False
class CompletionRequest(BaseModel):
@@ -76,6 +67,16 @@ class CompletionRequest(BaseModel):
temperature: float = 0.7
class ProCompletionRequest(BaseModel):
prefix: str
suffix: str
languageId: str = "markdown"
instruction: str = ""
pro_thinking: str = "medium"
privacy_mode: bool = False
user_preferences: Optional[UserPreferences] = None
class CancelCompletionRequest(BaseModel):
request_id: str
reason: str = "abort"
@@ -92,30 +93,21 @@ class ConvertRequest(BaseModel):
filename: str = "document.pdf"
ALLOWED_CONVERT_EXTENSIONS = {".txt", ".docx", ".pptx", ".pdf"}
IMAGE_MARKDOWN_RE = re.compile(r"!\[[^\]]*]\([^)]+\)")
IMAGE_HTML_RE = re.compile(r"<img\b[^>]*>", re.IGNORECASE)
class CompressRequest(BaseModel):
content: str
docType: str = "txt"
def _convert_docx_to_pdf(input_path: str, output_path: str) -> None: # pragma: no cover
node_executable = shutil.which("node")
if not node_executable:
raise RuntimeError("未找到 Node.js,无法转换 DOCX 为 PDF")
class TTSJobRequest(BaseModel):
text: str
instruct: str = ""
speaker: str = "Vivian"
format: str = "wav"
bridge_path = os.path.join(os.path.dirname(__file__), "docx2pdf_bridge.cjs")
if not os.path.exists(bridge_path):
raise RuntimeError("缺少 DOCX 转 PDF 桥接脚本")
result = subprocess.run(
[node_executable, bridge_path, input_path, output_path],
cwd=os.path.dirname(os.path.dirname(__file__)),
capture_output=True,
text=True,
)
if result.returncode != 0:
error_text = (result.stderr or result.stdout or "DOCX 转 PDF 失败").strip()
raise RuntimeError(error_text)
class ASRJobRequest(BaseModel):
audio_base64: str
language: Optional[str] = "zh-CN"
def _preview(text: str, limit: int = 80) -> str:
@@ -125,14 +117,6 @@ def _preview(text: str, limit: int = 80) -> str:
return value[:limit] + "..."
def _sanitize_converted_markdown(text: str) -> str:
value = (text or "").replace("\r\n", "\n").replace("\r", "\n")
value = IMAGE_MARKDOWN_RE.sub("", value)
value = IMAGE_HTML_RE.sub("", value)
value = re.sub(r"\n{3,}", "\n\n", value)
return value.strip()
def get_client_ip(request: Request) -> str:
if request.client:
return request.headers.get("X-Client-IP") or request.client.host
@@ -147,197 +131,56 @@ def _clamp_temperature(value: float, default: float = 0.7) -> float:
return max(0.0, min(numeric, 1.2))
@app.post("/v1/completions")
async def create_completion(request: Request, req: CompletionRequest, api_key: str = Security(get_api_key)):
request_id = request.headers.get("X-Request-Id") or str(uuid.uuid4())
request_tag = request_id[:8]
inference_task: Optional[asyncio.Task] = None
client_ip = "hidden"
location = ""
if not req.privacy_mode: # pragma: no cover
client_ip = get_client_ip(request)
location = get_ip_location_text(client_ip)
if location:
logger.info("[%s] client_location=%s", request_tag, location)
try:
logger.info(
"[%s] /v1/completions request_id=%s client_ip=%s prefix_chars=%d suffix_chars=%d lang=%s thinking=%s privacy=%s",
request_tag,
request_id,
client_ip,
len(req.prefix or ""),
len(req.suffix or ""),
req.languageId,
req.model_thinking,
req.privacy_mode,
)
llm_prefix, llm_suffix = prepare_prompt_context(req.prefix or "", req.suffix or "")
logger.info("[%s] llm_input_prefix=%r", request_tag, llm_prefix)
logger.info("[%s] llm_input_suffix=%r", request_tag, llm_suffix)
system_prompt, user_prompt, prefill = build_completion_prompts(
req.prefix,
req.suffix,
req.languageId,
location=location,
thinking_level=req.model_thinking,
preferences=req.user_preferences,
)
inference_task = asyncio.create_task(
call_ollama(
user_prompt,
system_prompt=system_prompt,
tag=f"{request_tag}-primary",
temperature=_clamp_temperature(req.temperature, 0.7),
thinking=req.model_thinking if req.model_thinking != "none" else None,
model=req.model,
prefill=prefill or None,
)
)
existing = ACTIVE_COMPLETIONS.get(request_id)
if existing and not existing.done():
existing.cancel()
ACTIVE_COMPLETIONS[request_id] = inference_task
result = await inference_task
content = result["content"] or ""
if not content.strip():
logger.warning("[%s] primary returned empty content, returning empty result", request_tag)
logger.info(
"[%s] completion resolved source=primary request_id=%s content_chars=%d content_preview='%s'",
request_tag,
request_id,
len(content),
_preview(content, 120),
)
return JSONResponse(content={"content": content, "request_id": request_id})
except asyncio.CancelledError:
logger.info("[%s] /v1/completions cancelled request_id=%s", request_tag, request_id)
return JSONResponse(content={"cancelled": True, "request_id": request_id}, status_code=499)
except Exception as e:
logger.exception("[%s] /v1/completions failed request_id=%s: %s", request_tag, request_id, e)
return JSONResponse(content={"error": str(e)}, status_code=500)
finally:
active = ACTIVE_COMPLETIONS.get(request_id)
if active is not None and active is inference_task:
ACTIVE_COMPLETIONS.pop(request_id, None)
async def get_api_key(api_key: str = Security(api_key_header)): # pragma: no cover
if api_key != API_KEY:
raise HTTPException(status_code=403, detail="Could not validate credentials")
return api_key
@app.post("/v1/pro/completions/stream")
async def create_pro_completion_stream(request: Request, req: CompletionRequest, api_key: str = Security(get_api_key)):
request_id = request.headers.get("X-Request-Id") or str(uuid.uuid4())
request_tag = request_id[:8]
queue: asyncio.Queue[Optional[tuple[str, str]]] = asyncio.Queue()
def _serialize_preferences(preferences: UserPreferences | None) -> dict | None:
if preferences is None:
return None
if hasattr(preferences, "dict"):
return preferences.dict()
return dict(preferences)
client_ip = "hidden"
location = ""
if not req.privacy_mode: # pragma: no cover
client_ip = get_client_ip(request)
location = get_ip_location_text(client_ip)
if location:
logger.info("[%s] client_location=%s", request_tag, location)
def _request_id(request: Request) -> str:
return request.headers.get("X-Request-Id") or str(uuid.uuid4())
logger.info(
"[%s] /v1/pro/completions/stream request_id=%s client_ip=%s prefix_chars=%d suffix_chars=%d lang=%s thinking=%s privacy=%s model=%s temp=%.2f",
request_tag,
request_id,
client_ip,
len(req.prefix or ""),
len(req.suffix or ""),
req.languageId,
req.model_thinking,
req.privacy_mode,
req.model or "",
_clamp_temperature(req.temperature, 0.7),
)
llm_prefix, llm_suffix = prepare_prompt_context(req.prefix or "", req.suffix or "")
logger.info("[%s] pro_llm_input_prefix=%r", request_tag, llm_prefix)
logger.info("[%s] pro_llm_input_suffix=%r", request_tag, llm_suffix)
def _register_handlers() -> None:
global _handlers_registered
if _handlers_registered:
return
manager = get_job_manager()
manager.register_handler("completion", completion_handler)
manager.register_handler("pro_completion", pro_completion_handler)
manager.register_handler("compress", compress_handler)
manager.register_handler("ocr", ocr_handler)
manager.register_handler("convert", convert_handler)
manager.register_handler("tts", tts_handler)
manager.register_handler("asr", asr_handler)
_handlers_registered = True
system_prompt, user_prompt, prefill = build_completion_prompts(
req.prefix,
req.suffix,
req.languageId,
location=location,
thinking_level=req.model_thinking,
preferences=req.user_preferences,
)
async def producer() -> None:
chunks: list[str] = []
try:
async for delta in stream_ollama(
user_prompt,
system_prompt=system_prompt,
tag=f"{request_tag}-pro",
temperature=_clamp_temperature(req.temperature, 0.7),
thinking=req.model_thinking if req.model_thinking != "none" else None,
model=req.model,
use_pro_model=True,
prefill=prefill or None,
):
chunks.append(delta)
await queue.put(("chunk", json.dumps({"delta": delta}, ensure_ascii=False)))
def _sse(event: str, data: dict) -> str:
return f"event: {event}\ndata: {json.dumps(data, ensure_ascii=False)}\n\n"
content = "".join(chunks)
logger.info(
"[%s] pro stream resolved request_id=%s content_chars=%d content_preview='%s'",
request_tag,
request_id,
len(content),
_preview(content, 120),
)
await queue.put((
"done",
json.dumps({"content": content, "request_id": request_id}, ensure_ascii=False),
))
except asyncio.CancelledError:
logger.info("[%s] /v1/pro/completions/stream cancelled request_id=%s", request_tag, request_id)
await queue.put((
"cancelled",
json.dumps({"cancelled": True, "request_id": request_id}, ensure_ascii=False),
))
raise
except Exception as e:
logger.exception("[%s] /v1/pro/completions/stream failed request_id=%s: %s", request_tag, request_id, e)
await queue.put((
"error",
json.dumps({"error": str(e), "request_id": request_id}, ensure_ascii=False),
))
finally:
await queue.put(None)
producer_task = asyncio.create_task(producer())
existing = ACTIVE_COMPLETIONS.get(request_id)
if existing and not existing.done():
existing.cancel()
ACTIVE_COMPLETIONS[request_id] = producer_task
async def _stream_job(job_id: str):
_register_handlers()
manager = get_job_manager()
async def event_stream():
try:
while True:
item = await queue.get()
if item is None:
break
event_name, data = item
yield f"event: {event_name}\ndata: {data}\n\n"
except asyncio.CancelledError:
producer_task.cancel()
raise
finally:
active = ACTIVE_COMPLETIONS.get(request_id)
if active is producer_task:
ACTIVE_COMPLETIONS.pop(request_id, None)
async for event in manager.stream_events(job_id):
event_name = event.get("event", "message")
payload = {k: v for k, v in event.items() if k != "event"}
yield _sse(event_name, payload)
except Exception as exc:
logger.exception("job stream failed job_id=%s", job_id)
yield _sse("error", {"job_id": job_id, "error": str(exc)})
return StreamingResponse(
event_stream(),
@@ -349,130 +192,266 @@ async def create_pro_completion_stream(request: Request, req: CompletionRequest,
)
async def _queue_job(job_type: str, payload: dict, request_id: str) -> str:
_register_handlers()
manager = get_job_manager()
return await manager.submit(job_type, payload, request_id=request_id)
async def _cancel_job(request_id: str, reason: str) -> dict:
_register_handlers()
manager = get_job_manager()
return await manager.cancel(request_id, reason)
async def _job_status(job_id: str) -> dict | None:
_register_handlers()
manager = get_job_manager()
return await manager.get_status(job_id)
async def _queue_load_snapshot() -> dict:
manager = get_job_manager()
if isinstance(manager, InMemoryJobManager):
return {job_type: manager._metrics(job_type) for job_type in manager.queues}
if isinstance(manager, RedisJobManager):
return {job_type: await manager._metrics(job_type) for job_type in JOB_TYPES}
return {}
@app.post("/v1/completions")
async def create_completion(
request: Request,
req: CompletionRequest,
api_key: str = Security(get_api_key),
):
del api_key
request_id = _request_id(request)
location = ""
if not req.privacy_mode: # pragma: no cover
location = get_ip_location_text(get_client_ip(request))
payload = {
"request_id": request_id,
"location": location,
"request": {
"prefix": req.prefix,
"suffix": req.suffix,
"languageId": req.languageId,
"model_thinking": req.model_thinking,
"privacy_mode": req.privacy_mode,
"user_preferences": _serialize_preferences(req.user_preferences),
"model": req.model,
"temperature": _clamp_temperature(req.temperature, 0.7),
},
}
try:
job_id = await _queue_job("completion", payload, request_id)
except QueueFullError as exc:
return JSONResponse({"error": str(exc), "request_id": request_id}, status_code=429)
except JobSystemError as exc:
return JSONResponse({"error": str(exc), "request_id": request_id}, status_code=503)
return await _stream_job(job_id)
@app.post("/v1/completions/cancel")
async def cancel_completion(req: CancelCompletionRequest, api_key: str = Security(get_api_key)):
request_tag = str(uuid.uuid4())[:8]
request_id = req.request_id or ""
del api_key
return await _cancel_job(req.request_id or "", req.reason)
async with ACTIVE_COMPLETIONS_LOCK:
task = ACTIVE_COMPLETIONS.get(request_id)
if task is None:
logger.info(
"[%s] /v1/completions/cancel request_id=%s status=not_found reason=%s",
request_tag,
request_id,
req.reason,
)
return {"cancelled": False, "status": "not_found"}
if task.done():
logger.info(
"[%s] /v1/completions/cancel request_id=%s status=already_done reason=%s",
request_tag,
request_id,
req.reason,
)
return {"cancelled": False, "status": "already_done"}
@app.post("/v1/pro/completions")
async def create_pro_completion(
request: Request,
req: ProCompletionRequest,
api_key: str = Security(get_api_key),
):
del api_key
request_id = _request_id(request)
location = ""
if not req.privacy_mode: # pragma: no cover
location = get_ip_location_text(get_client_ip(request))
payload = {
"request_id": request_id,
"location": location,
"request": {
"prefix": req.prefix,
"suffix": req.suffix,
"languageId": req.languageId,
"instruction": req.instruction,
"pro_thinking": req.pro_thinking,
"privacy_mode": req.privacy_mode,
"user_preferences": _serialize_preferences(req.user_preferences),
},
}
try:
job_id = await _queue_job("pro_completion", payload, request_id)
except QueueFullError as exc:
return JSONResponse({"error": str(exc), "request_id": request_id}, status_code=429)
except JobSystemError as exc:
return JSONResponse({"error": str(exc), "request_id": request_id}, status_code=503)
return await _stream_job(job_id)
task.cancel()
logger.info(
"[%s] /v1/completions/cancel request_id=%s status=ok reason=%s",
request_tag,
request_id,
req.reason,
)
return {"cancelled": True, "status": "ok"}
@app.post("/v1/pro/completions/cancel")
async def cancel_pro_completion(req: CancelCompletionRequest, api_key: str = Security(get_api_key)):
del api_key
return await _cancel_job(req.request_id or "", req.reason)
@app.get("/v1/pro/completions/status/{request_id}")
async def get_pro_completion_status(request_id: str, api_key: str = Security(get_api_key)):
del api_key
state = await _job_status(request_id)
if state is None:
raise HTTPException(status_code=404, detail="PRO request not found")
return state
@app.post("/v1/ocr")
async def ocr_image(request: OCRRequest, api_key: str = Security(get_api_key)):
request_id = str(uuid.uuid4())[:8]
async def ocr_image(req: OCRRequest, api_key: str = Security(get_api_key)):
del api_key
request_id = str(uuid.uuid4())
try:
logger.info(
"[%s] /v1/ocr filename=%s language=%s image_base64_chars=%d",
request_id,
request.filename,
request.language,
len(request.image or ""),
)
image_bytes = base64.b64decode(request.image)
logger.info("[%s] /v1/ocr decoded image_bytes=%d", request_id, len(image_bytes))
result = await call_vlm_ocr(image_bytes, request.language)
logger.info(
"[%s] /v1/ocr success text_chars=%d text_preview='%s'",
request_id,
len(result or ""),
_preview(result or "", 120),
)
return {"text": result, "filename": request.filename}
except Exception as e:
logger.exception("[%s] /v1/ocr failed: %s", request_id, e)
return JSONResponse(content={"error": str(e)}, status_code=500)
image_bytes = base64.b64decode(req.image)
except Exception as exc:
return JSONResponse({"error": str(exc)}, status_code=500)
input_path = persist_temp_input(image_bytes, os.path.splitext(req.filename)[1] or ".img")
try:
job_id = await _queue_job("ocr", {
"request_id": request_id,
"input_path": input_path,
"filename": req.filename,
"language": req.language,
}, request_id)
except Exception:
if os.path.exists(input_path):
os.unlink(input_path)
raise
return await _stream_job(job_id)
@app.post("/v1/convert")
async def convert_to_markdown(request: ConvertRequest, api_key: str = Security(get_api_key)):
"""Convert file to markdown"""
request_id = str(uuid.uuid4())[:8]
async def convert_to_markdown(req: ConvertRequest, api_key: str = Security(get_api_key)):
del api_key
request_id = str(uuid.uuid4())
ext = os.path.splitext(req.filename)[1].lower()
if ext not in ALLOWED_CONVERT_EXTENSIONS:
return JSONResponse({"error": "仅支持 txt、docx、pptx、pdf 格式"}, status_code=500)
try:
logger.info(
"[%s] /v1/convert filename=%s file_base64_chars=%d",
request_id,
request.filename,
len(request.file or ""),
file_bytes = base64.b64decode(req.file)
except Exception as exc:
return JSONResponse({"error": str(exc)}, status_code=500)
input_path = persist_temp_input(file_bytes, ext or ".bin")
try:
job_id = await _queue_job("convert", {
"request_id": request_id,
"input_path": input_path,
"filename": req.filename,
}, request_id)
except Exception:
if os.path.exists(input_path):
os.unlink(input_path)
raise
return await _stream_job(job_id)
@app.post("/v1/compress/submit")
async def submit_compress(req: CompressRequest, api_key: str = Security(get_api_key)):
del api_key
content = req.content or ""
if not content.strip():
raise HTTPException(status_code=400, detail="文档内容为空,无法压缩")
if len(content) > DOC_COMPRESS_CONTEXT_LIMIT:
raise HTTPException(
status_code=400,
detail=f"文档内容过长({len(content)} 字符),超过限制 {DOC_COMPRESS_CONTEXT_LIMIT},无法压缩",
)
task_id = str(uuid.uuid4())
await _queue_job("compress", {"request_id": task_id, "content": content, "docType": req.docType or "txt"}, task_id)
return {"task_id": task_id, "status": "queued"}
# Decode base64
file_bytes = base64.b64decode(request.file)
logger.info("[%s] /v1/convert decoded file_bytes=%d", request_id, len(file_bytes))
# Get file extension
ext = os.path.splitext(request.filename)[1].lower()
@app.get("/v1/compress/status")
async def get_compress_status(task_id: str, api_key: str = Security(get_api_key)):
del api_key
if not task_id:
raise HTTPException(status_code=400, detail="缺少 task_id 参数")
state = await _job_status(task_id)
if state is None:
raise HTTPException(status_code=404, detail="任务不存在或已过期")
if state["status"] == "completed":
result = state.get("result") or {}
return {"task_id": task_id, "status": "completed", "content": result.get("content", "")}
if state["status"] == "failed":
return {"task_id": task_id, "status": "error", "message": state.get("error") or ""}
if state["status"] == "cancelled":
return {"task_id": task_id, "status": "cancelled"}
return {
"task_id": task_id,
"status": "processing" if state["status"] == "running" else "queued",
"busy_level": state.get("busy_level"),
"queued_count": state.get("queued_count"),
"running_count": state.get("running_count"),
}
if ext not in ALLOWED_CONVERT_EXTENSIONS:
raise ValueError("仅支持 txt、docx、pptx、pdf 格式")
if ext == ".txt":
markdown_text = _sanitize_converted_markdown(file_bytes.decode("utf-8", errors="ignore"))
return {
"markdown": markdown_text,
"filename": request.filename
}
@app.post("/v1/tts-asr/tts")
async def queue_tts(req: TTSJobRequest, request: Request, api_key: str = Security(get_api_key)):
del api_key
request_id = _request_id(request)
job_id = await _queue_job("tts", {
"request_id": request_id,
"text": req.text,
"instruct": req.instruct,
"speaker": req.speaker,
"format": req.format,
}, request_id)
return await _stream_job(job_id)
# Create temporary file
with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as tmp:
tmp.write(file_bytes)
tmp_path = tmp.name
try:
# Convert using MarkItDown
md = _get_markitdown()
result = await asyncio.to_thread(md.convert, tmp_path)
markdown_text = _sanitize_converted_markdown(result.text_content)
@app.post("/v1/tts-asr/asr")
async def queue_asr(req: ASRJobRequest, request: Request, api_key: str = Security(get_api_key)):
del api_key
request_id = _request_id(request)
try:
audio_bytes = base64.b64decode(req.audio_base64)
except Exception as exc:
return JSONResponse({"error": str(exc)}, status_code=500)
input_path = persist_temp_input(audio_bytes, ".wav")
try:
job_id = await _queue_job("asr", {
"request_id": request_id,
"input_path": input_path,
"language": req.language or "zh-CN",
}, request_id)
except Exception:
if os.path.exists(input_path):
os.unlink(input_path)
raise
return await _stream_job(job_id)
logger.info(
"[%s] /v1/convert success text_chars=%d text_preview='%s'",
request_id,
len(markdown_text or ""),
_preview(markdown_text, 120),
)
return {
"markdown": markdown_text,
"filename": request.filename
}
finally:
# Clean up temporary file
if os.path.exists(tmp_path):
os.unlink(tmp_path)
@app.post("/v1/jobs/{job_id}/cancel")
async def cancel_job(job_id: str, req: CancelCompletionRequest, api_key: str = Security(get_api_key)):
del api_key
return await _cancel_job(req.request_id or job_id, req.reason)
@app.get("/v1/jobs/{job_id}/status")
async def get_job_status(job_id: str, api_key: str = Security(get_api_key)):
del api_key
state = await _job_status(job_id)
if state is None:
raise HTTPException(status_code=404, detail="job not found")
return state
@app.get("/v1/jobs/load")
async def get_job_load(api_key: str = Security(get_api_key)):
del api_key
return {"queues": await _queue_load_snapshot()}
except Exception as e:
logger.exception("[%s] /v1/convert failed: %s", request_id, e)
return JSONResponse(content={"error": str(e)}, status_code=500)
# TTS and ASR routes (lazy loaded to avoid heavy import on startup)
def _register_tts_asr_routes():
try:
from tts_asr import register_tts_asr_routes
@@ -484,14 +463,22 @@ def _register_tts_asr_routes():
return
try:
register_tts_asr_routes(app)
register_tts_asr_routes(app, include_generation_routes=False)
except Exception as exc:
logger.warning("Failed to register TTS/ASR routes: %s", exc)
_register_tts_asr_routes()
@app.on_event("shutdown")
async def _shutdown_job_manager(): # pragma: no cover
manager = get_job_manager()
close = getattr(manager, "close", None)
if close is not None:
await close()
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8001)
+3
View File
@@ -260,15 +260,18 @@ def register_pro_completion_routes(app: FastAPI, get_api_key):
enable_thinking=True,
timeout=PRO_COMPLETION_TIMEOUT,
):
# Handle 'thinking' event - just update state, don't accumulate
if event_type == "thinking":
await _send_sse_event(event_queue, "thinking", {"request_id": request_id})
continue
# Handle 'chunk' event - accumulate content
if not payload:
continue
chunks.append(payload)
await _send_sse_event(event_queue, "chunk", {"delta": payload, "request_id": request_id})
# Handle 'done' event - return full content
content = "".join(chunks)
async with PRO_STATES_LOCK:
if state.cancel_requested:
+1 -1
View File
@@ -1,3 +1,3 @@
{
"template": "You are the [PRO] model for LLM-IN-TEXT, specializing in high-precision markdown insertion for a {language_id} editor.\n\nReturn only the insertion text that should be placed between PREFIX and SUFFIX.\n\nPRO CORE PRINCIPLE:\n- Output insertion text only. No explanations, no analysis, no labels, no wrapper quotes.\n- Never output chain-of-thought or internal reasoning.\n- Never output control markers like <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>, assistant, final, channel.\n\nPRO MODE INTENT:\n- This is PRO_MODE=true. You may produce longer, structured markdown when instruction requires it.\n- Prioritize instruction fidelity first, then boundary safety, then style continuity.\n- If instruction is vague, continue naturally with concrete and useful content.\n\nBOUNDARY AND CONTEXT RULES:\n- Respect CURSOR_IN_FENCED_CODE_BLOCK, CURSOR_FENCE_LANGUAGE, MERMAID_CONTEXT, PREFIX_ENDS_WITH_NEWLINE, and SUFFIX_STARTS_WITH_NEWLINE.\n- Never repeat text from the beginning of SUFFIX.\n- Use minimum necessary newlines to avoid boundary collision.\n- Match PREFIX tone, language, and formatting conventions.\n\nSYNTAX PRIORITY:\n- Code block contexts must keep valid syntax and indentation.\n- Math must use $...$ for inline and $$...$$ for blocks unless inside latex fences.\n- Mermaid contexts must output valid mermaid statements; do not duplicate fences when already inside one.\n\nHIDDEN CONTEXT SAFETY:\n- OCR metadata and document-side context are hidden hints only.\n- Never copy hidden tags (e.g., <OCR:...>) into output.\n\nQUALITY BAR FOR PRO:\n- Prefer specific, information-dense output over generic filler.\n- For structured requests, preserve headings/list hierarchy and produce coherent section flow.\n- Keep output directly insertable without post-edit cleanups."
"template": "You are the [PRO] model for LLM-IN-TEXT, specializing in high-precision markdown insertion for a {language_id} editor.\n\nReturn only the insertion text that should be placed between PREFIX and SUFFIX.\n\nPRO CORE PRINCIPLE:\n- Output insertion text only. No explanations, no analysis, no labels, no wrapper quotes.\n- Never wrap the entire answer in an outer ```markdown code fence; only use fenced code blocks when the inserted content itself requires code.\n- Never output chain-of-thought or internal reasoning.\n- Never output control markers like <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>, assistant, final, channel.\n\nPRO MODE INTENT:\n- This is PRO_MODE=true. You may produce longer, structured markdown when instruction requires it.\n- Prioritize instruction fidelity first, then boundary safety, then style continuity.\n- If instruction is vague, continue naturally with concrete and useful content.\n\nBOUNDARY AND CONTEXT RULES:\n- Respect CURSOR_IN_FENCED_CODE_BLOCK, CURSOR_FENCE_LANGUAGE, MERMAID_CONTEXT, PREFIX_ENDS_WITH_NEWLINE, and SUFFIX_STARTS_WITH_NEWLINE.\n- Never repeat text from the beginning of SUFFIX.\n- Use minimum necessary newlines to avoid boundary collision.\n- Match PREFIX tone, language, and formatting conventions.\n\nSYNTAX PRIORITY:\n- Code block contexts must keep valid syntax and indentation.\n- Math must use $...$ for inline and $$...$$ for blocks unless inside latex fences.\n- Mermaid contexts must output valid mermaid statements; do not duplicate fences when already inside one.\n\nHIDDEN CONTEXT SAFETY:\n- OCR metadata and document-side context are hidden hints only.\n- Never copy hidden tags (e.g., <OCR:...>) into output.\n\nQUALITY BAR FOR PRO:\n- Prefer specific, information-dense output over generic filler.\n- For structured requests, preserve headings/list hierarchy and produce coherent section flow.\n- Keep output directly insertable without post-edit cleanups."
}
+2
View File
@@ -2,6 +2,8 @@ fastapi>=0.95.0
uvicorn[standard]>=0.23.0
pydantic>=1.10.0
httpx>=0.24.0
redis>=5.0.0
python-dotenv>=1.0.0
numpy>=1.23.0
soundfile>=0.10.3
+79
View File
@@ -0,0 +1,79 @@
import importlib
import os
import sys
from pathlib import Path
from fastapi.testclient import TestClient
os.environ["JOB_BACKEND"] = "memory"
CURRENT_DIR = Path(__file__).resolve().parent
BACKEND_DIR = CURRENT_DIR.parent
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
import job_handlers # type: ignore
import job_system # type: ignore
main = importlib.import_module("main")
API_KEY = main.API_KEY
HEADERS = {"X-API-Key": API_KEY}
def setup_function():
job_system.reset_job_manager()
main._handlers_registered = False
def _submit(client, content="test document", doc_type="txt", headers=None):
return client.post("/v1/compress/submit", headers=headers if headers is not None else HEADERS, json={
"content": content,
"docType": doc_type,
})
def _status(client, task_id, headers=None):
return client.get(f"/v1/compress/status?task_id={task_id}", headers=headers if headers is not None else HEADERS)
def test_submit_empty_content_returns_400():
with TestClient(main.app) as client:
resp = _submit(client, "")
assert resp.status_code == 400
def test_submit_too_long_returns_400(monkeypatch):
monkeypatch.setattr(main, "DOC_COMPRESS_CONTEXT_LIMIT", 10)
with TestClient(main.app) as client:
resp = _submit(client, "a" * 100)
assert resp.status_code == 400
def test_submit_success_returns_task_id():
with TestClient(main.app) as client:
resp = _submit(client, "hello world")
assert resp.status_code == 200
data = resp.json()
assert "task_id" in data
assert data["status"] == "queued"
def test_status_not_found_returns_404():
with TestClient(main.app) as client:
resp = _status(client, "nonexistent-id")
assert resp.status_code == 404
def test_status_completed(monkeypatch):
async def fake_call_ollama(prompt, system_prompt=None, **kwargs): # noqa: ARG001
return {"content": f"[compressed] {prompt[:20]}"}
monkeypatch.setattr(job_handlers, "call_ollama", fake_call_ollama)
with TestClient(main.app) as client:
resp = _submit(client, "important document text")
task_id = resp.json()["task_id"]
status_resp = _status(client, task_id)
data = status_resp.json()
assert status_resp.status_code == 200
assert data["status"] in {"queued", "processing", "completed"}
+19 -45
View File
@@ -1,26 +1,31 @@
import asyncio
import importlib
import os
import sys
import threading
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
os.environ["JOB_BACKEND"] = "memory"
BACKEND_DIR = Path(__file__).resolve().parents[1]
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
try:
main = importlib.import_module("main")
except ModuleNotFoundError:
pytest.skip("main module dependencies are not available", allow_module_level=True)
import job_handlers # type: ignore
import job_system # type: ignore
main = importlib.import_module("main")
API_KEY_HEADERS = {"X-API-Key": "your-secret-key-here"}
def setup_function():
job_system.reset_job_manager()
main._handlers_registered = False
def _completion_payload():
return {
"prefix": "hello",
@@ -32,7 +37,6 @@ def _completion_payload():
def test_cancel_endpoint_cancels_running_task(monkeypatch):
main.ACTIVE_COMPLETIONS.clear()
started = threading.Event()
cancelled = threading.Event()
@@ -45,21 +49,21 @@ def test_cancel_endpoint_cancels_running_task(monkeypatch):
cancelled.set()
raise
monkeypatch.setattr(main, "call_ollama", fake_call_ollama)
monkeypatch.setattr(main, "build_completion_prompts", lambda *a, **k: ("system", "user"))
monkeypatch.setattr(main, "prepare_prompt_context", lambda *a, **k: ("prefix", "suffix"))
monkeypatch.setattr(job_handlers, "call_ollama", fake_call_ollama)
request_id = "req-cancel-1"
with TestClient(main.app) as client:
request_id = "req-cancel-1"
completion_headers = {**API_KEY_HEADERS, "X-Request-Id": request_id}
response_box = {}
def send_completion():
response_box["response"] = client.post(
with client.stream(
"POST",
"/v1/completions",
headers=completion_headers,
headers={**API_KEY_HEADERS, "X-Request-Id": request_id},
json=_completion_payload(),
)
) as response:
response_box["status_code"] = response.status_code
response_box["body"] = "".join(response.iter_text())
completion_thread = threading.Thread(target=send_completion, daemon=True)
completion_thread.start()
@@ -77,16 +81,10 @@ def test_cancel_endpoint_cancels_running_task(monkeypatch):
completion_thread.join(timeout=5.0)
assert not completion_thread.is_alive()
assert cancelled.wait(timeout=2.0)
completion_response = response_box["response"]
# 499 = client disconnected (TestClient timeout during cancel)
assert completion_response.status_code in (200, 499)
if completion_response.status_code == 200:
assert completion_response.json()["cancelled"] is True
assert "event: cancelled" in response_box["body"]
def test_cancel_not_found():
main.ACTIVE_COMPLETIONS.clear()
with TestClient(main.app) as client:
response = client.post(
"/v1/completions/cancel",
@@ -95,27 +93,3 @@ def test_cancel_not_found():
)
assert response.status_code == 200
assert response.json() == {"cancelled": False, "status": "not_found"}
def test_completion_normal_flow(monkeypatch):
main.ACTIVE_COMPLETIONS.clear()
async def fake_call_ollama(*args, **kwargs):
return {"content": "completion text", "think": ""}
monkeypatch.setattr(main, "call_ollama", fake_call_ollama)
monkeypatch.setattr(main, "build_completion_prompts", lambda *a, **k: ("system", "user"))
monkeypatch.setattr(main, "prepare_prompt_context", lambda *a, **k: ("prefix", "suffix"))
with TestClient(main.app) as client:
response = client.post(
"/v1/completions",
headers=API_KEY_HEADERS,
json=_completion_payload(),
)
assert response.status_code == 200
data = response.json()
assert data["content"] == "completion text"
assert data["request_id"] is not None
assert main.ACTIVE_COMPLETIONS == {}
+54 -239
View File
@@ -1,35 +1,29 @@
import base64
import importlib
import os
import sys
import base64
import types
import pytest
from unittest.mock import MagicMock
from pathlib import Path
from fastapi.testclient import TestClient
CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
BACKEND_DIR = os.path.abspath(os.path.join(CURRENT_DIR, ".."))
if BACKEND_DIR not in sys.path:
sys.path.insert(0, BACKEND_DIR)
os.environ["JOB_BACKEND"] = "memory"
if "tts_asr" not in sys.modules:
fake_tts_asr = types.ModuleType("tts_asr")
fake_tts_asr.register_tts_asr_routes = lambda app: None
sys.modules["tts_asr"] = fake_tts_asr
CURRENT_DIR = Path(__file__).resolve().parent
BACKEND_DIR = CURRENT_DIR.parent
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
import main # type: ignore
import pro_completions # type: ignore
import job_handlers # type: ignore
import job_system # type: ignore
API_KEY = main.API_KEY
HEADERS = {"X-API-Key": API_KEY}
main = importlib.import_module("main")
HEADERS = {"X-API-Key": main.API_KEY}
@pytest.fixture(autouse=True)
def _clear_active_completions():
main.ACTIVE_COMPLETIONS.clear()
pro_completions.PRO_STATES.clear()
yield
main.ACTIVE_COMPLETIONS.clear()
pro_completions.PRO_STATES.clear()
def setup_function():
job_system.reset_job_manager()
main._handlers_registered = False
class DummyRequest:
@@ -51,64 +45,12 @@ def test_preview_long_text_truncated():
assert main._preview(long_text) == long_text[:80] + "..."
def test_preview_none_input():
assert main._preview(None) == ""
def test_preview_newlines_replaced():
assert main._preview("line1\nline2") == "line1\\nline2"
def test_sanitize_markdown_strips_image_markdown():
assert "![alt](image.png)" not in main._sanitize_converted_markdown(
"text with image ![alt](image.png) end"
)
def test_sanitize_markdown_strips_img_tag():
assert "<img" not in main._sanitize_converted_markdown("<img src='x.png'/>")
def test_sanitize_markdown_collapse_newlines():
assert main._sanitize_converted_markdown("a\n\n\nb\n\n\n\nc") == "a\n\nb\n\nc"
def test_sanitize_markdown_normalize_crlf():
result = main._sanitize_converted_markdown("line1\r\nline2\r\n")
assert "line1\nline2" in result
assert "\r" not in result
assert "![alt](image.png)" not in main._sanitize_converted_markdown("text ![alt](image.png)")
def test_sanitize_inline_completion_strips_prefill():
assert main.sanitize_inline_completion_content(
"系统非常适合写作",
prefill="系统",
) == "非常适合写作"
def test_sanitize_inline_completion_extracts_fim_middle():
assert main.sanitize_inline_completion_content(
"<|fim_middle|>系统非常适合写作<|end|>",
prefill="系统",
) == "非常适合写作"
def test_sanitize_inline_completion_extracts_polluted_chat_output():
polluted = (
"on new line? Prefix ends with newline already. The suffix starts with no newline. "
"We need to consider if output should end with newline? The suffix starts with no newline. "
"So we output: \"让我们一起探索 AI 的无限可能。\""
"<|end|><|start|>assistant<|channel|>final|fim_middle|>系统让我们一起探索 AI 的无限可能。"
)
assert main.sanitize_inline_completion_content(
polluted,
prefill="系统",
) == "让我们一起探索 AI 的无限可能。"
def test_get_client_ip_from_host():
req = DummyRequest(host="1.2.3.4", headers={})
assert main.get_client_ip(req) == "1.2.3.4"
assert main.sanitize_inline_completion_content("系统非常适合写作", prefill="系统") == "非常适合写作"
def test_get_client_ip_header_overrides_host():
@@ -116,191 +58,64 @@ def test_get_client_ip_header_overrides_host():
assert main.get_client_ip(req) == "5.6.7.8"
def test_get_client_ip_when_client_missing():
req = DummyRequest(host=None, headers={"X-Client-IP": "9.9.9.9"})
req.client = None
assert main.get_client_ip(req) == "9.9.9.9"
def test_post_completions_wrong_api_key_returns_401():
client = TestClient(main.app)
resp = client.post("/v1/completions", json={
"prefix": "hello", "suffix": "", "languageId": "markdown",
"model_thinking": "low", "privacy_mode": True,
})
with TestClient(main.app) as client:
resp = client.post("/v1/completions", json={
"prefix": "hello", "suffix": "", "languageId": "markdown",
"model_thinking": "low", "privacy_mode": True,
})
assert resp.status_code == 401
def test_post_completions_privacy_mode(monkeypatch):
captured = {}
def test_post_completions_returns_sse_done(monkeypatch):
async def fake_call(*args, **kwargs):
captured["kwargs"] = kwargs
return {"content": "done", "think": ""}
monkeypatch.setattr(main, "call_ollama", fake_call)
monkeypatch.setattr(main, "build_completion_prompts", lambda *a, **k: ("sys", "user", ""))
monkeypatch.setattr(main, "prepare_prompt_context", lambda *a, **k: ("p", "s"))
client = TestClient(main.app)
resp = client.post("/v1/completions", headers=HEADERS, json={
"prefix": "hello", "suffix": "", "languageId": "markdown",
"model_thinking": "low", "privacy_mode": True,
})
assert resp.status_code == 200
data = resp.json()
assert data.get("content") == "done"
# enable_thinking removed in OpenAI-compatible rewrite
assert captured["kwargs"]["thinking"] == "low"
def test_old_post_pro_stream_returns_404():
client = TestClient(main.app)
resp = client.post("/v1/pro/completions/stream", headers=HEADERS, json={
"prefix": "hello",
"suffix": "",
"languageId": "markdown",
"model_thinking": "high",
"privacy_mode": True,
})
assert resp.status_code == 404
def test_post_pro_completion_returns_sse_and_status(monkeypatch):
captured = {}
async def fake_stream_events(*args, **kwargs):
captured["kwargs"] = kwargs
yield "thinking", ""
yield "content", "深度"
yield "content", "回答"
monkeypatch.setattr(pro_completions, "stream_ollama_events", fake_stream_events)
client = TestClient(main.app)
with client.stream("POST", "/v1/pro/completions", headers=HEADERS, json={
"prefix": "hello",
"suffix": "",
"languageId": "markdown",
"instruction": "expand",
"pro_thinking": "high",
"privacy_mode": True,
}) as resp:
assert resp.status_code == 200
body = "".join(resp.iter_text())
return {"content": "系统done", "think": ""}
monkeypatch.setattr(job_handlers, "call_ollama", fake_call)
with TestClient(main.app) as client:
with client.stream("POST", "/v1/completions", headers=HEADERS, json={
"prefix": "hello", "suffix": "", "languageId": "markdown",
"model_thinking": "low", "privacy_mode": True,
}) as resp:
assert resp.status_code == 200
body = "".join(resp.iter_text())
assert "event: queued" in body
assert "event: started" in body
assert "event: thinking" in body
assert "event: chunk" in body
assert "event: result" in body
assert "event: done" in body
assert "深度" in body
assert "回答" in body
assert captured["kwargs"]["use_pro_model"] is True
assert captured["kwargs"]["thinking"] == "high"
request_id = next(iter(pro_completions.PRO_STATES))
status_resp = client.get(f"/v1/pro/completions/status/{request_id}", headers=HEADERS)
assert status_resp.status_code == 200
assert status_resp.json()["status"] == "done"
assert main.ACTIVE_COMPLETIONS == {}
def test_post_ocr_mocked(monkeypatch):
async def fake_ocr(*args, **kwargs):
return "OCR result text"
monkeypatch.setattr(main, "call_vlm_ocr", fake_ocr)
client = TestClient(main.app)
monkeypatch.setattr(job_handlers, "call_vlm_ocr", fake_ocr)
img_b64 = base64.b64encode(b"pretend image data").decode()
resp = client.post("/v1/ocr", headers=HEADERS, json={
"image": img_b64, "filename": "test.jpg", "language": "auto",
})
assert resp.status_code == 200
j = resp.json()
assert j["text"] == "OCR result text"
assert j["filename"] == "test.jpg"
def test_post_ocr_invalid_base64_returns_500():
client = TestClient(main.app)
resp = client.post("/v1/ocr", headers=HEADERS, json={
"image": "not-base64!!!", "filename": "test.jpg",
})
assert resp.status_code == 500
with TestClient(main.app) as client:
with client.stream("POST", "/v1/ocr", headers=HEADERS, json={
"image": img_b64, "filename": "test.jpg", "language": "auto",
}) as resp:
assert resp.status_code == 200
body = "".join(resp.iter_text())
assert "OCR result text" in body
def test_post_convert_txt_returns_markdown():
client = TestClient(main.app)
content = base64.b64encode(b"hello world").decode()
resp = client.post("/v1/convert", headers=HEADERS, json={
"file": content, "filename": "sample.txt",
})
assert resp.status_code == 200
j = resp.json()
assert j["markdown"] == "hello world"
assert j["filename"] == "sample.txt"
with TestClient(main.app) as client:
with client.stream("POST", "/v1/convert", headers=HEADERS, json={
"file": content, "filename": "sample.txt",
}) as resp:
assert resp.status_code == 200
body = "".join(resp.iter_text())
assert "hello world" in body
def test_post_convert_unsupported_extension_returns_500():
client = TestClient(main.app)
content = base64.b64encode(b"data").decode()
resp = client.post("/v1/convert", headers=HEADERS, json={
"file": content, "filename": "sample.xlsx",
})
with TestClient(main.app) as client:
resp = client.post("/v1/convert", headers=HEADERS, json={
"file": content, "filename": "sample.xlsx",
})
assert resp.status_code == 500
assert "仅支持" in resp.json()["error"]
def test_post_convert_docx_with_mocked_markitdown(monkeypatch):
class FakeResult:
text_content = "markdown from docx"
class FakeMD:
def convert(self, path):
return FakeResult()
monkeypatch.setattr(main, "_get_markitdown", lambda: FakeMD())
client = TestClient(main.app)
content = base64.b64encode(b"docx content").decode()
resp = client.post("/v1/convert", headers=HEADERS, json={
"file": content, "filename": "sample.docx",
})
assert resp.status_code == 200
j = resp.json()
assert j["markdown"] == "markdown from docx"
def test_post_cancel_non_existent_returns_not_found():
client = TestClient(main.app)
resp = client.post("/v1/completions/cancel", headers=HEADERS, json={
"request_id": "non-existent", "reason": "abort",
})
assert resp.status_code == 200
data = resp.json()
assert data["cancelled"] is False
assert data["status"] == "not_found"
def test_post_cancel_wrong_api_key_returns_401():
client = TestClient(main.app)
resp = client.post("/v1/completions/cancel", json={
"request_id": "id", "reason": "abort",
})
assert resp.status_code == 401
def test_post_cancel_already_done(monkeypatch):
main.ACTIVE_COMPLETIONS.clear()
# Create a mock task that appears done
mock_task = MagicMock()
mock_task.done.return_value = True
mock_task.cancel = MagicMock()
main.ACTIVE_COMPLETIONS["done-id"] = mock_task
client = TestClient(main.app)
resp = client.post("/v1/completions/cancel", headers=HEADERS, json={
"request_id": "done-id", "reason": "abort",
})
assert resp.status_code == 200
data = resp.json()
assert data["cancelled"] is False
assert data["status"] == "already_done"
main.ACTIVE_COMPLETIONS.clear()
+41 -71
View File
@@ -1,28 +1,30 @@
import importlib
import os
import sys
import types
import asyncio
import threading
from pathlib import Path
from fastapi.testclient import TestClient
os.environ["JOB_BACKEND"] = "memory"
CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
BACKEND_DIR = os.path.abspath(os.path.join(CURRENT_DIR, ".."))
if BACKEND_DIR not in sys.path:
sys.path.insert(0, BACKEND_DIR)
BACKEND_DIR = Path(__file__).resolve().parents[1]
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
if "tts_asr" not in sys.modules:
fake_tts_asr = types.ModuleType("tts_asr")
fake_tts_asr.register_tts_asr_routes = lambda app: None
sys.modules["tts_asr"] = fake_tts_asr
import main # type: ignore
import pro_completions # type: ignore
import job_handlers # type: ignore
import job_system # type: ignore
import prompt # type: ignore
main = importlib.import_module("main")
HEADERS = {"X-API-Key": main.API_KEY}
def setup_function():
job_system.reset_job_manager()
main._handlers_registered = False
def _payload():
return {
"prefix": "Before",
@@ -34,84 +36,52 @@ def _payload():
}
def setup_function():
pro_completions.PRO_STATES.clear()
def teardown_function():
pro_completions.PRO_STATES.clear()
def test_pro_queue_full_returns_429(monkeypatch):
monkeypatch.setattr(pro_completions, "PRO_QUEUE_MAX_SIZE", 0)
client = TestClient(main.app)
response = client.post("/v1/pro/completions", headers=HEADERS, json=_payload())
async def fake_queue_job(*args, **kwargs):
raise job_system.QueueFullError("pro_completion queue is full")
monkeypatch.setattr(main, "_queue_job", fake_queue_job)
with TestClient(main.app) as client:
response = client.post("/v1/pro/completions", headers=HEADERS, json=_payload())
assert response.status_code == 429
assert response.json()["error"] == "PRO queue is full"
def test_pro_status_missing_returns_404():
client = TestClient(main.app)
response = client.get("/v1/pro/completions/status/missing", headers=HEADERS)
with TestClient(main.app) as client:
response = client.get("/v1/pro/completions/status/missing", headers=HEADERS)
assert response.status_code == 404
def test_pro_prompt_uses_pro_specific_instruction():
system_prompt, user_prompt = pro_completions._build_pro_prompts(
system_prompt, user_prompt = prompt.build_pro_completion_prompts(
prefix="欢迎使用 LLM-IN-TEXT\n\n即时可用的 LLM 系统",
suffix="",
language_id="markdown",
instruction="",
pro_thinking="high",
pro_thinking_level="high",
)
combined = f"{system_prompt}\n{user_prompt}".lower()
assert "[pro] model for llm-in-text" in combined
assert "pro_mode: true" in combined
assert "pro_thinking_level: high" in combined
assert "long paragraphs or section-level output are allowed" in combined
assert "highest priority" in combined
assert "never copy tags to output" in combined
assert "write only the markdown that belongs at the cursor" not in combined
assert "continue the markdown naturally" in combined
def test_pro_cancel_waits_for_stream_cleanup(monkeypatch):
started = threading.Event()
cleaned = threading.Event()
def test_pro_stream_returns_standard_events(monkeypatch):
async def fake_stream_events(*args, **kwargs):
started.set()
try:
yield "thinking", ""
while True:
await asyncio.sleep(0.05)
finally:
cleaned.set()
monkeypatch.setattr(pro_completions, "stream_ollama_events", fake_stream_events)
request_id = "pro-cancel-cleanup"
headers = {**HEADERS, "X-Request-Id": request_id}
response_box = {}
yield "thinking", ""
yield "content", "深度"
yield "content", "回答"
monkeypatch.setattr(job_handlers, "stream_ollama_events", fake_stream_events)
with TestClient(main.app) as client:
def send_stream():
with client.stream("POST", "/v1/pro/completions", headers=headers, json=_payload()) as response:
response_box["status_code"] = response.status_code
response_box["body"] = "".join(response.iter_text())
with client.stream("POST", "/v1/pro/completions", headers=HEADERS, json=_payload()) as resp:
assert resp.status_code == 200
body = "".join(resp.iter_text())
stream_thread = threading.Thread(target=send_stream, daemon=True)
stream_thread.start()
assert started.wait(timeout=2.0)
cancel_response = client.post(
"/v1/pro/completions/cancel",
headers=HEADERS,
json={"request_id": request_id, "reason": "test"},
)
assert cancel_response.status_code == 200
assert cancel_response.json() == {"cancelled": True, "status": "ok"}
assert cleaned.wait(timeout=2.0)
stream_thread.join(timeout=5.0)
assert not stream_thread.is_alive()
assert "event: queued" in body
assert "event: started" in body
assert "event: progress" in body
assert "event: result" in body
assert "event: done" in body
assert "深度" in body
assert "回答" in body
+44 -48
View File
@@ -40,7 +40,8 @@ try:
except Exception as e: # pragma: no cover
logger.debug("modelscope import failed (optional): %s", e)
router = APIRouter()
meta_router = APIRouter()
generation_router = APIRouter()
# Global model instances
_tts_model: Optional["Qwen3TTSModel"] = None
@@ -314,7 +315,7 @@ def _ensure_align_model():
return _align_model
@router.get("/status", response_model=ModelStatus)
@meta_router.get("/status", response_model=ModelStatus)
async def get_status():
"""获取模型状态"""
return ModelStatus(
@@ -324,7 +325,7 @@ async def get_status():
)
@router.get("/config")
@meta_router.get("/config")
async def get_config():
"""获取配置信息"""
return {
@@ -340,7 +341,7 @@ async def get_config():
}
@router.post("/warmup")
@meta_router.post("/warmup")
async def warmup_models():
"""手动触发模型预热"""
await _warmup_tts()
@@ -355,39 +356,34 @@ async def warmup_models():
}
@router.post("/tts", response_model=TTSResponse)
async def tts_endpoint(req: TTSRequest):
"""TTS 文字转语音端点"""
async def generate_tts_response(
text: str,
instruct: str = "",
speaker: str = "Vivian",
output_format: str = "wav",
) -> TTSResponse:
del speaker # current model path does not expose multi-speaker routing
del output_format # current implementation always returns wav
try:
model = _ensure_tts_model()
except Exception as e: # noqa: ANN001
raise HTTPException(status_code=500, detail=str(e))
text = req.text
instruct = req.instruct or ""
try:
# VoiceDesign 模型使用 generate_voice_design 方法
wavs, sr = model.generate_voice_design( # type: ignore
text=text,
language="Chinese",
instruct=instruct,
instruct=instruct or "",
)
except Exception as e: # noqa: ANN001
logger.exception("TTS 推理失败")
raise HTTPException(status_code=500, detail=f"TTS 推理失败: {e}")
# Get first audio data
wav_data = wavs[0] if isinstance(wavs, (list, tuple)) else wavs
# Convert to numpy array
if hasattr(wav_data, 'numpy'): # type: ignore
wav_data = wav_data.cpu().numpy() # type: ignore
wav_data = np.asarray(wav_data, dtype=np.float32)
logger.debug("wav_data shape: %s, dtype: %s, sr: %s", wav_data.shape, wav_data.dtype, sr)
# Encode WAV to memory
tmp_path = None
try:
import soundfile as sf # type: ignore
@@ -404,22 +400,15 @@ async def tts_endpoint(req: TTSRequest):
if tmp_path and os.path.exists(tmp_path): # noqa: SIM201
try:
os.unlink(tmp_path)
except Exception as e: # noqa: ANN001
except Exception:
pass
duration_ms = int(len(wav_data) / sr * 1000) if sr > 0 else 0
audio_base64 = base64.b64encode(audio_bytes).decode("utf-8")
return TTSResponse(
audio_base64=audio_base64,
format="wav",
duration_ms=duration_ms,
)
return TTSResponse(audio_base64=audio_base64, format="wav", duration_ms=duration_ms)
@router.post("/asr", response_model=ASRResponse)
async def asr_endpoint(req: ASRRequest):
"""语音识别端点(非流式)"""
async def generate_asr_response(audio_bytes: bytes, language: Optional[str] = "zh-CN") -> ASRResponse:
if Qwen3ASRModel is None:
raise HTTPException(status_code=501, detail="mlx_audio 未安装,ASR 功能不可用")
@@ -429,10 +418,6 @@ async def asr_endpoint(req: ASRRequest):
raise HTTPException(status_code=500, detail=f"ASR 模型加载失败: {e}")
try:
# Decode base64 audio to WAV bytes
audio_bytes = base64.b64decode(req.audio_base64)
# Load WAV file and convert to 16kHz mono numpy array
wav_buffer = io.BytesIO(audio_bytes)
with wave.open(wav_buffer, 'rb') as wf: # noqa: SIM115
n_channels = wf.getnchannels()
@@ -443,11 +428,9 @@ async def asr_endpoint(req: ASRRequest):
raw_data = wf.readframes(n_frames)
audio_array = np.frombuffer(raw_data, dtype=np.int16 if sampwidth == 2 else np.float32)
# Convert to mono
if n_channels > 1:
audio_array = np.mean(audio_array.reshape(-1, n_channels), axis=1)
# Resample to 16kHz if needed
if framerate != 16000:
try:
import scipy.signal as signal # type: ignore
@@ -457,34 +440,47 @@ async def asr_endpoint(req: ASRRequest):
except Exception as e2: # noqa: ANN001
logger.warning("重采样失败,使用原始音频: %s", e2)
# Convert to float32 normalized
if audio_array.dtype == np.int16:
audio_array = audio_array.astype(np.float32) / 32768.0
# Run ASR inference (non-streaming)
result = model.generate( # type: ignore
audio_array,
language=req.language if req.language else None,
language=language if language else None,
)
# Extract text and detected language from result (STTOutput)
recognized_text = getattr(result, 'text', str(result)) if hasattr(result, 'text') else str(result)
detected_lang = getattr(result, 'language', req.language or "zh-CN")
# If language is a list (from segments), take the first one
detected_lang = getattr(result, 'language', language or "zh-CN")
if isinstance(detected_lang, list) and len(detected_lang) > 0:
detected_lang = detected_lang[0]
return ASRResponse(
text=recognized_text,
language=str(detected_lang),
)
return ASRResponse(text=recognized_text, language=str(detected_lang))
except HTTPException:
raise
except Exception as e: # noqa: ANN001
logger.exception("ASR 推理失败")
raise HTTPException(status_code=500, detail=f"ASR 推理失败: {e}")
def register_tts_asr_routes(app):
@generation_router.post("/tts", response_model=TTSResponse)
async def tts_endpoint(req: TTSRequest):
"""TTS 文字转语音端点"""
return await generate_tts_response(
text=req.text,
instruct=req.instruct or "",
speaker=req.speaker,
output_format=req.format,
)
@generation_router.post("/asr", response_model=ASRResponse)
async def asr_endpoint(req: ASRRequest):
"""语音识别端点(非流式)"""
audio_bytes = base64.b64decode(req.audio_base64)
return await generate_asr_response(audio_bytes, req.language if req.language else None)
def register_tts_asr_routes(app, include_generation_routes: bool = True):
"""注册 TTS/ASR 路由到 FastAPI 应用"""
app.include_router(router, prefix="/v1/tts-asr")
app.include_router(meta_router, prefix="/v1/tts-asr")
if include_generation_routes:
app.include_router(generation_router, prefix="/v1/tts-asr")
+39
View File
@@ -0,0 +1,39 @@
import asyncio
import logging
from job_handlers import (
asr_handler,
completion_handler,
compress_handler,
convert_handler,
ocr_handler,
pro_completion_handler,
tts_handler,
)
from job_system import RedisJobManager, RedisWorker
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s - %(message)s",
)
logger = logging.getLogger("worker")
async def main() -> None:
manager = RedisJobManager()
manager.register_handler("completion", completion_handler)
manager.register_handler("pro_completion", pro_completion_handler)
manager.register_handler("compress", compress_handler)
manager.register_handler("ocr", ocr_handler)
manager.register_handler("convert", convert_handler)
manager.register_handler("tts", tts_handler)
manager.register_handler("asr", asr_handler)
worker = RedisWorker(manager)
try:
await worker.run_forever()
finally:
await manager.close()
if __name__ == "__main__":
asyncio.run(main())
+48
View File
@@ -0,0 +1,48 @@
version: "3.9"
services:
redis:
image: redis:7-alpine
ports:
- "6379:6379"
command: ["redis-server", "--appendonly", "yes"]
volumes:
- redis-data:/data
api:
image: python:3.11-slim
working_dir: /app
command: sh -c "pip install -r backend/requirements.txt && python backend/main.py"
env_file:
- backend/.env
environment:
JOB_BACKEND: redis
REDIS_URL: redis://redis:6379/0
JOB_SHARED_TEMP_DIR: /shared-jobs
volumes:
- .:/app
- job-shared:/shared-jobs
depends_on:
- redis
ports:
- "8001:8001"
worker:
image: python:3.11-slim
working_dir: /app
command: sh -c "pip install -r backend/requirements.txt && python backend/worker.py"
env_file:
- backend/.env
environment:
JOB_BACKEND: redis
REDIS_URL: redis://redis:6379/0
JOB_SHARED_TEMP_DIR: /shared-jobs
volumes:
- .:/app
- job-shared:/shared-jobs
depends_on:
- redis
volumes:
redis-data:
job-shared:
@@ -1,142 +0,0 @@
# Announcing Telemetry Inspector
There's a lot of questions from community asking that how can they know what plugins are enabled.
From Milkdown@7.2, we've added telemetries for milkdown, it can be available by inspectors.
With this API, you can inspect editor inner status.
You can even use visualizer to visualize the data. We create a simple example on [our playground](/playground).
![Milkdown Inspector](/blogs/announcing-telemetry-inspector/milkdown-inspector.gif)
## Get Started
Inspector will be a top-level API in Milkdown. You can use it like this:
```ts
import { Editor } from "@milkdown/core";
import { Telemetry } from "@milkdown/ctx";
const editor = await Editor.make()
// Inspector is disabled by default considering performance. You need to enable it manually.
.enableInspector()
// ...
.create();
const telemetry: Telemetry[] = editor.inspect();
```
The `Telemetry` interface will have the following fields:
```ts
interface Telemetry {
// User defined information for the plugin.
metadata: Meta;
// The slices and their current value defined by the plugin.
injectedSlices: { name: string; value: unknown }[];
// The slices and their current value consumed by the plugin.
consumedSlices: { name: string; value: unknown }[];
// The timers and their duration defined by the plugin.
recordedTimers: { name: string; duration: number; status: TimerStatus }[];
// The timers and their duration consumed by the plugin.
// Generally, the plugin will wait for them.
waitTimers: { name: string; duration: number; status: TimerStatus }[];
}
type TimerStatus = "pending" | "resolved" | "rejected";
interface Meta {
displayName: string;
description?: string;
package: string;
group?: string;
additional?: Record<string, any>;
}
```
For every plugin, it'll have a telemetry if it has metadata declared.
With the data, you'll know the sequence of the plugins loaded, the slices and timers they defined and consumed.
For example:
```ts
[
{
metadata: {
displayName: "Config",
package: "@milkdown/core",
group: "System",
},
injectedSlices: [],
consumedSlices: [
/* ... */
],
recordedTimers: [
{
name: "ConfigReady",
duration: 3,
status: "resolved",
},
],
waitTimers: [],
},
{
metadata: {
displayName: "Init",
package: "@milkdown/core",
group: "System",
},
injectedSlices: [],
consumedSlices: [
/* ... */
],
recordedTimers: [
{
name: "InitReady",
duration: 5,
status: "resolved",
},
],
waitTimers: [
{
name: "ConfigReady",
duration: 5,
status: "resolved",
},
],
},
];
```
From above information, we can know that the `Init` plugin wait for `Config` plugin to be ready.
We can build a sequence diagram from the data.
![Timer Sequence](/blogs/announcing-telemetry-inspector/timer-sequence.gif)
## Add Metadata for Plugin
For plugin maintainers, you can add metadata to your plugin to make it more friendly to the inspector.
```ts
import { MilkdownPlugin } from "@milkdown/ctx";
const yourMilkdownPlugin: MilkdownPlugin = () => {
/* your implementation */
};
yourMilkdownPlugin.metadata = {
displayName: "Your Plugin",
package: "your-plugin-package",
description: "Your plugin description",
group: "If you have a lot of plugins in your package, you can group them.",
addtitional: {
/* You can add any additional information here. */
version: "1.0.0",
authror: "Mike",
},
};
```
With metadata, your plugin will report telemetry correctly to the inspector.
@@ -1,247 +0,0 @@
# Build Your Own Milkdown Copilot
OpenAI introduced ChatGPT in 2020, which is a chatbot that can generate natural language responses to user input.
Which brings us a new way to interact with devices and applications.
Nowadays, there are more and more tools that are powered by AI. Such as Notion, GitHub and even Microsoft 365.
Since OpenAI also released the [API](https://openai.com/blog/openai-api) of it. And Milkdown is composed by plugins.
I think it's possible to build a Milkdown Copilot Plugin that can help you write documents. So I did it.
Let's see the result.
![Milkdown Copilot](/blogs/build-your-own-milkdown-copilot/milkdown-copilot.gif)
Looks cool, right? But how does it work? I'll explain it in the following sections.
## Prepare a Backend
**Before we start, you need to have a OpenAI API Key.** You'll need to get one [here](https://platform.openai.com/account/api-keys).
I'll not explain how to get it. You can find the details in their [official docs](https://platform.openai.com/).
I'll use Node.js to build the backend. You can use any language you like.
The backend is very simple. It just calls the OpenAI API and returns the result.
```ts
import { Configuration, OpenAIApi } from "openai";
const configuration = new Configuration({
// Get your API key from env variable
apiKey: process.env.OPENAPI_KEY,
});
const openai = new OpenAIApi(configuration);
export const handler = async (req, res, next) => {
if (req.path === "/api/copilot" && req.method === "POST") {
const buffers = [];
// Get the body of the request.
const body = JSON.parse(req.body);
// Get prompt from the body.
const { prompt } = body;
const completion = await openai.createCompletion({
// Pick a model you like
model: "text-davinci-003",
prompt,
});
const hint = completion.data.choices[0].text;
return res.end(JSON.stringify({ hint }));
}
next();
return;
};
```
We watch the `/api/copilot` route and call the OpenAI API when we receive a POST request.
The post request should contain a `prompt` field which is the text that we want to complete.
To call our API, we just need one single helper in browser environment:
```ts
async function fetchAIHint(prompt: string) {
const data: Record<string, string> = { prompt };
const response = await fetch("/api/copilot", {
method: "POST",
body: JSON.stringify(data),
});
const res = (await response.json()) as { hint: string };
return res.hint;
}
```
## Build a Milkdown Plugin
Now let's focus on the Milkdown Copilot Plugin.
Basically I want to implement two things:
1. When the user types `<Enter>` or `<Space>`, they will get a hint from the copilot.
2. When the user types `<Tab>`, they will apply the content from the hint to the editor.
### Overview
To build a bridge between the copilot and the editor,
we can build a prosemirror plugin and use the `onKeyDown` hook to listen to the keydown event.
```ts
function keyDownHandler(ctx: Ctx, event: Event) {
if (event.key === "Enter" || event.code === "Space") {
getHint(ctx);
return;
}
if (event.key === "Tab") {
// prevent the browser from focusing on the next element.
event.preventDefault();
applyHint(ctx);
return;
}
hideHint(ctx);
}
```
When the user types `<Enter>` or `<Space>`, we will call the `getHint` function to get a hint from the copilot.
And when the user types `<Tab>`, we will call the `applyHint` function to apply the hint to the editor.
If user types other keys, we will hide the hint.
And we also need a component to render the hint. Here I choose to use a simple [widget decoration in prosemirror](https://prosemirror.net/docs/ref/#view.Decoration^widget).
```ts
function renderHint(message: string) {
const dom = document.createElement("pre");
dom.className = "copilot-hint";
dom.innerHTML = message;
return dom;
}
```
So our component looks like:
```ts
import { Plugin, PluginKey } from "@milkdown/prose/state";
import { Decoration, DecorationSet } from "@milkdown/prose/view";
import { $prose } from "@milkdown/utils";
const initialState = {
deco: DecorationSet.empty,
message: "",
};
export const copilotPluginKey = new PluginKey("milkdown-copilot");
export const copilotPlugin = $prose(
(ctx) =>
new Plugin({
key: copilotPluginKey,
props: {
handleKeyDwon(view, event) {
keydownHandler(ctx, event);
},
decorations(state) {
return copilotPluginKey.getState(state).deco;
},
},
state: {
init() {
return { ...initialState };
},
apply(tr, value, _prevState, state) {
const message = tr.getMeta(copilotPluginKey);
if (typeof message !== "string") return value;
if (message.length === 0) {
return { ...initialState };
}
const { to } = tr.selection;
const widget = Decoration.widget(to + 1, () => renderHint(message));
return {
deco: DecorationSet.create(state.doc, [widget]),
message,
};
},
},
}),
);
```
### Get Hint
To get a hint from the copilot, we need to get the text before the cursor.
```ts
function getHint(ctx: Ctx) {
const view = ctx.get(editorViewCtx);
const { state } = view;
const { tr, schema } = state;
const { from } = tr.selection;
const slice = tr.doc.slice(0, from);
const serializer = ctx.get(serializerCtx);
const doc = schema.topNodeType.createAndFill(undefined, slice.content);
if (!doc) return;
const markdown = serializer(doc);
fetchAIHint(markdown).then((hint) => {
const tr = view.state.tr;
view.dispatch(tr.setMeta(copilotPluginKey, hint));
});
}
```
1. First of all, we get the `selection` from the `state` of the editor.
2. Then we get a `slice` of the document from the start to the cursor.
3. Then we use the `serializer` to convert the slice to markdown.
4. After that, we call the `fetchAIHint` function to get a hint from the copilot.
5. Finally, we dispatch a transaction with the hint message we get to update the state of the editor.
### Hide Hint
To hide the hint, we just need to dispatch a transaction with an empty message.
```ts
function hideHint(ctx: Ctx) {
const view = ctx.get(editorViewCtx);
const { state } = view;
const { tr } = state;
view.dispatch(tr.setMeta(copilotPluginKey, ""));
}
```
### Apply Hint
Since we pass markdown to the OpenAI API. It may return a markdown snippet.
So, before we apply the hint to the editor, we need to convert the markdown snippet to prosemirror node.
```ts
function applyHint(ctx: Ctx) {
const view = ctx.get(editorViewCtx);
const { state } = view;
const { tr, schema } = state;
const { message } = copilotPluginKey.getState(state);
const parser = ctx.get(parserCtx);
const slice = parser(message);
const dom = DOMSerializer.fromSchema(schema).serializeFragment(slice.content);
const node = DOMParser.fromSchema(schema).parseSlice(dom);
// Reset the hint since it's applied
tr.setMeta(copilotPluginKey, "")
// Replace the selection with the hint
.replaceSelection(node);
view.dispatch(tr);
}
```
1. First of all, we get the hint message from the state of the editor.
2. Then we use the `parser` to convert the markdown snippet to prosemirror node.
3. Finally, we dispatch a transaction to replace the selection with the hint.
## Conclusion
In this article, we have built a really simple Copilot plugin for Milkdown.
The plugin is not perfect, but it's a good start to help you build your own.
The source code is available on [Milkdown/examples/vanilla-openapi](https://github.com/Milkdown/examples/tree/main/vanilla-openai).
I hope it can give you some inspiration.
@@ -1,151 +0,0 @@
# Introducing Milkdown@7
It's been almost one year since the release of [milkdown](https://milkdown.dev) V6.
It helped a lot of users to build their own markdown based applications.
It has 13k downloads per month and I feel so grateful that users like that.
However, we noticed that there're some problems cannot be resolved if we don't make a new major version.
What big changes did we made? I'll introduce them to you in this blog.
## TL;DR
- The editor becomes a first-class headless component.
- Factory plugins are fully replaced by **composable plugins**.
- Runtime plugin toggling is supported.
- Universal widget plugins.
- Better Vue and React support.
- API documentation is provided.
## Why Headless?
In the past, milkdown had a lot of internal styles to make sure the editor can work out of box and the themes are easy to create.
However, I found it limits the users to design their own editor.
Even worse, if you have an well designed application,
it is really hard to keep the style of the milkdown editor same with the rest of the application.
You'll need to override lots of styles everywhere.
It stops a log of users from using milkdown.
If we think about why users need an editor,
the most important thing is always the functionality of the editor.
Users just want a component that can provide smooth editing experience.
Style will always be the second thing.
So, why not remove all the internal styles and make the editor a headless component?
The users can easily integrate the editor into their own application.
They can use their own styles and even use their own components to render the editor.
We just care about the functionality of the editor. Make sure it works well.
## Composable Plugins
Although the composable plugins have been existed in milkdown for a long time,
we use factory plugins to create most of the official plugins in V6.
But, the problem is that factory plugins limit the possibility of the plugins.
The factory plugins handle a bunch of complex logic and it is hard to extend.
So for users who want to create a plugin in a easy way, they must follow the factory plugin's way.
```ts
const nodePlugin = createPlugin(() => ({
id: 'node',
schema: someSchema,
inputRules: someInputRules
commands: someCommands
}))
```
See? You can define a lot of things inside the factory plugin.
But if you want to use some part of them in another plugin, it's really hard to do that.
However, the milkdown's plugin system is designed to be flexible and composable.
We want to let users to control the data flow entirely.
So, we decided to remove all the factory plugins and use composable plugins to replace them.
The composable plugins can keep the atomicity of the plugins and make the plugin system more flexible.
They also make the plugin system easier to maintain.
```ts
const nodeSchema = $node("node", someSchema);
const nodeInputRules = $inputRules(someInputRules);
const nodeCommands = $commands(someCommands);
```
If you want to reuse them, it also will be very easy.
```ts
const anotherCommand = $commands(() => {
return setBlockType(nodeSchema.type());
});
```
## Runtime Plugin Toggling
In the past, once you register a plugin, you cannot remove it.
In V7, we support runtime plugin toggling by providing two new API: `editor.remove` and `editor.removeConfig`.
They can let users remove the plugins and configs at runtime.
```ts
import { Editor } from "@milkdown/core";
import { someMilkdownPlugin } from "some-milkdown-plugin";
const editor = await Editor.config(configForPlugin)
.use(someMilkdownPlugin)
.create();
// remove plugin
await editor.remove(someMilkdownPlugin);
// remove config
editor.removeConfig(configForPlugin);
// add another plugin
editor.use(anotherMilkdownPlugin);
// Recreate the editor to apply changes.
await editor.create();
```
Also, if you call the `editor.create` method after the editor is created,
it will recreate the editor and apply all the changes.
## Universal Widget Plugins
We have 4 official widget plugins in V6: _slash_, _tooltip_, _block_ and _menu_.
They are all well designed and easy to use.
But if you want to customize them, what you can do is really limited.
Also, it's hard to reuse their logic even if you want to create something similar to them.
For example, if you want to create a mention plugin which will show a list of users when you type `@`,
you need to create a new plugin from scratch.
So, in V7, we make _slash_, _tooltip_ and _block_ plugins universal.
You can use them to build you features easily.
For example, if you want to create a mention plugin, you can use the new slash plugin to do that.
Another example is that you can also create tooltips for different types of nodes.
Display a tooltip with input when you focus on an image node, or display a tooltip with buttons when you select some text.
What about the _menu_ plugin? We removed it because we think it's easy to create a menu plugin by yourself.
We've already done that in the [official playground](https://milkdown.dev/playground).
And, trust me, [it won't need much code](https://github.com/milkdown/website/blob/main/src/component/Playground/Milkdown/index.tsx#L57).
## Better Vue and React Support
Thanks to the [Saul-Mirone/prosemirror-adapter project](https://github.com/Saul-Mirone/prosemirror-adapter).
In milkdown V7. We allow users to use vue and react to render lots of parts of the editor.
For example, you can use them to render your own code block, drag handle or even small icons.
- React Example: [![Open in StackBlitz](https://developer.stackblitz.com/img/open_in_stackblitz.svg)](https://stackblitz.com/github/Milkdown/examples/tree/main/react-custom-component)
- Vue Example: [![Open in StackBlitz](https://developer.stackblitz.com/img/open_in_stackblitz.svg)](https://stackblitz.com/github/Milkdown/examples/tree/main/vue-custom-component)
## API Documentation
What's the hardest thing to do when maintaining an open source project?
Keep the documentation up to date.
Thanks to the [marijnh/builddocs project](https://github.com/marijnh/builddocs),
we can generate the API documentation automatically from the source code.
We also redesigned the documentation website, provide a more powerful playground and lots of examples.
@@ -1,172 +0,0 @@
# Understanding Headless Slash Plugin
In the old Milkdown versions. The slash plugin can be used to display a list of commands when users type `/` in the editor.
It provides a way to insert nodes and commands into the editor, and it's really easy to use.
![legacy slash plugin](/blogs/understanding-headless-slash-plugin/legacy-slash-plugin.png)
However, it's hard to extend the slash plugin to support more commands, or if you want to change the UI of the slash plugin, you have to rewrite the whole plugin.
But, write a new plugin is always a hard work. You have to understand a lot of context and APIs of both ProseMirror and Milkdown.
## User Story
So, why don't we provide the slash plugin as a headless plugin?
In most cases, developers just want to make sure that when users type a special character, a dropdown menu will be displayed.
But the trigger character and the UI of the dropdown menu are different in different cases.
For example:
- When user type `/`, the menu contains a list of **commands**.
- When user type `:`, the menu contains a list of **emoji**.
- When user type `@`, the menu contains a list of **users**.
That's the story behind the headless slash plugin. We provide the plugin to solve a single problem: **display a dropdown menu when users input satisfy a condition**.
## How to use
In the new slash plugin, you'll need to control when to display the dropdown menu by yourself.
And you'll also need to provide the UI of the dropdown menu.
So, you'll need to create a `SlashProvider` instance.
```ts
import { slashPlugin, SlashProvider } from "@milkdown/plugin-slash";
const slashProvider = new SlashProvider({
content: YourDropdownUI,
shouldShow(this: SlashProvider, view: EditorView) {
const currentText = this.getContent(view);
if (currentText === "") {
return false;
}
// Display the menu if the last character is `/`.
if (currentText.endsWith("/")) {
return true;
}
return false;
},
});
```
Then, you can use the slash provider in your plugin view.
```ts
import { EditorState } from "@milkdown/prose/state";
import { EditorView, PluginView } from "@milkdown/prose/view";
function yourSlashView(): PluginView {
return {
update: (view: EditorView, prevState: EditorState) => {
slashProvider.update(view, prevState);
},
destroy: () => {
slashProvider.destroy();
},
};
}
```
Last, you'll need to add the slash plugin to your editor.
```ts
import { Editor } from "@milkdown/core";
import { slashFactory } from "@milkdown/plugin-slash";
const slash = slashFactory("my-slash");
Editor.make()
.config((ctx) => {
ctx.set(slash.key, {
view: slashPluginView,
});
})
.use(slash)
.create();
```
## Use with Prosemirror Adapter
If you're using milkdown with UI frameworks like React,
I recommend you to use the [Prosemirror Adapter](https://github.com/Saul-Mirone/prosemirror-adapter).
It can help you build prosemirror UI components with your favorite UI framework.
For example, if you're using React:
```tsx
import { SlashProvider } from "@milkdown/plugin-slash";
import { useInstance } from "@milkdown/react";
import { usePluginViewContext } from "@prosemirror-adapter/react";
export const DropdownMenu = () => {
const { view, prevState } = usePluginViewContext();
const slashProvider = useRef<SlashProvider>();
const divRef = useRef<HTMLDivElement>(null);
const [loading] = useInstance();
useEffect(() => {
if (!ref.current || loading) return;
slashProvider.current ??= new SlashProvider({
content: divRef.current,
// ...
});
return () => {
slashProvider.current?.destroy();
slashProvider.current = undefined;
};
}, [loading, root, setOpened, setSearch, setSelected]);
useEffect(() => {
slashProvider.current?.update(view, prevState);
});
// Add a wrapper `div` to hide the dropdown menu when initializing.
return (
<div className="hidden">
<div role="tooltip" ref={divRef}>
<h1>Hi! I'm a dropdown menu.</h1>
</div>
</div>
);
};
```
And in your editor component:
```ts
import { usePluginViewFactory } from "@prosemirror-adapter/react";
export const YourEditor = () => {
const pluginViewFactory = usePluginViewFactory();
useEditor((editor) => {
return Editor.make()
.config((ctx) => {
ctx.set(slash.key, {
view: pluginViewFactory({
component: DopdownMenu,
}),
});
})
.use(slash);
});
// ...
};
```
## Real World Example
In [milkdown playground](/playground), you can type `/` to display a dropdown menu.
![command dropdown](/blogs/understanding-headless-slash-plugin/command-dropdown.png)
You can also type `:(\S)+` (for example: `:mil`) to display a list of emojis.
![emoji dropdown](/blogs/understanding-headless-slash-plugin/emoji-dropdown.png)
You can find the source code of them in [Milkdown website](https://github.com/Milkdown/website).
I hope you enjoy the new slash plugin.
@@ -1,199 +0,0 @@
# Architecture Overview
Milkdown is built with a modular, layered architecture that provides flexibility and extensibility. This document explains the core architectural concepts and how they work together.
![0.75](/guide/milkdown-architecture.png "Milkdown Architecture")
## Core Architecture Layers
Milkdown's architecture is built upon four distinct layers, each providing specific functionality and extensibility:
### 🥛 Core Layer
The foundation of Milkdown that provides:
- Plugin loading and management system
- Core editor concepts and interfaces
- Base document model integration
- Essential utilities and helpers
### 🧇 Plugin Layer
A comprehensive collection of modular plugins that extend the editor's functionality:
- Syntax plugins (Markdown parsing, GFM, etc.)
- UI plugins (toolbar, menu, etc.)
- Feature plugins (image upload, table, etc.)
- Utility plugins (history, clipboard, etc.)
### 🍮 Component Layer
Headless UI components that serve as building blocks:
- Toolbar components
- Slash menu components
- Table components
### 🍰 Editor Layer
Ready-to-use, user-friendly editors:
- Crepe editor
- Custom editor implementations
## Architecture Benefits
This layered approach provides several key benefits:
1. **Modularity**: Each layer can be used independently
2. **Flexibility**: Mix and match components as needed
3. **Extensibility**: Create custom implementations at any layer
4. **Maintainability**: Clear separation of concerns
5. **Reusability**: Components can be shared across implementations
## Markdown Transformation
![0.75](/guide/transformer.png "Transformer")
Milkdown's transformation system handles the conversion between Markdown and the editor's internal document model:
### Parsing Process
1. Markdown text → Remark AST
2. Remark AST → ProseMirror Schema
3. Schema → ProseMirror Document
### Serialization Process
1. ProseMirror Document → ProseMirror Schema
2. Schema → Remark AST
3. Remark AST → Markdown text
This transformation system ensures:
- Accurate Markdown parsing
- Consistent document structure
- Reliable serialization
- Extensible transformation pipeline
## Context System
The Context System is a powerful state management and dependency coordination system that enables plugins to work together seamlessly.
![1.00](/guide/plugin-sequence.png "Plugin Sequence")
### Core Concepts
#### 1. Context (Ctx)
The main interface for plugins to interact with the system:
```typescript
interface Ctx {
get: <T>(slice: Slice<T>) => T;
set: <T>(slice: Slice<T>, value: T) => void;
wait: (timer: Timer) => Promise<void>;
done: (timer: Timer) => void;
inject: <T>(slice: Slice<T>, value: T) => void;
remove: <T>(slice: Slice<T>) => void;
}
```
#### 2. Slices
State containers that can be shared between plugins:
```typescript
// Create a slice with initial value and name
const themeSlice = createSlice("light", "theme");
// Use in a plugin
const themePlugin: MilkdownPlugin = (ctx) => {
return () => {
// Read current theme
const theme = ctx.get(themeSlice);
// Update theme
ctx.set(themeSlice, "dark");
// React to theme changes
ctx.watch(themeSlice, (newTheme) => {
// Handle theme change
});
};
};
```
#### 3. Timers
Dependency management system for plugin coordination:
```typescript
// Define a timer
const dataReady = createTimer("DataReady");
// Use in a plugin
const dataPlugin: MilkdownPlugin = (ctx) => {
ctx.record(dataReady);
return async () => {
// Wait for dependencies
await ctx.wait(SchemaReady);
// Do work
// ...
// Mark as ready
ctx.done(dataReady);
};
};
```
### Plugin Lifecycle
Plugins follow a consistent lifecycle pattern:
```typescript
const examplePlugin: MilkdownPlugin = (ctx) => {
// 1. Setup Phase
ctx.inject(mySlice, defaultValue);
ctx.record(myTimer);
return async () => {
// 2. Initialization Phase
await ctx.wait(RequiredTimer);
// 3. Runtime Phase
const value = ctx.get(mySlice);
ctx.set(mySlice, newValue);
// 4. Cleanup Phase
return () => {
ctx.remove(mySlice);
};
};
};
```
### Best Practices
1. **State Management**
- Use slices for shared state
- Keep state minimal and focused
- Watch for state changes when needed
2. **Dependency Management**
- Use timers for coordination
- Wait for required dependencies
- Mark completion appropriately
3. **Plugin Organization**
- Follow the lifecycle pattern
- Clean up resources properly
- Document dependencies clearly
## Next Steps
- Start to [use Crepe editor](/docs/guide/using-crepe)
- Learn more about [writing plugins](/docs/plugin/plugins-101)
- Explore [available plugins](/docs/plugin/using-plugins)
-160
View File
@@ -1,160 +0,0 @@
# Code Highlighting
Milkdown supports syntax highlighting for code blocks through the `@milkdown/plugin-highlight` plugin. This plugin provides several options for highlighting code with different syntax highlighters.
## Installation
```bash
npm install @milkdown/plugin-highlight
```
## Basic Usage
The highlight plugin requires a parser to be configured. Here's a basic example using the Shiki parser:
```typescript
import { Editor } from "@milkdown/core";
import { commonmark } from "@milkdown/preset-commonmark";
import { highlight, highlightPluginConfig } from "@milkdown/plugin-highlight";
import { createParser } from "@milkdown/plugin-highlight/shiki";
const editor = Editor.make()
.config(async (ctx) => {
const parser = await createParser({
theme: "github-light",
langs: ["javascript", "typescript", "python", "html", "css"],
});
ctx.set(highlightPluginConfig.key, { parser });
})
.use(commonmark)
.use(highlight)
.create();
```
## Available Parsers
The plugin supports multiple syntax highlighting libraries:
### Shiki
Provides high-quality syntax highlighting with VS Code themes. Learn more at [Shiki](https://shiki.style/):
```typescript
import { createParser } from "@milkdown/plugin-highlight/shiki";
const parser = await createParser({
theme: "github-light",
langs: ["javascript", "typescript", "python"],
});
ctx.set(highlightPluginConfig.key, { parser });
```
### Lowlight
Based on [highlight.js](https://highlightjs.org/), supports many languages:
```typescript
import { createParser } from "@milkdown/plugin-highlight/lowlight";
import { common } from "lowlight";
const parser = createParser({ common });
ctx.set(highlightPluginConfig.key, { parser });
```
Learn more about Lowlight at [lowlight](https://github.com/wooorm/lowlight).
### Refractor
Based on [Prism.js](https://prismjs.com/):
```typescript
import { createParser } from "@milkdown/plugin-highlight/refractor";
import { refractor } from "refractor";
const parser = createParser({ refractor });
ctx.set(highlightPluginConfig.key, { parser });
```
Learn more about Refractor at [refractor](https://github.com/wooorm/refractor).
### Sugar High
A lightweight and fast syntax highlighter. Learn more at [Sugar High](https://github.com/huozhi/sugar-high):
```typescript
import { createParser } from "@milkdown/plugin-highlight/sugar-high";
const parser = createParser();
ctx.set(highlightPluginConfig.key, { parser });
```
## Styling
The highlighted code will have CSS classes applied based on the chosen parser. You'll need to include appropriate CSS to style the highlighted tokens.
### Sugar High Classes
Sugar High uses classes like:
- `sh__token--identifier`
- `sh__token--string`
- `sh__token--keyword`
- `sh__token--sign`
- `sh__token--property`
You can style these using CSS variables:
```css
.sh__token--identifier {
color: var(--sh-identifier);
}
.sh__token--string {
color: var(--sh-string);
}
.sh__token--keyword {
color: var(--sh-keyword);
}
```
### Other Parsers
For Lowlight, Refractor, and Shiki, refer to their respective documentation for styling information.
## Example
Here's a complete example with Shiki:
```typescript
import { Editor } from "@milkdown/core";
import { commonmark } from "@milkdown/preset-commonmark";
import { highlight, highlightPluginConfig } from "@milkdown/plugin-highlight";
import { createParser } from "@milkdown/plugin-highlight/shiki";
async function createHighlightedEditor() {
const parser = await createParser({
theme: "github-light",
langs: ["javascript", "typescript", "python", "html", "css", "json"],
});
const editor = Editor.make()
.config((ctx) => {
ctx.set(highlightPluginConfig.key, { parser });
})
.use(commonmark)
.use(highlight);
await editor.create();
return editor;
}
```
With this setup, your code blocks will be automatically highlighted:
````markdown
```javascript
console.log("Hello, world!");
const greeting = (name) => `Hello, ${name}!`;
```
````
The code above will render with syntax highlighting applied to keywords, strings, and other language constructs.
@@ -1,122 +0,0 @@
# Collaborative Editing
Milkdown supports collaborative editing powered by [Y.js](https://docs.yjs.dev/).
We provide the [@milkdown/plugin-collab](/docs/api/plugin-collab) plugin to help you use milkdown with yjs easily.
This plugin includes basic collaborative editing features like:
- Sync between clients.
- Remote cursor support.
- Undo/Redo support.
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vanilla-collab"}
## Configure Plugin
First you need to install the plugin and yjs through npm:
```bash
npm install @milkdown/plugin-collab
npm install yjs y-protocols y-prosemirror
```
And you also need to choose a [provider for yjs](https://docs.yjs.dev/ecosystem/connection-provider), here we use [y-websocket](https://docs.yjs.dev/ecosystem/connection-provider/y-websocket) as an example.
After the installation, you can configure your editor:
```typescript
// ...import other plugins
import { collab, collabServiceCtx } from "@milkdown/plugin-collab";
async function setup() {
const editor = await Editor.make()
.config(nord)
.use(commonmark)
.use(collab)
.create();
const doc = new Doc();
const wsProvider = new WebsocketProvider("<YOUR_WS_HOST>", "milkdown", doc);
editor.action((ctx) => {
const collabService = ctx.get(collabServiceCtx);
collabService
// bind doc and awareness
.bindDoc(doc)
.setAwareness(wsProvider.awareness)
// connect yjs with milkdown
.connect();
});
}
```
Now your editor can support collaborative editing. Isn't it easy?
## Connect and Disconnect
You may want to control the connect status of the editor manually.
```typescript
editor.action((ctx) => {
const collabService = ctx.get(collabServiceCtx);
const doc = new Doc();
const wsProvider = new WebsocketProvider("<YOUR_WS_HOST>", "milkdown", doc);
collabService.bindDoc(doc).setAwareness(wsProvider.awareness);
document.getElementById("connect").onclick = () => {
wsProvider.connect();
collabService.connect();
};
document.getElementById("disconnect").onclick = () => {
wsProvider.disconnect();
collabService.disconnect();
};
});
```
## Default Template
By default, the editor will show a empty document. You may want to use a template to show a document.
```typescript
const template = `# Heading`;
editor.action((ctx) => {
const collabService = ctx.get(collabServiceCtx);
const doc = new Doc();
const wsProvider = new WebsocketProvider("<YOUR_WS_HOST>", "milkdown", doc);
collabService.bindDoc(doc).setAwareness(wsProvider.awareness);
wsProvider.once("synced", async (isSynced: boolean) => {
if (isSynced) {
collabService
// apply your template
.applyTemplate(markdown)
// don't forget connect
.connect();
}
});
});
```
Keep in mind that applying a template multiple times may cause some unexpected behavior, such as duplicate content.
Because of this you need to make sure **the template is applied only once**.
By default, the template will only be applied if _document get from remote server is empty_.
You can control this behavior through passing second parameter to `applyTemplate`:
```typescript
collabService
.applyTemplate(markdown, (remoteNode, templateNode) => {
// return true to apply template
})
// don't forget connect
.connect();
```
Here the nodes we get are [prosemirror nodes](https://prosemirror.net/docs/ref/#model.Node).
You should return `true` if the template should be applied, and `false` if not.
-250
View File
@@ -1,250 +0,0 @@
# Commands
Commands are a powerful way to programmatically modify editor content. The command system in Milkdown provides a flexible and type-safe way to create, manage, and execute commands.
## Command Manager
---
The command manager is the central place for handling all editor commands. It provides methods to:
- Register new commands
- Execute commands
- Chain multiple commands together
- Handle command arguments
## Run a Command
---
You can execute commands using the command manager through the editor's action system:
```typescript
import { Editor, commandsCtx } from "@milkdown/kit/core";
import {
commonmark,
toggleEmphasisCommand,
} from "@milkdown/kit/preset/commonmark";
async function setup() {
const editor = await Editor.make().use(commonmark).create();
const toggleItalic = () =>
editor.action((ctx) => {
// get command manager
const commandManager = ctx.get(commandsCtx);
// call command
commandManager.call(toggleEmphasisCommand.key);
});
// get markdown string:
$button.onClick = toggleItalic;
}
```
## Command Chaining
---
You can chain multiple commands together using the command manager's `chain` method. Commands in the chain will be executed in order until one of them returns `true`:
```typescript
import { Editor, commandsCtx } from "@milkdown/kit/core";
import {
commonmark,
toggleEmphasisCommand,
toggleStrongCommand,
} from "@milkdown/kit/preset/commonmark";
const editor = await Editor.make().use(commonmark).create();
editor.action((ctx) => {
const commandManager = ctx.get(commandsCtx);
// Chain multiple commands
commandManager
.chain()
.pipe(toggleEmphasisCommand.key) // Try to toggle emphasis
.pipe(toggleStrongCommand.key) // If emphasis fails, try to toggle strong
.run();
});
```
You can also mix inline commands with registered commands:
```typescript
import { chainCommands } from "@milkdown/prose/commands";
editor.action((ctx) => {
const commandManager = ctx.get(commandsCtx);
commandManager
.chain()
.inline(someInlineCommand) // Add an inline command
.pipe(toggleEmphasisCommand.key) // Add a registered command
.run();
});
```
## Create a Command
---
To create a command, use the `$command` utility from `@milkdown/utils`. Commands should be [prosemirror commands](https://prosemirror.net/docs/guide/#commands).
### Example: Command without argument
```typescript
import { Editor } from "@milkdown/kit/core";
import { blockquoteSchema } from "@milkdown/kit/preset/commonmark";
import { wrapIn } from "@milkdown/kit/prose/commands";
import { $command, callCommand } from "@milkdown/kit/utils";
const wrapInBlockquoteCommand = $command(
"WrapInBlockquote",
(ctx) => () => wrapIn(blockquoteSchema.type(ctx)),
);
// register the command when creating the editor
const editor = Editor().make().use(wrapInBlockquoteCommand).create();
// call command
editor.action(callCommand(wrapInBlockquoteCommand.key));
```
### Example: Command with argument
Commands can accept arguments of any type:
```typescript
import { headingSchema } from "@milkdown/kit/preset/commonmark";
import { setBlockType } from "@milkdown/kit/prose/commands";
import { $command, callCommand } from "@milkdown/kit/utils";
// use number as the type of argument
export const WrapInHeading = createCmdKey<number>();
const wrapInHeadingCommand = $command(
"WrapInHeading",
(ctx) =>
(level = 1) =>
setBlockType(headingSchema.type(ctx), { level }),
);
// call command
editor.action(callCommand(wrapInHeadingCommand.key)); // turn to h1 by default
editor.action(callCommand(wrapInHeadingCommand.key, 2)); // turn to h2
```
### Example: Command with Multiple Arguments
```typescript
interface TableConfig {
rows: number;
cols: number;
withHeader: boolean;
}
const insertTableCommand = $command(
"InsertTable",
(ctx) => (config: TableConfig) => {
// Implementation for inserting a table
return (state, dispatch) => {
// ... table insertion logic
return true;
};
},
);
// Usage
editor.action(
callCommand(insertTableCommand.key, {
rows: 3,
cols: 3,
withHeader: true,
}),
);
```
## Best Practices
---
1. **Command Naming**
- Use clear, descriptive names
- Follow the pattern: `[Action][Target]Command`
- Example: `toggleEmphasisCommand`, `insertTableCommand`
2. **Command Organization**
- Group related commands together
- Use namespaces for command keys
- Keep commands focused and single-purpose
3. **Error Handling**
- Always check if the command can be executed
- Return `false` if the command cannot be executed
- Handle edge cases gracefully
4. **Performance**
- Keep commands lightweight
- Avoid unnecessary state updates
- Use command chaining for complex operations
5. **Type Safety**
- Use TypeScript for command arguments
- Define clear interfaces for command payloads
- Use generics for type-safe command keys
## Common Patterns
---
### Toggle Commands
```typescript
const toggleCommand = $command(
"ToggleFeature",
(ctx) => () => (state, dispatch) => {
const isActive = checkIfActive(state);
return isActive
? removeFeature(state, dispatch)
: addFeature(state, dispatch);
},
);
```
### Insert Commands
```typescript
const insertCommand = $command(
"InsertContent",
(ctx) => (content: string) => (state, dispatch) => {
const { selection } = state;
if (!selection) return false;
const tr = state.tr.insertText(content, selection.from);
dispatch?.(tr);
return true;
},
);
```
### Transform Commands
```typescript
const transformCommand = $command(
"TransformContent",
(ctx) => (transform: (node: ProseNode) => ProseNode) => (state, dispatch) => {
const { selection } = state;
if (!selection) return false;
const tr = state.tr.replaceWith(
selection.from,
selection.to,
transform(state.doc.nodeAt(selection.from)!),
);
dispatch?.(tr);
return true;
},
);
```
-39
View File
@@ -1,39 +0,0 @@
# FAQ
This page lists answers of FAQ.
---
### How can I change contents programmatically?
You should use `editor.action` to change the contents.
We provide two macros for that allow you to change content in milkdown, `insert` and `replaceAll`.
```typescript
import { insert, replaceAll } from "@milkdown/kit/utils";
const editor = await Editor.make()
// .use(<All Your Plugins>)
.create();
editor.action(insert("# New Heading"));
editor.action(replaceAll("# New Document"));
```
---
### How to configure remark?
```typescript
import { remarkStringifyOptionsCtx } from "@milkdown/kit/core";
editor.config((ctx) => {
ctx.set(remarkStringifyOptionsCtx, {
// some options, for example:
bullet: "*",
fences: true,
incrementListMarker: false,
});
});
```
-165
View File
@@ -1,165 +0,0 @@
# Getting Started with Milkdown
Milkdown is a powerful WYSIWYG markdown editor that combines the simplicity of markdown with the flexibility of a modern editor. It's designed to be lightweight yet extensible, making it perfect for both simple and complex editing needs.
## Quick Start
The fastest way to get started is using `@milkdown/crepe`:
```bash
npm install @milkdown/crepe
```
```typescript
import { Crepe } from "@milkdown/crepe";
import "@milkdown/crepe/theme/common/style.css";
import "@milkdown/crepe/theme/frame.css";
const crepe = new Crepe({
root: "#app",
defaultValue: "Hello, Milkdown!",
});
crepe.create();
```
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/editor-crepe"}
## Core Concepts
Milkdown consists of two main parts:
1. **Core Package** (`@milkdown/core`)
- Plugin loader
- Internal plugins
2. **Additional Plugins**
- Syntax support
- Commands
- UI components
- Custom features
This modular architecture allows you to enable or disable features as needed, from basic markdown support to advanced features like tables, LaTeX equations, and collaborative editing.
## Key Features
- 📝 **WYSIWYG Markdown** - Write markdown in an elegant way
- 🎨 **Themable** - Create your own theme and publish it as an npm package
- 🎮 **Hackable** - Create your own plugin to support your awesome idea
- 🦾 **Reliable** - Built on top of [prosemirror](https://prosemirror.net/) and [remark](https://github.com/remarkjs/remark)
-**Slash & Tooltip** - Write faster than ever, enabled by a plugin
- 🧮 **Math** - LaTeX math equations support via math plugin
- 📊 **Table** - Table support with fluent ui, via table plugin
- 🍻 **Collaborate** - Shared editing support with [yjs](https://docs.yjs.dev/)
- 💾 **Clipboard** - Support copy and paste markdown, via clipboard plugin
- 👍 **Emoji** - Support emoji shortcut and picker, via emoji plugin
## Tech Stack
Milkdown is built on top of these powerful libraries:
- [Prosemirror](https://prosemirror.net/) - A toolkit for building rich-text editors on the web
- [Remark](https://github.com/remarkjs/remark) - Markdown parser done right
- [TypeScript](https://www.typescriptlang.org/) - For type safety and better developer experience
## Creating Your First Editor
Milkdown provides two distinct approaches to create an editor, each suited for different needs:
### 1. 🍼 Using `@milkdown/kit` (Build from Scratch)
This approach gives you complete control over your editor. Use this if you want to:
- Build a custom editor from the ground up
- Have full control over which features to include
- Create a highly customized editing experience
- Integrate with specific frameworks or requirements
First, install the required packages:
```bash
npm install @milkdown/kit
```
Create a basic editor with commonmark syntax:
```typescript
import { Editor } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
// This is the must have css for prosemirror
import "@milkdown/kit/prose/view/style/prosemirror.css";
Editor.make().use(commonmark).create();
```
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vanilla-commonmark"}
Add undo & redo support:
```typescript
import { Editor } from "@milkdown/kit/core";
import { history } from "@milkdown/kit/plugin/history";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
import "@milkdown/theme-nord/style.css";
const milkdown = Editor.make()
.config(nord)
.use(commonmark)
.use(history)
.create()
.then(() => {
console.log("Editor created");
});
// To destroy the editor
milkdown.destroy();
```
> **Note**: `<Mod>` is `<Cmd>` for macOS and `<Ctrl>` for other platforms.
### 2. 🥞 Using `@milkdown/crepe` (Ready to Use)
This is the quickest way to get started with a fully-featured editor. Use this if you want to:
- Get up and running quickly
- Have a well-designed editor out of the box
- Focus on content rather than configuration
- Have a production-ready solution with minimal setup
```bash
npm install @milkdown/crepe
```
```typescript
import { Crepe } from "@milkdown/crepe";
import "@milkdown/crepe/theme/common/style.css";
/**
* Available themes:
* frame, classic, nord
* frame-dark, classic-dark, nord-dark
*/
import "@milkdown/crepe/theme/frame.css";
const crepe = new Crepe({
root: "#app",
defaultValue: "Hello, Milkdown!",
});
crepe.create().then(() => {
console.log("Editor created");
});
// To destroy the editor
crepe.destroy();
```
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/editor-crepe"}
## Next Steps
- Learn more about [overview](/guide/architecture-overview)
- Explore [available plugins](/plugins/using-plugins)
- Check out [theming](/guide/theming)
> 🍼 Fun fact: This documentation is rendered by Milkdown itself!
@@ -1,398 +0,0 @@
# Interacting with Editor
This guide covers the essential ways to interact with the Milkdown editor, including initialization, content management, and editor lifecycle.
## Using Crepe Editor
---
Crepe is a high-level wrapper around Milkdown that provides a simpler API for common editor operations. Here's how to use it:
```typescript
import { Crepe } from "@milkdown/crepe";
// Create a new editor instance
const editor = new Crepe({
// Optional: specify root element (DOM node or selector)
root: "#editor",
// Optional: set default content, supports markdown, json and dom.
defaultValue: "# Hello Crepe!",
});
// Create the editor
await editor.create();
// Get markdown content
const markdown = editor.getMarkdown();
// Set readonly mode
editor.setReadonly(true);
// Register event listeners
editor.on((listener) => {
listener.markdownUpdated((ctx, markdown) => {
console.log("Content updated:", markdown);
});
listener.focus((ctx) => {
console.log("Editor focused");
});
listener.blur((ctx) => {
console.log("Editor blurred");
});
listener.selectionUpdated((ctx, selection, prevSelection) => {
console.log("Selection updated:", selection);
});
listener.updated((ctx, doc, prevDoc) => {
console.log("Document updated:", doc);
});
});
// Destroy the editor when done
await editor.destroy();
```
## Register to DOM
---
By default, milkdown will create editor on the `document.body`. Alternatively, you can also point out which dom node you want it to load into:
```typescript
import { rootCtx } from "@milkdown/kit/core";
Editor.make().config((ctx) => {
ctx.set(rootCtx, document.querySelector("#editor"));
});
```
It's also possible to just pass a selector to `rootCtx`:
> The selector will be passed to `document.querySelector` to get the dom.
```typescript
import { rootCtx } from "@milkdown/kit/core";
Editor.make().config((ctx) => {
ctx.set(rootCtx, "#editor");
});
```
## Setting Default Value
---
We support three types of default values:
- Markdown strings
- HTML DOM
- Prosemirror documentation JSON
### Markdown
You can set a markdown string as the default value of the editor.
```typescript
import { defaultValueCtx } from "@milkdown/kit/core";
const defaultValue = "# Hello milkdown";
Editor.make().config((ctx) => {
ctx.set(defaultValueCtx, defaultValue);
});
```
### Dom
You can also use HTML as default value.
Let's assume that we have the following html snippets:
```html
<div id="pre">
<h1>Hello milkdown!</h1>
</div>
```
Then we can use it as a defaultValue with a `type` specification:
```typescript
import { defaultValueCtx } from "@milkdown/kit/core";
const defaultValue = {
type: "html",
dom: document.querySelector("#pre"),
};
Editor.make().config((ctx) => {
ctx.set(defaultValueCtx, defaultValue);
});
```
### JSON
We can also use a JSON object as a default value.
This JSON object can be obtained by a listener through the [listener-plugin](https://www.npmjs.com/package/@milkdown/plugin-listener), for example:
```typescript
import { listener, listenerCtx } from "@milkdown/kit/plugin/listener";
let jsonOutput;
Editor.make()
.config((ctx) => {
ctx.get(listenerCtx).updated((ctx, doc, prevDoc) => {
jsonOutput = doc.toJSON();
});
})
.use(listener);
```
Then we can use this `jsonOutput` as default Value:
```typescript
import { defaultValueCtx } from "@milkdown/kit/core";
const defaultValue = {
type: "json",
value: jsonOutput,
};
Editor.make().config((ctx) => {
ctx.set(defaultValueCtx, defaultValue);
});
```
## Inspecting Editor Status
---
You can inspect the editor's status through the `status` property.
```typescript
import { Editor, EditorStatus } from "@milkdown/kit/core";
const editor = Editor.make().use(/* some plugins */);
assert(editor.status === EditorStatus.Idle);
editor.create().then(() => {
assert(editor.status === EditorStatus.Created);
});
assert(editor.status === EditorStatus.OnCreate);
editor.destroy().then(() => {
assert(editor.status === EditorStatus.Destroyed);
});
assert(editor.status === EditorStatus.OnDestroyed);
```
You can also listen to the status changes:
```typescript
import { Editor, EditorStatus } from "@milkdown/kit/core";
const editor = Editor.make().use(/* some plugins */);
editor.onStatusChange((status: EditorStatus) => {
console.log(status);
});
```
### Status Lifecycle
1. `Idle`: Initial state
2. `OnCreate`: During creation
3. `Created`: Successfully created
4. `OnDestroyed`: During destruction
5. `Destroyed`: Successfully destroyed
## Adding Listeners
---
As mentioned above, you can add a listener to the editor, in order to get its value when needed.
You can add as many listeners as you want, all the listeners will be triggered at once.
### Markdown Listener
You can add markdown listener to get the editor's contents as a markdown string.
> ⚠️ Markdown listener will influence the performance for large documents, please use it carefully.
> If you have a large document, I suggest you to only `parse` and `serialize` the document when needed.
```typescript
import { listener, listenerCtx } from "@milkdown/kit/plugin/listener";
let output = "";
Editor.make()
.config((ctx) => {
ctx.get(listenerCtx).markdownUpdated((ctx, markdown, prevMarkdown) => {
output = markdown;
});
})
.use(listener);
```
### Doc Listener
You can also listen to the [raw prosemirror document node](https://prosemirror.net/docs/ref/#model.Node), and do things you want from there.
```typescript
import { listener, listenerCtx } from "@milkdown/kit/plugin/listener";
let jsonOutput;
Editor.make()
.config((ctx) => {
ctx.get(listenerCtx).updated((ctx, doc, prevDoc) => {
jsonOutput = doc.toJSON();
});
})
.use(listener);
```
### Selection Listener
You can track changes to the editor's selection using the `selectionUpdated` event. This is useful for implementing features like:
- Custom toolbars that update based on selection
- Context menus
- Selection-based formatting controls
```typescript
import { listener, listenerCtx } from "@milkdown/kit/plugin/listener";
import { Selection, TextSelection } from "@milkdown/prose/state";
Editor.make()
.config((ctx) => {
ctx.get(listenerCtx).selectionUpdated((ctx, selection, prevSelection) => {
if (selection instanceof TextSelection) {
// Get selection range
const { from, to } = selection;
// Example: Update toolbar based on selection
updateToolbar({
hasSelection: from !== to,
selectionStart: from,
selectionEnd: to,
});
}
});
})
.use(listener);
```
The selection listener will be triggered when the selection is changed.
So you don't need to compare them manually.
For more details about listeners, please check [Using Listeners](/docs/api/plugin-listener).
## Readonly Mode
---
You can set the editor to readonly mode by setting the `editable` property.
```typescript
import { editorViewOptionsCtx } from "@milkdown/kit/core";
let readonly = false;
const editable = () => !readonly;
Editor.make().config((ctx) => {
ctx.update(editorViewOptionsCtx, (prev) => ({
...prev,
editable,
}));
});
// set to readonly after 5 secs.
setTimeout(() => {
readonly = true;
}, 5000);
```
### Use Cases for Readonly Mode
- Preview mode
- Document review
- Print-friendly views
- Mobile device optimization
## Using Actions
---
You can use an action to get the context value in a running editor on demand.
For example, to get the markdown string by running an action:
```typescript
import { Editor, editorViewCtx, serializerCtx } from "@milkdown/kit/core";
async function playWithEditor() {
const editor = await Editor.make().use(commonmark).create();
const getMarkdown = () =>
editor.action((ctx) => {
const editorView = ctx.get(editorViewCtx);
const serializer = ctx.get(serializerCtx);
return serializer(editorView.state.doc);
});
// get markdown string:
getMarkdown();
}
```
We provide some macros out of the box, you can use them as actions:
```typescript
import { insert } from "@milkdown/kit/utils";
editor.action(insert("# Hello milkdown"));
```
### Common Actions
- Insert content
- Get current selection
- Apply formatting
- Execute commands
For more details about macros, please check [macros](/docs/guide/macros).
## Destroying
---
You can call `editor.destroy` to destroy an existing editor. You can create a new editor again with `editor.create`.
```typescript
await editor.destroy();
// Then create again
await editor.create();
```
If you just want to recreate the editor, you can use `editor.create`, it will **destroy the old editor and create a new one**.
```typescript
await editor.create();
// This equals to call `editor.destroy` and `editor.create` again.
await editor.create();
```
If you want to **clear the plugins and configs for the editor** when calling `editor.destroy`, you can pass `true` to `editor.destroy`.
```typescript
await editor.destroy(true);
```
-252
View File
@@ -1,252 +0,0 @@
# Keyboard Shortcuts
Keyboard shortcuts are a crucial part of the editor's user experience. Milkdown provides a flexible system for configuring keyboard shortcuts through presets and plugins.
## Default Shortcuts
---
Milkdown comes with a set of default keyboard shortcuts from both presets and plugins. Here's a comprehensive list of all internal shortcuts:
> #### 💡 Note
>
> `Mod` represents the platform-specific modifier key:
>
> - Windows/Linux: `Ctrl`
> - macOS: `Command`
### Commonmark Preset Shortcuts
#### Headings
| Shortcut | Description |
| -------------------- | ----------------------- |
| `Mod-Alt-1` | Turn block into h1 |
| `Mod-Alt-2` | Turn block into h2 |
| `Mod-Alt-3` | Turn block into h3 |
| `Mod-Alt-4` | Turn block into h4 |
| `Mod-Alt-5` | Turn block into h5 |
| `Mod-Alt-6` | Turn block into h6 |
| `Delete`/`Backspace` | Downgrade heading level |
#### Block Elements
| Shortcut | Description |
| ------------- | ---------------------------- |
| `Mod-Shift-b` | Wrap selection in blockquote |
| `Mod-Shift-8` | Wrap in bullet list |
| `Mod-Shift-7` | Wrap in ordered list |
| `Mod-Shift-c` | Wrap in code block |
| `Shift-Enter` | Insert hard break |
| `Mod-Alt-0` | Wrap in paragraph |
#### Text Formatting
| Shortcut | Description |
| -------- | ------------------ |
| `Mod-b` | Toggle bold |
| `Mod-i` | Toggle italic |
| `Mod-e` | Toggle inline code |
### GFM Preset Shortcuts
#### Text Formatting
| Shortcut | Description |
| ----------- | -------------------- |
| `Mod-Alt-x` | Toggle strikethrough |
#### Tables
| Shortcut | Description |
| ------------------- | -------------------------------- |
| `Mod-]` | Move to next cell |
| `Mod-[` | Move to previous cell |
| `Mod-Enter`/`Enter` | Exit table and break if possible |
## Configuring Shortcuts
---
You can customize keyboard shortcuts by configuring the keymap in the editor setup:
```typescript
import { blockquoteKeymap, commonmark } from "@milkdown/kit/preset/commonmark";
Editor.make()
.config((ctx) => {
ctx.set(blockquoteKeymap.key, {
WrapInBlockquote: "Mod-Shift-b",
// or you may want to bind multiple keys:
WrapInBlockquote: ["Mod-Shift-b", "Mod-b"],
});
})
.use(commonmark);
```
## Defining Keymaps
---
Keymaps in Milkdown are defined using the `$useKeymap` utility. Here's how to define keymaps for different features:
### Heading Keymap Example
```typescript
import { $useKeymap } from "@milkdown/utils";
import { commandsCtx } from "@milkdown/core";
export const headingKeymap = $useKeymap("headingKeymap", {
TurnIntoH1: {
shortcuts: "Mod-Alt-1",
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(wrapInHeadingCommand.key, 1);
},
},
TurnIntoH2: {
shortcuts: "Mod-Alt-2",
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(wrapInHeadingCommand.key, 2);
},
},
// ... more heading levels
DowngradeHeading: {
shortcuts: ["Delete", "Backspace"],
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(downgradeHeadingCommand.key);
},
},
});
```
### Strong (Bold) Keymap Example
```typescript
import { $useKeymap } from "@milkdown/utils";
import { commandsCtx } from "@milkdown/core";
export const strongKeymap = $useKeymap("strongKeymap", {
ToggleBold: {
shortcuts: ["Mod-b"],
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(toggleStrongCommand.key);
},
},
});
```
### Keymap Structure
Each keymap definition follows this structure:
```typescript
$useKeymap('keymapName', {
CommandName: {
shortcuts: string | string[], // Single shortcut or array of shortcuts
priority?: number, // (Optional) Priority of the shortcut
command: (ctx) => () => { // Command to execute
const commands = ctx.get(commandsCtx);
return () => commands.call(commandKey, ...args);
},
},
});
```
## Creating Custom Shortcuts
---
If you need to add custom shortcuts, you can create a keymap plugin:
```typescript
import { $useKeymap } from "@milkdown/utils";
import { commandsCtx } from "@milkdown/core";
const customKeymap = $useKeymap("customKeymap", {
CustomCommand: {
shortcuts: "F1",
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(someCommand.key);
},
},
});
// Usage
Editor.make().use(customKeymap).use(commonmark);
```
### Example: Custom Command with Shortcut
```typescript
import { $command, $useKeymap } from "@milkdown/utils";
import { commandsCtx } from "@milkdown/core";
// Create a custom command
const customCommand = $command("CustomCommand", (ctx) => () => {
return (state, dispatch) => {
// Command implementation
return true;
};
});
// Create a keymap
const customKeymap = $useKeymap("customKeymap", {
CustomCommand: {
shortcuts: ["F1", "Mod-F1"], // Multiple shortcuts
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(customCommand.key);
},
},
});
// Usage
Editor.make().use(customCommand).use(customKeymap);
```
## Shortcut Priority
You can control the order in which shortcuts are handled by specifying a `priority` property. Shortcuts with higher priority values are handled before those with lower values. This is useful if you want your custom shortcut to override or take precedence over other shortcuts that use the same key combination.
When multiple shortcuts are registered for the same key, they are executed in order of priority. If a shortcut command returns `false`, the next shortcut with the same key will be tried. If it returns `true`, no further commands for that key will be run. This allows you to chain or override shortcut behaviors as needed.
- The default priority is **50**.
- Normal priority values should be between **1** and **100**.
- Use higher numbers to ensure your shortcut is registered before others with the same key.
#### Example: Using Priority
```typescript
import { $useKeymap } from "@milkdown/utils";
import { commandsCtx } from "@milkdown/core";
export const customKeymap = $useKeymap("customKeymap", {
CustomBold: {
shortcuts: "Mod-b",
priority: 100, // Highest in the normal range, so this runs first
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => {
// Custom bold logic
return true;
};
},
},
CustomAnotherBold: {
shortcuts: "Mod-b",
priority: 75, // Lower priority, will run only if CustomBold returns false
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => {
// Custom italic logic
return true;
};
},
},
});
```
-278
View File
@@ -1,278 +0,0 @@
# Macros
Macros are helper functions that provide a convenient way to interact with the editor. They take a payload (or nothing) as parameters and return a callback function that takes the `ctx` of milkdown as a parameter. When called with `ctx`, they apply the specified action to the editor.
## Usage
There are two main ways to use macros:
```typescript
import { insert } from "@milkdown/kit/utils";
import { listenerCtx } from "@milkdown/plugin-listener";
// Method 1: Using editor.action()
editor.action(insert("# Hello Macro"));
// Method 2: Using listener
editor.config((ctx) => {
ctx.get(listenerCtx).mounted(insert("# Default Title"));
});
```
## Available Macros
### Content Manipulation
#### `insert`
Inserts content at the current cursor position. The macro accepts two parameters:
- `markdown`: The markdown string to insert
- `inline`: Optional boolean flag (default: false) that determines how the content is inserted
```typescript
import { insert } from "@milkdown/kit/utils";
// Insert as block content (default)
editor.action(insert("# Hello World"));
// Insert as inline content
editor.action(insert("inline text", true));
```
The behavior differs based on the `inline` parameter:
- When `inline` is `false` (default):
- Replaces the current selection with the parsed markdown content
- Maintains the selection's open start/end positions
- Scrolls the view to show the inserted content
- When `inline` is `true`:
- Attempts to insert the content as inline text
- If the content is text-only, replaces the selection with a text node
- Otherwise, replaces the selection with the parsed content
#### `insertPos`
Inserts markdown at a given position. The macro accepts two parameters:
- `markdown`: The markdown string to insert
- `pos`: The position to insert the content at
```typescript
import { insertPos } from "@milkdown/kit/utils";
// Insert "Hello" at the beginning of the document
editor.action(insertPos("Hello", 0));
```
#### `replaceAll`
Replaces all content in the editor. The macro accepts two parameters:
- `markdown`: The markdown string to replace the current content with
- `flush`: Optional boolean flag (default: false) that determines how the replacement is performed
```typescript
import { replaceAll } from "@milkdown/kit/utils";
// Replace content without flushing state
editor.action(replaceAll("# New Content"));
// Replace content and flush editor state
editor.action(replaceAll("# New Content", true));
```
The behavior differs based on the `flush` parameter:
- When `flush` is `false` (default):
- Replaces the entire document content with the new markdown
- Maintains the current editor state
- More efficient for simple content replacements
- When `flush` is `true`:
- Creates a new editor state with the new content
- Reinitializes all plugins
- Useful when you need a completely fresh editor state
#### `replaceRange`
Replaces the content of the given range with a markdown string.
```typescript
import { replaceRange } from "@milkdown/kit/utils";
// Replace content from position 0 to 5 with "Hello"
editor.action(replaceRange("Hello", { from: 0, to: 5 }));
```
### Content Retrieval
#### `getMarkdown`
Gets the current content as markdown. If a range is provided, it will return the markdown for that range; otherwise, it will return the markdown for the entire document.
```typescript
import { getMarkdown } from "@milkdown/kit/utils";
// Get markdown for the entire document
const markdown = editor.action(getMarkdown());
// Get markdown for a specific range
const selectionMarkdown = editor.action(getMarkdown({ from: 0, to: 5 }));
```
#### `getHTML`
Gets the current content as HTML.
```typescript
import { getHTML } from "@milkdown/kit/utils";
const html = editor.action(getHTML());
```
### Editor State
#### `forceUpdate`
Forces the editor to update its state.
```typescript
import { forceUpdate } from "@milkdown/kit/utils";
editor.action(forceUpdate());
```
#### `setAttr`
Sets attributes for a node at a specific position. The macro accepts two parameters:
- `pos`: The position of the node to update
- `update`: A function that takes the previous attributes and returns the new attributes
```typescript
import { setAttr } from "@milkdown/kit/utils";
// Update node attributes at position 10
editor.action(
setAttr(10, (prevAttrs) => ({
...prevAttrs,
class: "custom-class",
})),
);
// Example: Update heading level
editor.action(
setAttr(10, (prevAttrs) => ({
...prevAttrs,
level: 2,
})),
);
```
The macro:
- Takes a specific position in the document
- Retrieves the node at that position
- Applies the update function to modify the node's attributes
- Dispatches the changes to update the editor state
Note: The position must be valid and contain a node, otherwise the operation will be ignored.
### Navigation
#### `outline`
Gets the outline of the document.
```typescript
import { outline } from "@milkdown/kit/utils";
const docOutline = editor.action(outline());
```
### Command Execution
#### `callCommand`
Calls a registered command with optional payload. The macro has two overloads:
Examples:
```typescript
import { callCommand } from "@milkdown/kit/utils";
import { wrapInHeadingCommand } from "@milkdown/plugin-heading";
// Using command key
editor.action(callCommand(wrapInHeadingCommand.key, 1));
// With complex payload
editor.action(
callCommand("CustomCommand", {
type: "heading",
level: 1,
content: "New Heading",
}),
);
```
The macro:
- Takes a command key
- Optionally accepts a payload parameter
- Returns a boolean indicating whether the command was successful
Note: The command must be registered in the editor's command context before it can be called.
### Utility Macros
#### `markdownToSlice`
Converts a markdown string to a [slice](https://prosemirror.net/docs/ref/#model.Slice). This is useful when you need to manipulate the content before inserting it into the editor.
```typescript
import { markdownToSlice } from "@milkdown/kit/utils";
const slice = editor.action(markdownToSlice("# Hello Slice"));
```
## Examples
### Adding Content
```typescript
import { insert } from "@milkdown/kit/utils";
import { listenerCtx } from "@milkdown/plugin-listener";
editor.config((ctx) => {
ctx.get(listenerCtx).mounted(insert("# Welcome\nStart editing..."));
});
```
### Saving Content
```typescript
import { getMarkdown } from "@milkdown/kit/utils";
editor.config((ctx) => {
ctx.get(listenerCtx).updated(() => {
const content = getMarkdown()(ctx);
localStorage.setItem("editor-content", content);
});
});
```
### Custom Command with Macro
```typescript
import { callCommand } from "@milkdown/kit/utils";
editor.action(
callCommand("customCommand", {
type: "heading",
level: 1,
content: "New Heading",
}),
);
```
For more details about each macro's parameters and return types, check the [API Reference](/docs/api/utils#macros).
-38
View File
@@ -1,38 +0,0 @@
# Prosemirror API
Milkdown is built on top of prosemirror. Which means you can use the entire prosemirror API in Milkdown.
To access the prosemirror API, you can use the `@milkdown/prose` package. It re-exports all of the prosemirror API.
Using this package you can make sure that you are using the same version of prosemirror as Milkdown.
## Installation
To access a certain API in the `prosemirror-x` package, you need to import them from `@milkdown/kit/prose/x`.
For example:
```ts
// Originally in prosemirror-state
import { EditorState } from "@milkdown/kit/prose/state";
// Originally in prosemirror-view
import { EditorView } from "@milkdown/kit/prose/view";
```
## List of packages
The following is a list of all the re-exported prosemirror API.
- `@milkdown/kit/prose/changeset`
- `@milkdown/kit/prose/commands`
- `@milkdown/kit/prose/dropcursor`
- `@milkdown/kit/prose/gapcursor`
- `@milkdown/kit/prose/history`
- `@milkdown/kit/prose/inputrules`
- `@milkdown/kit/prose/keymap`
- `@milkdown/kit/prose/model`
- `@milkdown/kit/prose/schema-list`
- `@milkdown/kit/prose/state`
- `@milkdown/kit/prose/transform`
- `@milkdown/kit/prose/view`
- `@milkdown/kit/prose/tables`
You can find the documentation of the prosemirror API [here](https://prosemirror.net/docs/ref/).
-215
View File
@@ -1,215 +0,0 @@
# Styling Guide
Milkdown is a headless editor, which means it doesn't come with any default styles. This gives you complete control over the appearance of your editor. You can either use existing themes or create your own custom styling solution.
# Styling Crepe Theme
---
Crepe is a collection of themes for Milkdown that provides both light and dark variants. The theme structure is organized as follows:
```
theme/
├── common/ # Shared styles and utilities
├── crepe/ # Light theme variant
├── crepe-dark/ # Dark theme variant
├── frame/ # Frame theme (light)
├── frame-dark/ # Frame theme (dark)
├── nord/ # Nord theme (light)
└── nord-dark/ # Nord theme (dark)
```
## Using Crepe Theme
To use the Crepe theme in your project:
```ts
// Import base styles first
import "@milkdown/crepe/theme/common/style.css";
// Choose the theme you want to use
import "@milkdown/crepe/theme/crepe.css";
```
## Theme Variables
Crepe theme uses CSS variables for consistent styling. Here are all the available variables:
### Colors
```css
.milkdown {
/* Background Colors */
--crepe-color-background: #fffdfb; /* Main background color */
--crepe-color-surface: #fff8f4; /* Surface color for cards/panels */
--crepe-color-surface-low: #fff1e5; /* Lower surface color for depth */
/* Text Colors */
--crepe-color-on-background: #1f1b16; /* Text color on background */
--crepe-color-on-surface: #201b13; /* Text color on surface */
--crepe-color-on-surface-variant: #4f4539; /* Secondary text color */
/* Accent Colors */
--crepe-color-primary: #805610; /* Primary brand color */
--crepe-color-secondary: #fbdebc; /* Secondary accent color */
--crepe-color-on-secondary: #271904; /* Text color on secondary */
/* UI Colors */
--crepe-color-outline: #817567; /* Border/outline color */
--crepe-color-inverse: #362f27; /* Inverse color for contrast */
--crepe-color-on-inverse: #fcefe2; /* Text color on inverse */
--crepe-color-inline-code: #ba1a1a; /* Inline code color */
--crepe-color-error: #ba1a1a; /* Error state color */
/* Interactive Colors */
--crepe-color-hover: #f9ecdf; /* Hover state color */
--crepe-color-selected: #ede0d4; /* Selected state color */
--crepe-color-inline-area: #e4d8cc; /* Inline editing area color */
}
```
### Typography
```css
.milkdown {
/* Font Families */
--crepe-font-title: Georgia, Cambria, "Times New Roman", Times, serif;
--crepe-font-default: "Open Sans", Arial, Helvetica, sans-serif;
--crepe-font-code:
Fira Code, Menlo, Monaco, "Courier New", Courier, monospace;
}
```
### Shadows
```css
.milkdown {
/* Small Shadow */
--crepe-shadow-1:
0px 1px 3px 1px rgba(0, 0, 0, 0.15), 0px 1px 2px 0px rgba(0, 0, 0, 0.3);
/* Large Shadow */
--crepe-shadow-2:
0px 2px 6px 2px rgba(0, 0, 0, 0.15), 0px 1px 2px 0px rgba(0, 0, 0, 0.3);
}
```
## Customizing Crepe Theme
You can customize the Crepe theme by overriding its variables:
```css
/* custom-overrides.css */
.crepe .milkdown {
/* Override colors */
--crepe-color-primary: #your-primary-color;
--crepe-color-background: #your-background-color;
/* Override typography */
--crepe-font-default: "Your Font", sans-serif;
/* Override shadows */
--crepe-shadow-1: your-shadow-value;
}
```
# Styling Milkdown
---
## Basic Styling
The editor is rendered within a container that has the class `.milkdown`, and the editable content area is wrapped in a container with the class `.editor`. You can use these classes to scope your styles:
```css
/* Basic styling example */
.milkdown .editor {
max-width: 800px;
margin: 0 auto;
padding: 1rem;
}
.milkdown .editor p {
margin: 1rem 0;
line-height: 1.6;
}
```
## Node and Mark Classes
Milkdown provides default class names for each node and mark. Here are some common examples:
```css
/* Paragraph styling */
.milkdown .editor .paragraph {
margin: 1rem 0;
}
/* Heading styling */
.milkdown .editor .heading {
font-weight: 600;
margin: 1.5rem 0 1rem;
}
/* List styling */
.milkdown .editor .bullet-list {
padding-left: 1.5rem;
}
.milkdown .editor .ordered-list {
padding-left: 1.5rem;
}
```
## Custom Attributes
You can add custom attributes to nodes and marks, which is particularly useful when working with CSS frameworks like Tailwind CSS.
```typescript
import { Editor, editorViewOptionsCtx } from "@milkdown/kit/core";
import {
commonmark,
headingAttr,
paragraphAttr,
} from "@milkdown/kit/preset/commonmark";
Editor.make()
.config((ctx) => {
// Add attributes to the editor container
ctx.update(editorViewOptionsCtx, (prev) => ({
...prev,
attributes: {
class: "milkdown-editor mx-auto outline-hidden",
spellcheck: "false",
},
}));
// Add attributes to nodes and marks
ctx.set(headingAttr.key, (node) => {
const level = node.attrs.level;
return {
class: `heading-${level} font-bold`,
"data-level": level,
};
});
ctx.set(paragraphAttr.key, () => ({
class: "text-base leading-relaxed",
}));
})
.use(commonmark);
```
# Best Practices
---
1. **Use CSS Variables**: Define your theme's colors and spacing using CSS variables for easy customization.
2. **Responsive Design**: Ensure your editor styles work well on different screen sizes.
3. **Dark Mode Support**: Consider adding dark mode support using CSS variables and media queries.
4. **Accessibility**: Maintain good contrast ratios and readable font sizes.
5. **Performance**: Keep your CSS selectors specific and avoid overly complex rules.
For more examples and inspiration, check out:
- [@milkdown/theme-nord](https://github.com/Milkdown/milkdown/tree/main/packages/theme-nord)
- [@milkdown/crepe/theme](https://github.com/Milkdown/milkdown/tree/main/packages/crepe/src/theme)
-234
View File
@@ -1,234 +0,0 @@
# Using Crepe Editor
Crepe is a powerful, feature-rich Markdown editor built on top of Milkdown. It provides a complete editing experience with a beautiful UI and extensive customization options.
## Why Choose Crepe?
---
- 🚀 **Ready to Use**: Works out of the box with sensible defaults
- 🎨 **Beautiful UI**: Modern design with multiple theme options
- 🔧 **Highly Customizable**: Extensive configuration options
- 📦 **Feature Complete**: Includes all essential Markdown editing features
- 🛠️ **Extensible**: Built on Milkdown's plugin system
## Quick Start
---
### Installation
```bash
# Using npm
npm install @milkdown/crepe
# Using yarn
yarn add @milkdown/crepe
# Using pnpm
pnpm add @milkdown/crepe
```
### Basic Usage
```typescript
import { Crepe } from "@milkdown/crepe";
import "@milkdown/crepe/theme/common/style.css";
import "@milkdown/crepe/theme/frame.css";
// Choose your preferred theme
// Create editor instance
const crepe = new Crepe({
root: document.getElementById("app"),
defaultValue: "# Hello, Crepe!\n\nStart writing your markdown...",
});
// Initialize the editor
await crepe.create();
// Clean up when done
crepe.destroy();
```
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/editor-crepe"}
## Themes
---
Crepe comes with several beautiful themes out of the box:
### Light Themes
- `frame` - Modern frame-based design
- `classic` - Traditional editor look
- `nord` - Clean, minimal Nord color scheme
### Dark Themes
- `frame-dark` - Dark version of frame theme
- `classic-dark` - Dark version of classic theme
- `nord-dark` - Dark version of nord theme
To use a theme:
```typescript
// Import base styles first
import "@milkdown/crepe/theme/common/style.css";
// Then import your chosen theme
import "@milkdown/crepe/theme/frame.css";
```
### Custom Themes
You can create your own theme by extending the base styles. Check out the [existing themes](https://github.com/Milkdown/milkdown/tree/main/packages/crepe/src/theme) for reference.
## Features
---
Crepe includes a comprehensive set of features that can be enabled or disabled as needed.
### Feature Configuration
> **Note**: For any configuration that ends with `Icon` (like `boldIcon`, `linkIcon`, etc.), you can use a HTML string or a simply string. This applies to all icon configurations throughout Crepe's features.
```typescript
const crepe = new Crepe({
features: {
// Disable specific features
[Crepe.Feature.CodeMirror]: false,
[Crepe.Feature.Table]: false,
},
featureConfigs: {
// Configure feature behavior
[Crepe.Feature.LinkTooltip]: {
inputPlaceholder: "Enter URL...",
},
},
});
```
### Available Features
#### 1. Code Editor (`CodeMirror`)
Syntax highlighting and editing for code blocks with language support, theme customization, and preview capabilities.
#### 2. List Management (`ListItem`)
Support for bullet lists, ordered lists, and todo lists with customizable icons and formatting.
#### 3. Link Management (`LinkTooltip`)
Enhanced link editing and preview with customizable tooltips, edit/remove actions, and copy functionality.
#### 4. Image Handling (`ImageBlock`)
Image upload and management with resizing, captions, and support for both inline and block images.
#### 5. Block Editing (`BlockEdit`)
Drag-and-drop block management and slash commands for quick content insertion and organization.
#### 6. Table Support (`Table`)
Full-featured table editing with row/column management, alignment options, and drag-and-drop functionality.
#### 7. Toolbar (`Toolbar`)
Formatting toolbar for selected text with customizable icons and actions.
#### 8. Cursor (`Cursor`)
Enhanced cursor experience with drop cursor and gap cursor for better content placement.
#### 9. Placeholder (`Placeholder`)
Document or block level placeholders to guide users when content is empty.
#### 10. Latex (`Latex`)
Mathematical formula support with both inline and block math rendering using KaTeX.
For detailed configuration options of each feature, please refer to the [API documentation](/docs/api/crepe).
## Editor Instance Methods
---
#### `crepe.editor`
Access the underlying Milkdown editor instance.
```typescript
const editor = crepe.editor;
editor.use(customPlugin);
editor.action(insert("Hello"));
```
#### `crepe.create()`
Initialize the editor.
```typescript
await crepe.create();
```
#### `crepe.destroy()`
Clean up the editor instance.
```typescript
crepe.destroy();
```
#### `crepe.setReadonly(value: boolean)`
Toggle readonly mode.
```typescript
crepe.setReadonly(true); // Make editor read-only
crepe.setReadonly(false); // Make editor editable
```
#### `crepe.on`
Add event listeners.
```typescript
crepe.on((listener) => {
listener.markdownUpdated((markdown) => {
console.log("Markdown updated:", markdown);
});
listener.updated((doc) => {
console.log("Document updated");
});
listener.focus(() => {
console.log("Editor focused");
});
listener.blur(() => {
console.log("Editor blurred");
});
});
```
#### `crepe.getMarkdown()`
Get current markdown content.
```typescript
const markdown = crepe.getMarkdown();
```
## Next Steps
---
- Learn about [Milkdown's architecture](/docs/guide/architecture-overview)
- Explore [available plugins](/docs/plugin/using-plugins)
- Read the [API reference](/docs/api/crepe)
-37
View File
@@ -1,37 +0,0 @@
# Using @milkdown/kit
Milkdown provides a set of utilities to help you build your editor.
These utilities are re-exported from the `@milkdown/kit` package.
Thus, you don't need to install the common dependencies manually like `@milkdown/prose`, `@milkdown/core` or `@milkdown/preset-common` in your project.
## What's included
`@milkdown/kit` re-exports the following packages:
| Package | Import path | Scope |
| ---------------------------------------------------------- | ----------------------------------------- | --------- |
| [@milkdown/core](/docs/api/core) | `@milkdown/kit/core` | Framework |
| [@milkdown/ctx](/docs/api/ctx) | `@milkdown/kit/ctx` | Framework |
| [@milkdown/prose](/docs/guide/prosemirror-api) | `@milkdown/kit/prose` | Framework |
| [@milkdown/prose/\*](/docs/guide/prosemirror-api) | `@milkdown/kit/prose/*` | Framework |
| [@milkdown/transformer](/docs/api/transformer) | `@milkdown/kit/transformer` | Framework |
| [@milkdown/utils](/docs/api/utils) | `@milkdown/kit/utils` | Framework |
| [@milkdown/preset-commonmark](/docs/api/preset-commonmark) | `@milkdown/kit/preset/commonmark` | Preset |
| [@milkdown/preset-gfm](/docs/api/preset-gfm) | `@milkdown/kit/preset/gfm` | Preset |
| [@milkdown/plugin-block](/docs/api/plugin-block) | `@milkdown/kit/plugin/block` | Plugin |
| [@milkdown/plugin-clipboard](/docs/api/plugin-clipboard) | `@milkdown/kit/plugin/clipboard` | Plugin |
| [@milkdown/plugin-cursor](/docs/api/plugin-cursor) | `@milkdown/kit/plugin/cursor` | Plugin |
| [@milkdown/plugin-history](/docs/api/plugin-history) | `@milkdown/kit/plugin/history` | Plugin |
| [@milkdown/plugin-indent](/docs/api/plugin-indent) | `@milkdown/kit/plugin/indent` | Plugin |
| [@milkdown/plugin-listener](/docs/api/plugin-listener) | `@milkdown/kit/plugin/listener` | Plugin |
| [@milkdown/plugin-slash](/docs/api/plugin-slash) | `@milkdown/kit/plugin/slash` | Plugin |
| [@milkdown/plugin-tooltip](/docs/api/plugin-tooltip) | `@milkdown/kit/plugin/tooltip` | Plugin |
| [@milkdown/plugin-trailing](/docs/api/plugin-trailing) | `@milkdown/kit/plugin/trailing` | Plugin |
| [@milkdown/plugin-upload](/docs/api/plugin-upload) | `@milkdown/kit/plugin/upload` | Plugin |
| @milkdown/component | `@milkdown/kit/component` | Component |
| @milkdown/component/code-block | `@milkdown/kit/component/code-block` | Component |
| @milkdown/component/image-block | `@milkdown/kit/component/image-block` | Component |
| @milkdown/component/image-inline | `@milkdown/kit/component/image-inline` | Component |
| @milkdown/component/link-tooltip | `@milkdown/kit/component/link-tooltip` | Component |
| @milkdown/component/list-item-block | `@milkdown/kit/component/list-item-block` | Component |
| @milkdown/component/table-block | `@milkdown/kit/component/table-block` | Component |
-30
View File
@@ -1,30 +0,0 @@
# Why Milkdown
There are different kinds of markdown editors, such as [Typora](https://typora.io/), [tui](https://github.com/nhn/tui.editor) and [Bear](https://bear.app/).
They work pretty well for writing notes in markdown on different platforms. So why bother making Milkdown?
Milkdown aims to provide an **open source solution** for developers to make their editors more powerful, and attractive, it also ensures it runs everywhere.
---
## Open Source & Easy to Integrate
Different from industrial apps such as [Notion](https://notion.so) and [Typora](https://typora.io/),
Milkdown is open source and fully free. You can integrate it everywhere legally.
> If you like milkdown, please consider to fund me in order to help with the maintenance.
## Plugin Driven
Milkdown treats every feature as a plugin.
With this pattern, developers can choose what they need in an editor instead of bundling all features even they won't need.
Developers can extend their plugins to satisfy their habits such as defining a vim keymap via a custom plugin.
## Reliable
Milkdown is powered by [Prosemirror](https://prosemirror.net/) and [Remark](https://github.com/remarkjs/remark), which has a large community and stands the test of the industry.
What's more, plugins from the prosemirror and remark community can be easily reused in order to build a Milkdown plugin.
## Themable & Hackable
Themes and plugins for Milkdown can be shared and installed using npm packages. Milkdown is a headless component, which means you can fully control its style.
-90
View File
@@ -1,90 +0,0 @@
# Milkdown
👋 Welcome to Milkdown. We are so glad to see you here!
💭 You may wonder, what is Milkdown? Please write something here.
> ⚠️ **Not the right side!**
>
> Please try something on the left side.
![1.00](/polar.jpeg "Hello by a polar bear")
You're seeing this editor called **🥞Crepe**, which is an editor built on top of Milkdown.
If you want to install this editor, you can run `npm install @milkdown/crepe`. Then you can use it like this:
```js
import { Crepe } from "@milkdown/crepe";
import "@milkdown/crepe/theme/common/style.css";
// We have some themes for you to choose, ex.
import "@milkdown/crepe/theme/frame.css";
// Or you can create your own theme
import "./your-theme.css";
const crepe = new Crepe({
root: "#app",
defaultValue: "# Hello, Milkdown!",
});
crepe.create().then(() => {
console.log("Milkdown is ready!");
});
// Before unmount
crepe.destroy();
```
---
## Structure
> 🍼 [Milkdown][repo] is a WYSIWYG markdown editor framework.
>
> Which means you can build your own markdown editor with Milkdown.
In the real world, a typical milkdown editor is built on top of 3 layers:
- [x] 🥛 Core: The core of Milkdown, which provides the plugin loading system with the editor concepts.
- [x] 🧇 Plugins: A set of plugins that can be used to extend the functionalities of the editor.
- [x] 🍮 Components: Some headless components that can be used to build your own editor.
At the start, you may find it hard to understand all these concepts.
But don't worry, we have this `@milkdown/crepe` editor for you to get started quickly.
---
## You can do more with Milkdown
In Milkdown, you can extend the editor in many ways:
| Feature | Description | Example |
| ------------ | ---------------------------------------------------- | ------------------------- |
| 🎨 Theme | Create your own theme with CSS | Nord, Dracula |
| 🧩 Plugin | Create your own plugin to extend the editor | Search, Collab |
| 📦 Component | Create your own component to build your own editor | Slash Menu, Toolbar |
| 📚 Syntax | Create your own syntax to extend the markdown parser | Image with Caption, LaTex |
We have provided a lot of plugins and components, with an out-of-the-box crepe editor for you to use and learn.
---
## Open Source
- Milkdown is an open-source project under the MIT license.
- Everyone is welcome to contribute to the project, and you can use it in your own project for free.
- Please let me know what you are building with Milkdown, I would be so glad to see that!
Maintaining Milkdown is a lot of work, and we are working on it in our spare time.
If you like Milkdown, please consider supporting us by [sponsoring][sponsor] the project.
We'll be so grateful for your support.
## Who built Milkdown?
Milkdown is built by [Mirone][mirone] and designed by [Meo][meo].
[repo]: https://github.com/Milkdown/milkdown
[mirone]: https://github.com/Saul-Mirone
[meo]: https://meo.cool
[sponsor]: https://github.com/sponsors/Saul-Mirone
@@ -1,85 +0,0 @@
# Composable Plugins
In the previous section, we showed you how to create a plugin from scratch. Luckily, you don't need to do that in most cases. Milkdown provides a lot of helpers in [@milkdown/utils](/docs/api/utils) to make it easier to create plugins. The **composable** here means that you can use the plugin in other plugins. For example, you can use a command plugin in a keymap plugin. This is a very common pattern in Milkdown.
I'll show you some examples of how to use composable plugins. But I won't go into detail about the options and the usage of each plugin. You can find the details in the [API reference](/docs/api/utils#composable).
## Schema
The schema plugin is the most important plugin in Milkdown. It defines the structure of the document. A schema plugin in milkdown is a super set of the [node schema spec](https://prosemirror.net/docs/ref/#model.NodeSpec) or [mark schema spec](https://prosemirror.net/docs/ref/#model.MarkSpec) in ProseMirror.
Let's create a simple blockquote node plugin as an example:
```typescript
import { $node } from "@milkdown/kit/utils";
const blockquote = $node("blockquote", () => ({
content: "block+",
group: "block",
defining: true,
parseDOM: [{ tag: "blockquote" }],
toDOM: (node) => ["blockquote", ctx.get(blockquoteAttr.key)(node), 0],
parseMarkdown: {
match: ({ type }) => type === "blockquote",
runner: (state, node, type) => {
state.openNode(type).next(node.children).closeNode();
},
},
toMarkdown: {
match: (node) => node.type.name === "blockquote",
runner: (state, node) => {
state.openNode("blockquote").next(node.content).closeNode();
},
},
}));
```
## Input Rule
Since we have a blockquote node, we can create an input rule plugin to make it easier to create a blockquote node.
We expect that when we type `> ` at the beginning of a line, the blockquote node will be created.
```typescript
import { wrappingInputRule } from "@milkdown/kit/prose/inputrules";
import { $inputRule } from "@milkdown/kit/utils";
export const wrapInBlockquoteInputRule = $inputRule(() =>
wrappingInputRule(/^\s*>\s$/, blockquoteSchema.type()),
);
```
## Command
We can also create a command plugin to create a blockquote node.
The command is useful when we want to create a button to create a blockquote node.
```typescript
import { wrapIn } from "@milkdown/kit/prose/commands";
import { $command } from "@milkdown/kit/utils";
export const wrapInBlockquoteCommand = $command(
"WrapInBlockquote",
() => () => wrapIn(blockquoteSchema.type()),
);
```
## Shortcut
We can also create a shortcut plugin for blockquote.
Here we use `Ctrl + Shift + B` as the shortcut. When we press this shortcut, the blockquote node will be created.
And we can also use the command we created in the previous section.
```typescript
import { commandsCtx } from "@milkdown/kit/core";
import { $useKeymap } from "@milkdown/kit/utils";
export const blockquoteKeymap = $useKeymap("blockquoteKeymap", {
WrapInBlockquote: {
shortcuts: "Mod-Shift-b",
command: (ctx) => {
const commands = ctx.get(commandsCtx);
return () => commands.call(wrapInBlockquoteCommand.key);
},
},
});
```
@@ -1,125 +0,0 @@
# Example: Block Plugin
The **block plugin** adds a positional hook next to every top-level node (paragraphs, headings, lists, etc.).
It is the foundation for features such as drag handles, quick-insert buttons or block toolbars.
In Milkdown this functionality lives in `@milkdown/plugin-block` and consistent with tooltip & slash consists of:
- a **BlockProvider** that deals with DOM positioning/lifecycle
- a **blockFactory** _implemented internally_ exposed as two ctx slices: `blockSpec`, `blockPlugin`
This guide covers:
- Understanding the provider/service architecture.
- Writing a **vanilla TypeScript** drag handle that lets you reorder blocks.
- Mounting custom UIs in **React** and **Vue**.
- Studying the production-ready _Block Handle_ feature inside Crepe.
---
## 1. Anatomy of a Block Plugin
Unlike tooltip/slash, `@milkdown/plugin-block` ships its factory slices directly:
```ts
import { blockSpec, blockPlugin } from "@milkdown/plugin-block";
```
You normally interact with **BlockProvider** which talks to an internal _BlockService_: the service listens to mouse / drag events, figures out which node is **active** and sends `show` / `hide` messages to the provider.
Your job is to decide how to render a UI for that active node.
Key APIs:
- `new BlockProvider({ ctx, content, ... })` similar to Tooltip/Slash.
- `provider.active` info about the currently focused block (`node`, `pos`, `el`).
- Optional callbacks: `getOffset`, `getPlacement`, `getPosition` for fine-grained positioning.
---
## 2. Minimal Vanilla Drag Handle
Below we build a small **drag handle** that appears on hover and lets you drag-n-drop any block.
```ts
import { block, blockPlugin } from "@milkdown/plugin-block";
import { BlockProvider } from "@milkdown/plugin-block/block-provider"; // path depending on bundler
import { Editor } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
// 1️⃣ Create DOM element for the handle
const handle = document.createElement("div");
handle.className = "drag-handle";
handle.innerHTML = "≡";
handle.style.cssText = `
width:20px;height:20px;display:flex;align-items:center;justify-content:center;
cursor:grab;border-radius:4px;background:#f2f3f5;color:#555;user-select:none;
`;
// 2️⃣ Build provider show only when mouse is over a block
const provider = (ctx: Ctx) => {
const provider = new BlockProvider({
ctx,
content: handle,
getOffset: () => 8,
});
return {
update: provider.update,
destroy: provider.destroy,
};
};
// 3️⃣ Wire provider to Milkdown
const blockConfig = (ctx: Ctx) => {
ctx.set(blockSpec.key, {
view: provider(ctx),
});
};
Editor.make().config(blockConfig).use(commonmark).use(block).create();
```
Drag & Drop:
The HTML element has `cursor:grab`. The internal `BlockService` automatically sets `draggable` and wires ProseMirror's drag-events so you can reorder blocks without extra code 👉 nice!
---
## 3. Framework Examples
### React
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/react-block"}
The React demo renders a `<BlockHandle/>` component, keeps drag state in hooks and feeds the root element to `BlockProvider`.
### Vue
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vue-block"}
Vue's `<BlockHandle>` uses `Teleport` and reactive refs exactly like the tooltip/slash examples.
---
## 4. Real-world Feature Crepe Block Handle
Crepe brings all the pieces together to create a **block edit** experience that combines a drag handle **and** a plus-button to open the slash menu:
```text
packages/crepe/src/feature/block-edit/handle/
```
Things worth exploring:
1. **Dynamic placement** via `getPlacement` (centred vs top-aligned depending on node height).
2. Filtering nodes with `blockConfig.filterNodes` so handles do not appear inside tables / math / blockquotes.
3. Programmatically showing the _slash menu_ after pressing the "+" button.
---
## 5. Summary & Next Steps
`@milkdown/plugin-block` is the Swiss-army knife for any block-level UI: drag handles, add-buttons, side toolbars…
Combine it with tooltip/slash to build sophisticated editors.
Hack on the examples, tweak positioning callbacks, and ship your own block goodies 🚀.
@@ -1,159 +0,0 @@
# Example: Iframe Plugin
This guide demonstrates how to create a custom iframe syntax plugin for Milkdown. This plugin allows you to embed iframes directly in your markdown content using a simple directive syntax.
## Overview
---
The iframe plugin enables you to embed external web content using the following syntax:
```markdown
::iframe{src="https://example.com"}
```
This will render as an embedded iframe in your document.
## Implementation Steps
---
To create a custom syntax plugin in Milkdown, we need to implement five key components:
1. **Remark Plugin**: Parse the custom syntax
2. **Schema Definition**: Define the node structure
3. **Parser**: Convert markdown to ProseMirror nodes
4. **Serializer**: Convert ProseMirror nodes back to markdown
5. **Input Rules**: Handle user input
Let's implement each component:
## 1. Remark Plugin
---
First, we use the `remark-directive` plugin to support our custom syntax. This plugin allows us to define custom directives in markdown.
```typescript
import directive from "remark-directive";
import { $remark } from "@milkdown/kit/utils";
const remarkDirective = $remark("remarkDirective", () => directive);
```
## 2. Schema Definition
---
Next, we define the schema for our iframe node. The schema specifies how the node behaves and appears in the editor.
```typescript
import { $node } from "@milkdown/kit/utils";
import { Node } from "@milkdown/kit/prose/model";
const iframeNode = $node("iframe", () => ({
group: "block", // Block-level node
atom: true, // Cannot be split
isolating: true, // Cannot be merged with adjacent nodes
marks: "", // No marks allowed
attrs: {
src: { default: null }, // URL attribute
},
parseDOM: [
{
tag: "iframe",
getAttrs: (dom) => ({
src: (dom as HTMLElement).getAttribute("src"),
}),
},
],
toDOM: (node: Node) => [
"iframe",
{ ...node.attrs, contenteditable: false }, // Prevent editing iframe content
0,
],
}));
```
## 3. Parser
---
The parser converts our markdown syntax into ProseMirror nodes. It looks for the `leafDirective` type with the name "iframe".
```typescript
parseMarkdown: {
match: (node) => node.type === 'leafDirective' && node.name === 'iframe',
runner: (state, node, type) => {
state.addNode(type, { src: (node.attributes as { src: string }).src });
},
},
```
## 4. Serializer
---
The serializer converts ProseMirror nodes back to markdown format.
```typescript
toMarkdown: {
match: (node) => node.type.name === 'iframe',
runner: (state, node) => {
state.addNode('leafDirective', undefined, undefined, {
name: 'iframe',
attributes: { src: node.attrs.src },
});
},
},
```
## 5. Input Rules
---
Input rules handle user typing and convert the syntax into an iframe node.
```typescript
import { InputRule } from "@milkdown/kit/prose";
import { $inputRule } from "@milkdown/kit/utils";
const iframeInputRule = $inputRule(
() =>
new InputRule(
/::iframe\{src\="(?<src>[^"]+)?"?\}/,
(state, match, start, end) => {
const [okay, src = ""] = match;
const { tr } = state;
if (okay) {
tr.replaceWith(start - 1, end, iframeNode.type().create({ src }));
}
return tr;
},
),
);
```
## Usage
---
To use the iframe plugin, add it to your Milkdown editor configuration:
```typescript
import { Editor } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
Editor.make()
.use([remarkDirective, iframeNode, iframeInputRule])
.use(commonmark)
.create();
```
## Example
---
Here's a complete example of the iframe plugin in action:
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vanilla-iframe-syntax"}
@@ -1,189 +0,0 @@
# Example: Marker Plugin
This guide demonstrates how to create a custom marker syntax plugin for Milkdown. This plugin allows you to mark text with custom colors using a simple markdown syntax.
## Overview
---
The marker plugin enables you to mark text using the following syntax:
```markdown
==marked text==
=={#EE4B2B}marked text with color==
```
This will render as marked text in your document, with the option to specify custom colors.
## Implementation Steps
---
To create a custom marker syntax plugin in Milkdown, we need to implement several components:
1. **Remark Plugin**: Parse the custom syntax
2. **Schema Definition**: Define the mark structure
3. **Parser**: Convert markdown to ProseMirror marks
4. **Serializer**: Convert ProseMirror marks back to markdown
5. **Input Rules**: Handle user input
6. **Color Picker**: Add UI for color selection
Let's implement each component:
## 1. Remark Plugin
---
First, we create a remark plugin to handle our custom marker syntax:
> ⚠️ The real implementation is more complex, but we simplify it for the sake of the example.
> Under the hood, you'll need to write a [micromark extension](https://github.com/micromark/micromark) to make it works correctly.
```typescript
import { $remark } from "@milkdown/kit/utils";
const remarkMarkColor = () => {
return (tree: any) => {
visit(tree, "text", (node: any, index: number, parent: any) => {
const match = node.value.match(/==(?:{#([^}]+)})?([^=]+)==/);
if (match) {
const [_, color, text] = match;
const mark = {
type: "mark",
data: { color },
children: [{ type: "text", value: text }],
};
parent.children.splice(index, 1, mark);
}
});
};
};
const milkdownMarkColorPlugin = $remark("markColor", () => remarkMarkColor);
```
## 2. Schema Definition
---
Next, we define the schema for our marker:
```typescript
import { $markSchema } from "@milkdown/kit/utils";
import { Mark } from "mdast";
export const DEFAULT_COLOR = "#ffff00";
export const markSchema = $markSchema("mark", () => ({
attrs: {
color: {
default: DEFAULT_COLOR,
validate: "string",
},
},
parseDOM: [
{
tag: "mark",
getAttrs: (node: HTMLElement) => ({
color: node.style.backgroundColor,
}),
},
],
toDOM: (mark) => ["mark", { style: `background-color: ${mark.attrs.color}` }],
parseMarkdown: {
match: (node) => node.type === "mark",
runner: (state, node, markType) => {
const color = (node as Mark).data?.color;
state.openMark(markType, { color });
state.next(node.children);
state.closeMark(markType);
},
},
toMarkdown: {
match: (node) => node.type.name === "mark",
runner: (state, mark) => {
let color = mark.attrs.color;
if (color?.toLowerCase() === DEFAULT_COLOR.toLowerCase()) {
color = undefined;
}
state.withMark(mark, "mark", undefined, {
data: { color },
});
},
},
}));
```
## 3. Input Rules
---
We add input rules to handle user typing:
```typescript
import { $inputRule } from "@milkdown/kit/utils";
import { InputRule } from "@milkdown/kit/prose";
const markInputRule = $inputRule(
() =>
new InputRule(/==(?:{#([^}]+)})?([^=]+)==/, (state, match, start, end) => {
const [okay, color, text] = match;
const { tr } = state;
if (okay) {
tr.addMark(
start,
end,
markSchema.type().create({ color: color || DEFAULT_COLOR }),
);
}
return tr;
}),
);
```
## 4. Color Picker Tooltip
---
To enhance the user experience, we add a color picker tooltip:
```typescript
export const colorPickerTooltip = tooltipFactory("color-picker");
class TooltipPluginView {
// ... implementation
}
export const colorPickerTooltipConfig = (ctx: Ctx) => {
ctx.set(colorPickerTooltip.key, {
view: () => new TooltipPluginView(ctx),
});
};
```
## Usage
---
To use the marker plugin, add it to your Milkdown editor configuration:
```typescript
import { Editor } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
Editor.make()
.use(milkdownMarkColorPlugin)
.use(markSchema)
.use(markInputRule)
.use(colorPickerTooltip)
.use(commonmark)
.create();
```
## Example
---
Here's a complete example of the marker plugin in action:
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vanilla-highlight-syntax"}
@@ -1,137 +0,0 @@
# Example: Slash Plugin
After reading the tooltip guide you already know how Milkdown separates **positioning logic** (provider) from **editor wiring** (ctx slices produced by a factory).
The `@milkdown/plugin-slash` package applies exactly the same idea but focuses on _command palettes_ triggered by a character familiar to `/` menus in modern editors.
This document shows you how to:
- Understand what the slash plugin gives you out-of-the-box.
- Build a **vanilla TypeScript** implementation of a basic `/` menu.
- Use the slash provider with **React** and **Vue**.
- Explore a full-blown menu feature that ships inside Milkdown's Crepe UI.
---
## 1. Anatomy of a Slash Plugin
`@milkdown/plugin-slash` exports two utilities:
1. **`SlashProvider`** Measures the caret position and manages show / hide of your menu.
2. **`slashFactory(id)`** Generates a ctx slice & ProseMirror plugin pair that plugs the provider into the editor.
```ts
import { slashFactory } from "@milkdown/plugin-slash";
export const [mySlashSpec, mySlashPlugin] = slashFactory("my");
```
Just like the tooltip factory:
- `mySlashSpec` is where you put a `PluginSpec` (what ProseMirror needs).
- `mySlashPlugin` turns that spec into a runtime plugin.
---
## 2. A Minimal Vanilla `/` Menu
Below we create a small menu that suggests two commands whenever the user types `/`.
```ts
import { SlashProvider, slashFactory } from "@milkdown/plugin-slash";
import { Editor } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
// DOM content of the menu plain HTML for the demo
const menu = document.createElement("div");
menu.className = "slash-menu";
menu.style.cssText = `
position:absolute;padding:4px 0;background:white;border:1px solid #eee;
box-shadow:0 2px 8px rgba(0,0,0,.15);border-radius:6px;font-size:14px;
`;
menu.innerHTML = `<ul style="margin:0;padding:0;list-style:none">
<li data-cmd="h1" style="padding:4px 12px;cursor:pointer">Heading 1</li>
<li data-cmd="bullet" style="padding:4px 12px;cursor:pointer">Bullet List</li>
</ul>`;
// Click handler replace with real commands
menu.addEventListener("click", (e) => {
const target = e.target as HTMLElement;
const cmd = target.dataset.cmd;
alert(`Run command: ${cmd}`);
});
// Provider positions & shows above DOM element
const provider = new SlashProvider({
content: menu,
// show the menu when the last character before caret is '/'
shouldShow(view) {
return provider.getContent(view)?.endsWith("/") ?? false;
},
offset: 8,
});
const slash = slashFactory("demo");
const slashConfig = (ctx: Ctx) => {
ctx.set(slash.key, {
view: () => ({
update: provider.update,
destroy: provider.destroy,
}),
});
};
Editor.make().config(slashConfig).use(commonmark).use(slash).create();
```
Key takeaways:
- `SlashProvider` has a helper `getContent(view)` to fetch text before the caret handy for filtering.
- You decide **when to show** the menu via the `shouldShow` callback (default: when last char is `/`).
- The provider only manipulates **position + visibility**; rendering & commands are completely yours.
---
## 3. Framework Examples
### React
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/react-slash"}
Highlights:
1. A `<SlashMenu/>` React component renders the list.
2. The component root is passed to `SlashProvider` (just like the tooltip demo).
3. React hooks manage internal focus & keyboard navigation.
### Vue
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vue-slash"}
The Vue version uses `Teleport` to append the menu to `document.body` and `ref` / `watch` for reactivity.
---
## 4. Real-world Feature Crepe Block Menu
Milkdown's **Crepe** UI implements an extensible block-level menu on top of the slash plugin. You'll find the source code at:
```text
packages/crepe/src/feature/block-edit/menu/
```
Notable patterns to look for:
- **Context slices** (`menu` / `menuAPI`) to expose imperative `show` & `hide` methods.
- Filtering commands based on the current text after `/`.
- Preventing the menu inside `code` blocks or lists.
Studying this folder is a great next step once you master the basics.
---
## 5. Summary & Next Steps
- `@milkdown/plugin-slash` gives you caret detection + positioning nothing else.
- UI, behaviour, and commands are fully customisable.
Fork one of the examples above, add your own commands, and you'll have a modern `/` command palette in minutes ✨.
@@ -1,140 +0,0 @@
# Example: Tooltip Plugin
This guide walks you through creating and using **tooltip-based plugins** in Milkdown.
You will learn how the low-level `@milkdown/plugin-tooltip` works and how to build richer experiences on top of it in **vanilla TypeScript**, **React**, and **Vue**.
> **TL;DR** A tooltip in Milkdown is nothing more than a ProseMirror plugin created by `tooltipFactory(id)`.
> It receives position information from the editor and renders any DOM of your choice.
> Everything else (buttons, inputs, styling, framework bindings) can be composed on top of that.
## 1. Anatomy of a Tooltip
---
At its core the tooltip plugin exported from `@milkdown/plugin-tooltip` contains two helpers:
1. **`TooltipProvider`** An utility class powered by [floating-ui](https://floating-ui.com/) to calculate the tooltip position.
2. **`tooltipFactory(id)`** A factory that returns a pair of Milkdown plugin slices which wire the provider into the editor.
The factory is extremely small (≈40 lines):
```ts
import { tooltipFactory } from "@milkdown/plugin-tooltip";
// Create a tooltip identified by the string "my".
export const [myTooltipSpec, myTooltipPlugin] = tooltipFactory("my");
```
The first element (`myTooltipSpec`) is a **ctx slice** that stores a `PluginSpec`, while the second one (`myTooltipPlugin`) is the real ProseMirror plugin which consumes that spec.
## 2. A Minimal Vanilla Tooltip
---
Below is the complete code for a tooltip that shows the **length of the current selection**.
```ts
import { Editor } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { TooltipProvider, tooltipFactory } from "@milkdown/plugin-tooltip";
// 1) Prepare DOM that we will mount into the page.
const el = document.createElement("div");
el.className = "selection-length";
el.style.cssText = `
pointer-events:none;
background:#333;color:#fff;padding:2px 6px;border-radius:4px;font-size:12px;
`;
// 2) Build a provider which updates the content.
const provider = new TooltipProvider({
content: el,
shouldShow: (view) => !!view.state.selection.content().size,
});
// 3) Bridge provider & editor.
const tooltip = tooltipFactory("sel-length");
const tooltipConfig = (ctx: Ctx) => {
ctx.set(selectionTooltipSpec.key, {
view: () => ({
update: provider.update,
destroy: provider.destroy,
}),
});
};
Editor.make().config(tooltipConfig).use(commonmark).use(tooltip).create();
```
Key points:
- We **create** any DOM element we like (`el`).
- `TooltipProvider` tracks the editor position and moves the element.
- `tooltipFactory` wraps the provider into a pluggable slice.
## 3. Framework Examples
---
Sometimes building UI is easier in your favourite framework.
Because the tooltip provider only deals with **DOM elements**, you can freely render React, Vue or Svelte components and pass their root node to the provider.
### React
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/react-tooltip"}
The React example shows how to:
1. Create a React component (`<SelectionTooltip/>`).
2. Render it into a portal and give the root HTML element to `TooltipProvider`.
3. Re-use React state/hooks while Milkdown takes care of positioning.
### Vue
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vue-tooltip"}
The Vue example follows the same pattern with `defineComponent` and `teleport`.
## 4. Real-world Examples
---
### 4-1. Link Tooltip (_@milkdown/component/link-tooltip_)
The [link tooltip](https://github.com/Milkdown/milkdown/tree/main/packages/components/src/link-tooltip) demonstrates how to:
- Maintain UI **state** (`preview` vs `edit`) in ctx slices.
- Communicate with the editor through an **API slice** (add / edit / remove links).
- Render framework-agnostic UI inside a tooltip provider.
Have a look at the files below to see those techniques in action:
```text
packages/components/src/link-tooltip/
├── slices.ts # state & API slices
├── tooltips.ts # preview & edit providers
└── component.tsx # (framework examples)
```
### 4-2. Toolbar Feature (_@milkdown/crepe/feature/toolbar_)
The toolbar in the [crepe](https://github.com/Milkdown/milkdown/tree/main/packages/crepe) package pushes the idea further by:
- Using multiple tooltip instances (one per button group).
- Rendering the UI with Vue _inside_ the provider.
- Sharing configuration via ctx slices so that every button is extensible by third-party plugins.
You can browse the implementation starting from
```text
packages/crepe/src/feature/toolbar/component.tsx
```
## 5. Summary & Next Steps
---
- `@milkdown/plugin-tooltip` offers **just enough** abstraction: positioning & lifecycle.
- Everything else **state, styling, framework integration** is totally up to you.
Try to customise one of the examples above, then ship your own tooltip-powered features 🤟.
-174
View File
@@ -1,174 +0,0 @@
# Plugins 101
In this section we will show you the basic information of the plugin.
In most cases, you will not need to write plugins without helpers.
But it can help you understand the plugin system and what happens under the hood.
## Structure Overview
Generally speaking, a plugin will have following structure:
```typescript
import { MilkdownPlugin } from "@milkdown/kit/ctx";
const myPlugin: MilkdownPlugin = (ctx) => {
// #1 prepare plugin
return async () => {
// #2 run plugin
return async () => {
// #3 clean up plugin
};
};
};
```
Each plugin is composed by three parts:
1. _Prepare_: this part will be executed when plugin is registered in milkdown by `.use` method.
2. _Run_: this part will be executed when plugin is actually loaded.
3. _Post_: this part will be executed when plugin is removed by `.remove` method or editor is destroyed.
## Timer
Timer can be used to decide when to load the current plugin and how current plugin can influence other plugin's loading status.
You can use `ctx.wait` to wait a timer to finish.
```typescript
import { MilkdownPlugin, Complete } from "@milkdown/kit/core";
const myPlugin: MilkdownPlugin = (ctx) => {
return async () => {
const start = Date.now();
await ctx.wait(Complete);
const end = Date.now();
console.log("Milkdown load duration: ", end - start);
};
};
```
You can also create your own timer and influence other plugins load time.
For example, let's create a plugin that will fetch markdown content from remote server as editor's default value.
```typescript
import {
MilkdownPlugin,
editorStateTimerCtx,
defaultValueCtx,
createTimer,
} from "@milkdown/kit/core";
const RemoteTimer = createTimer("RemoteTimer");
const remotePlugin: MilkdownPlugin = (ctx) => {
// register timer
ctx.record(RemoteTimer);
return async () => {
// the editorState plugin will wait for this timer to finish before initialize editor state.
ctx.update(editorStateTimerCtx, (timers) => timers.concat(RemoteTimer));
const defaultMarkdown = await fetchMarkdownAPI();
ctx.set(defaultValueCtx, defaultMarkdown);
// mark timer as complete
ctx.done(RemoteTimer);
return async () => {
await SomeAPI();
// remove timer when plugin is removed
ctx.clearTimer(RemoteTimer);
};
};
};
```
It has following steps:
1. We use `createTimer` to create a timer, and use `pre.record` to register it into milkdown.
2. We update `editorStateTimerCtx` to tell the internal `editorState` plugin that before initialize editor state, it should wait our remote fetch process finished.
3. After we get value from `fetchMarkdownAPI`, we set it as `defaultValue` and use `ctx.done` to mark a timer as complete.
## Ctx
We have used `ctx` several times in the above example, now we can try to understand what it is.
Ctx is a data container which is shared in the entire editor instance. It's composed by a lot of slices. Every `slice` has a unique key and a value. You can change the value of a slice by `ctx.set` and `ctx.update`. And you can get the value of a slice by `ctx.get` with the slice key or name. Last but not least, you can remove a slice by `post.remove`.
```typescript
import { MilkdownPlugin, createSlice } from "@milkdown/kit/ctx";
const counterCtx = createSlice(0, "counter");
const counterPlugin: MilkdownPlugin = (ctx) => {
ctx.inject(counterCtx);
return () => {
// count is 0
const count0 = ctx.get(counterCtx);
// set count to 1
ctx.set(counterCtx, 1);
// now count is 1
const count1 = ctx.get(counterCtx);
// set count to n + 2
ctx.update(counterCtx, (prev) => prev + 2);
// now count is 3
const count2 = ctx.get(counterCtx);
// we can also get value by the slice name
const count3 = ctx.get("counter");
return () => {
// remove the slice
ctx.remove(counterCtx);
};
};
};
```
We can use `createSlice` to create a ctx, and use `pre.inject` to inject the ctx into the editor.
And when plugin processing, `ctx.get` can get the value of a ctx, `ctx.set` can set the value of a ctx, and `ctx.update` can update a ctx using callback function.
So, we can use `ctx` combine with `timer` to decide when should a plugin be processed.
```typescript
import {
MilkdownPlugin,
SchemaReady,
Timer,
createSlice,
} from "@milkdown/kit/core";
const examplePluginTimersCtx = createSlice<Timer[]>([], "example-timer");
const examplePlugin: MilkdownPlugin = (ctx) => {
ctx.inject(examplePluginTimersCtx, [SchemaReady]);
return async () => {
await Promise.all(
ctx.get(examplePluginTimersCtx).map((timer) => ctx.wait(timer)),
);
// or we can use a simplified syntax sugar
await ctx.waitTimers(examplePluginTimersCtx);
// do something
};
};
```
With this pattern, if other plugins want to delay the process of `examplePlugin`, all they need to do is just add a timer into `examplePluginTimersCtx` with `ctx.update`.
## Summary
Now let's go back to the plugin structure. Since we have the knowledge of `timer` and `ctx`, we can understand what we should do in each part of a plugin.
1. In `prepare` stage of the plugin, we can use `ctx.record` to register a timer, and use `ctx.inject` to inject a slice.
2. In `run` stage of the plugin, we can use `ctx.wait` to wait a timer to finish, and use `ctx.get` to get the value of a slice. We can also change values of slices by `ctx.set` and `ctx.update`. And we can use `ctx.done` to mark a timer as complete.
3. In `post` stage of the plugin, we can use `ctx.clearTimer` to clear a timer, and use `ctx.remove` to remove a slice.
-28
View File
@@ -1,28 +0,0 @@
# Using Components
Components are features work out of the box that built on top of plugins.
Each component is a separate module. You can use them by importing them from `@milkdown/kit/component/*`.
All components can be used just like plugins.
```ts
import { imageBlock } from "@milkdown/kit/component/image-block";
import { Editor } from "@milkdown/kit/core";
Editor.make().use(/* some other plugins */).use(imageBlock).create();
```
Components are designed to be headless, which means they are not opinionated about the UI.
You can use them to build your own editor UI. Components are built by web components and can be used in any framework.
---
# List of Components
| Name | Description |
| ------------------------------------------------ | ---------------------------------------------------------- |
| [Code Block](/docs/api/component-code-block) | Render code by [Codemirror](https://codemirror.net/) |
| [Image Block](/docs/api/component-image-block) | Render an image as a block |
| [Image Inline](/docs/api/component-image-inline) | Provide placeholder and uploader features for inline image |
| [Link Tooltip](/docs/api/component-link-tooltip) | Provide edit and preview feature for link |
| [List Item](/docs/api/component-list-item-block) | Renderers bullet, ordered and task list by custom renderer |
| [Table Block](/docs/api/component-table-block) | Render table and provides table editing features |
-87
View File
@@ -1,87 +0,0 @@
# Using Plugins
All features in milkdown are provided by plugin.
Such as syntax, components, etc.
Now we can try more plugins:
```typescript
import { Editor } from "@milkdown/kit/core";
import { slash } from "@milkdown/kit/plugin/slash";
import { tooltip } from "@milkdown/kit/plugin/tooltip";
import { commonmark } from "@milkdown/kit/preset/commonmark";
Editor.make().use(commonmark).use(tooltip).use(slash).create();
```
---
## Toggling Plugins
You can also toggle plugins programmatically:
```typescript
import { Editor } from "@milkdown/kit/core";
import { someMilkdownPlugin } from "some-milkdown-plugin";
const editor = await Editor.config(configForPlugin)
.use(someMilkdownPlugin)
.create();
// remove plugin
await editor.remove(someMilkdownPlugin);
// remove config
editor.removeConfig(configForPlugin);
// add another plugin
editor.use(anotherMilkdownPlugin);
// Recreate the editor to apply changes.
await editor.create();
```
---
## Official Plugins
Milkdown provides the following official plugins:
### Plugins provided by `@milkdown/kit`:
> 🙋‍♀️Why not all plugins are available in `@milkdown/kit`?
>
> `@milkdown/kit` is a collection of plugins that are commonly used in the editor.
> If you want to use a plugin that is not in `@milkdown/kit`, you can install it separately.
> The plugins in `@milkdown/kit` are also stable and well-tested.
| Package Name | Description |
| -------------------------------------------------------------- | --------------------------------------------------------- |
| [@milkdown/kit/preset/commonmark](/docs/api/preset-commonmark) | Add [commonmark](https://commonmark.org/) syntax support. |
| [@milkdown/kit/preset/gfm](/docs/api/preset-gfm) | Add [gfm](https://github.github.com/gfm/) syntax support. |
| [@milkdown/kit/plugin/history](/docs/api/plugin-history) | Add undo & redo support. |
| [@milkdown/kit/plugin/clipboard](/docs/api/plugin-clipboard) | Add markdown copy & paste support. |
| [@milkdown/kit/plugin/cursor](/docs/api/plugin-cursor) | Add drop & gap cursor. |
| [@milkdown/kit/plugin/listener](/docs/api/plugin-listener) | Add listener support. |
| [@milkdown/kit/plugin/indent](/docs/api/plugin-indent) | Add tab indent support. |
| [@milkdown/kit/plugin/upload](/docs/api/plugin-upload) | Add drop and upload support. |
| [@milkdown/kit/plugin/block](/docs/api/plugin-block) | Add a drag handle for every block node. |
| [@milkdown/kit/plugin/tooltip](/docs/api/plugin-tooltip) | Add universal tooltip support. |
| [@milkdown/kit/plugin/slash](/docs/api/plugin-slash) | Add universal slash commands support. |
### Other Plugins:
- [@milkdown/plugin-collab](/docs/api/plugin-collab)
Add collaborative editing support, powered by [yjs](https://docs.yjs.dev/).
- [@milkdown/plugin-prism](/docs/api/plugin-prism)
Add [prism](https://prismjs.com/) support for code block highlight.
- [@milkdown/plugin-emoji](/docs/api/plugin-emoji)
Add emoji shortcut support (something like `:+1:`), and use [twemoji](https://twemoji.twitter.com/) to display emoji.
## Community plugins
Check out [awesome-milkdown](https://github.com/Milkdown/awesome-milkdown) to find community plugins. You can also submit a PR to list your plugins there.
-48
View File
@@ -1,48 +0,0 @@
# Angular
We don't provide Angular support out of box, but you can use the vanilla version with it easily.
## Install the Dependencies
```bash
# install with npm
npm install @milkdown/kit
npm install @milkdown/theme-nord
```
## Create a Component
Create a component is pretty easy.
```html
<!-- editor.component.html -->
<div #editorRef></div>
```
```typescript
// editor.component.ts
import { Component, ElementRef, ViewChild } from "@angular/core";
import { defaultValueCtx, Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
@Component({
templateUrl: "./editor.component.html",
})
export class AppComponent {
@ViewChild("editorRef") editorRef: ElementRef;
defaultValue = "# Milkdown x Angular";
ngAfterViewInit() {
Editor.make()
.config((ctx) => {
ctx.set(rootCtx, this.editorRef.nativeElement);
ctx.set(defaultValueCtx, this.defaultValue);
})
.config(nord)
.use(commonmark)
.create();
}
}
```
-51
View File
@@ -1,51 +0,0 @@
# Next.js
Since we provide [react](/docs/recipes/react) support out of box, we can use it directly in [Next.js](https://nextjs.org/).
## Install the Dependencies
Except the `@milkdown/kit` and theme. We need to install the `@milkdown/react`, which provide lots of abilities for react in milkdown.
```bash
# install with npm
npm install @milkdown/react
npm install @milkdown/kit
npm install @milkdown/theme-nord
```
## Create a Component
Create a component is pretty easy.
```tsx
import { Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { Milkdown, MilkdownProvider, useEditor } from "@milkdown/react";
import { nord } from "@milkdown/theme-nord";
import React from "react";
const MilkdownEditor: React.FC = () => {
const { editor } = useEditor((root) =>
Editor.make()
.config(nord)
.config((ctx) => {
ctx.set(rootCtx, root);
})
.use(commonmark),
);
return <Milkdown />;
};
export const MilkdownEditorWrapper: React.FC = () => {
return (
<MilkdownProvider>
<MilkdownEditor />
</MilkdownProvider>
);
};
```
## Online Demo
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/next-commonmark"}
-82
View File
@@ -1,82 +0,0 @@
# NuxtJS
Since we provide [vue](/docs/recipes/vue) support out of box, we can use it directly in [NuxtJS](https://v3.nuxtjs.org/).
> NuxtJS version should be 3.x.
## Install the Dependencies
Except the `@milkdown/kit` and theme. We need to install the `@milkdown/vue`, which provide lots of abilities for vue in milkdown.
```bash
# install with npm
npm install @milkdown/vue
npm install @milkdown/kit
npm install @milkdown/theme-nord
```
## Create a Component
Create a component is pretty easy.
First, we need to create a `MilkdownEditor` component.
```html
<!-- MilkdownEditor.vue -->
<template>
<Milkdown />
</template>
<script>
import { Editor, rootCtx, defaultValueCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
import { Milkdown, useEditor } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "Milkdown",
components: {
Milkdown,
},
setup: () => {
useEditor((root) =>
Editor.make()
.config((ctx) => {
ctx.set(rootCtx, root);
})
.config(nord)
.use(commonmark),
);
},
});
</script>
```
Then, we need to create a `MilkdownEditorWrapper` component.
```html
<!-- MilkdownEditorWrapper.vue -->
<template>
<MilkdownProvider>
<MilkdownEditor />
</MilkdownProvider>
</template>
<script>
import { MilkdownProvider } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "MilkdownEditorWrapper",
components: {
MilkdownProvider,
},
setup: () => {},
});
</script>
```
## Online Demo
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/nuxt-commonmark"}
-213
View File
@@ -1,213 +0,0 @@
# React Integration
Milkdown provides first-class React support with dedicated packages and hooks for seamless integration. You can choose between Crepe, our feature-rich WYSIWYG editor, or the core Milkdown editor for more customization options.
## Using Crepe
---
Crepe is a powerful, feature-rich Markdown editor built on top of Milkdown that provides a more user-friendly editing experience.
### Installation
```bash
npm install @milkdown/crepe @milkdown/react @milkdown/kit
```
### Implementation
```tsx
import { Crepe } from "@milkdown/crepe";
import { Milkdown, MilkdownProvider, useEditor } from "@milkdown/react";
const CrepeEditor: React.FC = () => {
const { get } = useEditor((root) => {
return new Crepe({ root });
});
return <Milkdown />;
};
export const MilkdownEditorWrapper: React.FC = () => {
return (
<MilkdownProvider>
<CrepeEditor />
</MilkdownProvider>
);
};
```
### Online Demo
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/react-crepe"}
## Using Milkdown
---
For more advanced use cases or when you need full control over the editor's configuration, you can use the core Milkdown editor directly.
### Install Dependencies
```bash
npm install @milkdown/react @milkdown/kit
```
### Basic Usage
Here's a minimal example to get started:
```tsx
import { Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { Milkdown, MilkdownProvider, useEditor } from "@milkdown/react";
import { nord } from "@milkdown/theme-nord";
const MilkdownEditor: React.FC = () => {
const { get } = useEditor((root) =>
Editor.make()
.config(nord)
.config((ctx) => {
ctx.set(rootCtx, root);
})
.use(commonmark),
);
return <Milkdown />;
};
export const MilkdownEditorWrapper: React.FC = () => {
return (
<MilkdownProvider>
<MilkdownEditor />
</MilkdownProvider>
);
};
```
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/react-commonmark"}
## Advanced Usage
---
### Accessing Editor Instance
The `useInstance()` hook can only be used within components that are children of `MilkdownProvider`. It returns a tuple containing a loading state and a getter function to access the editor instance.
```tsx
import { useInstance } from "@milkdown/react";
import { getMarkdown } from "@milkdown/utils";
// ❌ This won't work - ParentComponent is outside MilkdownProvider
const ParentComponent: React.FC = () => {
const [isLoading, getInstance] = useInstance(); // This will be [true, () => undefined]
return <MilkdownEditorWrapper />;
};
// ✅ This is the correct way - EditorControls is inside MilkdownProvider
const EditorControls: React.FC = () => {
const [isLoading, getInstance] = useInstance();
const handleSave = () => {
if (isLoading) return;
const editor = getInstance();
if (!editor) return;
const content = editor.action(getMarkdown());
// Do something with the content
};
return (
<button onClick={handleSave} disabled={isLoading}>
Save
</button>
);
};
// ✅ Proper component structure
const EditorWithControls: React.FC = () => {
return (
<MilkdownProvider>
<MilkdownEditorWrapper />
<EditorControls />
</MilkdownProvider>
);
};
```
### Best Practices
1. **Component Structure**
- Keep the editor component separate from business logic
- Wrap the editor with `MilkdownProvider` at the highest necessary level
- Use TypeScript for better type safety
2. **Performance**
- Memoize the editor configuration if it's complex
- Use React.memo for the editor component if needed
- Avoid unnecessary re-renders of the editor
### Common Use Cases
**Form Integration**
```tsx
const FormWithEditor: React.FC = () => {
const [isLoading, getInstance] = useInstance();
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault();
if (isLoading) return;
const editor = getInstance();
if (!editor) return;
const content = editor.action(getMarkdown());
// Submit form with content
};
return (
<form onSubmit={handleSubmit}>
<MilkdownEditorWrapper />
<button type="submit" disabled={isLoading}>
Submit
</button>
</form>
);
};
```
**Auto-save**
```tsx
import { Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { listener, listenerCtx } from "@milkdown/kit/plugin/listener";
import { Milkdown, useEditor } from "@milkdown/react";
const AutoSaveEditor: React.FC = () => {
const { get } = useEditor((root) =>
Editor.make()
.config((ctx) => {
ctx.set(rootCtx, root);
// Add markdown listener for auto-save
ctx.get(listenerCtx).markdownUpdated((ctx, markdown) => {
// Save content to your backend or storage
saveToBackend(markdown);
});
})
.use(commonmark)
.use(listener),
);
return <Milkdown />;
};
```
## More Examples
---
- [Examples Repository](https://github.com/Milkdown/examples)
-46
View File
@@ -1,46 +0,0 @@
# SolidJS
We don't provide SolidJS support out of box, but you can use the vanilla version with it easily.
## Install the Dependencies
```bash
# install with npm
npm install @milkdown/kit
npm install @milkdown/theme-nord
```
## Create a Component
Create a component is pretty easy.
```tsx
import { defaultValueCtx, Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
import { onCleanup, onMount } from "solid-js";
const Milkdown = () => {
let ref;
let editor;
onMount(async () => {
editor = await Editor.make()
.config((ctx) => {
ctx.set(rootCtx, ref);
})
.config(nord)
.use(commonmark)
.create();
});
onCleanup(() => {
editor.destroy();
});
return <div ref={ref} />;
};
```
## Online Demo
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/solid-commonmark"}
-45
View File
@@ -1,45 +0,0 @@
# Svelte
We don't provide Svelte support out of box, but you can use the vanilla version with it easily.
## Install the Dependencies
```bash
# install with npm
npm install @milkdown/kit
npm install @milkdown/theme-nord
```
## Creating a Component
Creating a component is pretty easy.
```html
<script>
import { Editor, rootCtx, defaultValueCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
function editor(dom) {
// to obtain the editor instance we need to store a reference of the editor.
const MakeEditor = Editor.make()
.config((ctx) => {
ctx.set(rootCtx, dom);
})
.config(nord)
.use(commonmark)
.create();
MakeEditor.then((editor) => {
// here you have access to the editor instance.
// const exampleContent = "# Hello World!";
// editor.action(replaceAll(exampleContent));
});
}
</script>
<div use:editor />
```
## Online Demo
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/svelte-commonmark"}
-294
View File
@@ -1,294 +0,0 @@
# Vue Integration
Milkdown provides first-class Vue support with dedicated packages and hooks for seamless integration. You can choose between Crepe, our feature-rich WYSIWYG editor, or the core Milkdown editor for more customization options.
> Vue version should be 3.x
## Using Crepe
---
Crepe is a powerful, feature-rich Markdown editor built on top of Milkdown that provides a more user-friendly editing experience.
### Installation
```bash
npm install @milkdown/crepe @milkdown/vue @milkdown/kit
```
### Implementation
```vue
<!-- MilkdownEditor.vue -->
<template>
<Milkdown />
</template>
<script>
import { Crepe } from "@milkdown/crepe";
import { Milkdown, MilkdownProvider, useEditor } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "MilkdownEditor",
components: {
Milkdown,
},
setup: () => {
const { get } = useEditor((root) => {
return new Crepe({ root });
});
},
});
</script>
<!-- MilkdownEditorWrapper.vue -->
<template>
<MilkdownProvider>
<MilkdownEditor />
</MilkdownProvider>
</template>
<script>
import { MilkdownProvider } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "MilkdownEditorWrapper",
components: {
MilkdownProvider,
},
});
</script>
```
### Online Demo
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vue-crepe"}
## Using Milkdown
---
For more advanced use cases or when you need full control over the editor's configuration, you can use the core Milkdown editor directly.
### Install Dependencies
```bash
npm install @milkdown/vue @milkdown/kit @milkdown/theme-nord
```
### Basic Usage
Here's a minimal example to get started:
```vue
<!-- MilkdownEditor.vue -->
<template>
<Milkdown />
</template>
<script>
import { Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
import { Milkdown, useEditor } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "MilkdownEditor",
components: {
Milkdown,
},
setup: () => {
const { get } = useEditor((root) =>
Editor.make()
.config(nord)
.config((ctx) => {
ctx.set(rootCtx, root);
})
.use(commonmark),
);
},
});
</script>
<!-- MilkdownEditorWrapper.vue -->
<template>
<MilkdownProvider>
<MilkdownEditor />
</MilkdownProvider>
</template>
<script>
import { MilkdownProvider } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "MilkdownEditorWrapper",
components: {
MilkdownProvider,
},
});
</script>
```
::iframe{src="https://stackblitz.com/github/Milkdown/examples/tree/main/vue-commonmark"}
## Advanced Usage
---
### Accessing Editor Instance
The `useInstance()` hook can only be used within components that are children of `MilkdownProvider`. It returns a tuple containing a loading state and a getter function to access the editor instance.
```vue
<!-- EditorControls.vue -->
<template>
<button @click="handleSave" :disabled="isLoading">Save</button>
</template>
<script>
import { useInstance } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "EditorControls",
setup: () => {
const [isLoading, getInstance] = useInstance();
const handleSave = () => {
if (isLoading.value) return;
const editor = getInstance();
if (!editor) return;
const content = editor.getMarkdown();
// Do something with the content
};
return {
isLoading,
handleSave,
};
},
});
</script>
<!-- EditorWithControls.vue -->
<template>
<MilkdownProvider>
<MilkdownEditor />
<EditorControls />
</MilkdownProvider>
</template>
<script>
import { MilkdownProvider } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "EditorWithControls",
components: {
MilkdownProvider,
},
});
</script>
```
### Best Practices
1. **Component Structure**
- Keep the editor component separate from business logic
- Wrap the editor with `MilkdownProvider` at the highest necessary level
- Use TypeScript for better type safety
2. **Performance**
- Memoize the editor configuration if it's complex
- Use Vue's `shallowRef` for editor instance if needed
- Avoid unnecessary re-renders of the editor
### Common Use Cases
**Form Integration**
```vue
<template>
<form @submit.prevent="handleSubmit">
<MilkdownEditorWrapper />
<button type="submit" :disabled="isLoading">Submit</button>
</form>
</template>
<script>
import { useInstance } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "FormWithEditor",
setup: () => {
const [isLoading, getInstance] = useInstance();
const handleSubmit = () => {
if (isLoading.value) return;
const editor = getInstance();
if (!editor) return;
const content = editor.getMarkdown();
// Submit form with content
};
return {
isLoading,
handleSubmit,
};
},
});
</script>
```
**Auto-save**
```vue
<template>
<Milkdown />
</template>
<script>
import { Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { listener, listenerCtx } from "@milkdown/kit/plugin/listener";
import { Milkdown, useEditor } from "@milkdown/vue";
import { defineComponent } from "vue";
export default defineComponent({
name: "AutoSaveEditor",
components: {
Milkdown,
},
setup: () => {
const { get } = useEditor((root) =>
Editor.make()
.config((ctx) => {
ctx.set(rootCtx, root);
// Add markdown listener for auto-save
ctx.get(listenerCtx).markdownUpdated((ctx, markdown) => {
// Save content to your backend or storage
saveToBackend(markdown);
});
})
.use(commonmark)
.use(listener),
);
},
});
</script>
```
## More Examples
---
- [Examples Repository](https://github.com/Milkdown/examples)
-44
View File
@@ -1,44 +0,0 @@
# Vue2
We don't provide Vue2 support out of box, but you can use the vanilla version with it easily.
## Install the Dependencies
```bash
# install with npm
npm install @milkdown/kit
npm install @milkdown/theme-nord
```
## Create a Component
Create a component is pretty easy.
```html
<template>
<div ref="editor"></div>
</template>
<script>
import { defaultValueCtx, Editor, rootCtx } from "@milkdown/kit/core";
import { commonmark } from "@milkdown/kit/preset/commonmark";
import { nord } from "@milkdown/theme-nord";
export default {
name: "Editor",
props: {
msg: String,
},
mounted() {
Editor.make()
.config((ctx) => {
ctx.set(rootCtx, this.$refs.editor);
ctx.set(defaultValueCtx, this.$props.msg);
})
.config(nord)
.use(commonmark)
.create();
},
};
</script>
```
+94 -1
View File
@@ -7,6 +7,20 @@
<div class="doc-card__time">{{ displayTime }}</div>
</div>
<div class="doc-card__actions">
<button type="button" class="doc-card__btn" :title="'压缩文档'" @click="handleCompress">
<svg v-if="compressState === 'idle' || compressState === 'completed'" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M4 14h6v7H4z"/>
<path d="M14 9h6v12h-6z"/>
<path d="M4 9h6v5H4z"/>
</svg>
<span v-if="compressState === 'queued' || compressState === 'processing'" class="doc-card__spinner"></span>
<svg v-else-if="compressState === 'error'" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<circle cx="12" cy="12" r="10"/>
<line x1="15" y1="9" x2="9" y2="15"/>
<line x1="9" y1="9" x2="15" y2="15"/>
</svg>
</button>
<span v-if="compressState === 'queued'" class="doc-card__status-label">排队中</span>
<button type="button" class="doc-card__btn" :title="collapsedState ? '展开文件' : '折叠文件'" @click="toggleCollapse">
<svg v-if="collapsedState" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<polyline points="9 18 15 12 9 6"/>
@@ -39,7 +53,7 @@ import { Crepe } from '@milkdown/crepe'
import { editorViewCtx } from '@milkdown/kit/core'
import { copilotPlugin, copilotConfigCtx, copilotGhostMark, setCopilotEnabled, clearGhostSuggestion } from '../plugins/copilotPlugin'
import { hiddenTextInputPlugin, hiddenTextNode, hiddenTextRemark, hiddenTextView } from '../plugins/hiddenTextPlugin'
import { fetchSuggestion } from '../utils/api.js'
import { fetchSuggestion, submitCompress, pollCompressStatus } from '../utils/api.js'
import { isDocumentVisible, getRecommendedDebounce, getRecommendedSyncInterval } from '../composables/useVisibility.js'
const props = defineProps({
@@ -52,14 +66,52 @@ const props = defineProps({
onUpdateContent: { type: Function, default: null },
onUpdateCollapsed: { type: Function, default: null },
onDelete: { type: Function, default: null },
onCompress: { type: Function, default: null },
})
const editorRoot = ref(null)
const collapsedState = ref(Boolean(props.collapsed))
const currentContent = ref(props.content || '')
const compressState = ref('idle') // idle | queued | processing | error
let crepe = null
let syncTimer = null
let syncingExternal = false
let compressPoller = null
const handleCompress = () => {
if (compressState.value !== 'idle') return
//
const content = crepe?.getMarkdown() || ''
if (!content.trim()) {
compressState.value = 'error'
setTimeout(() => { compressState.value = 'idle' }, 2000)
return
}
submitCompress(content, props.docType).then((result) => {
compressState.value = 'queued'
if (compressPoller) compressPoller.stop()
compressPoller = pollCompressStatus(result.task_id, (status, compressedContent, message) => {
compressState.value = status
if (status === 'completed') {
//
crepe.editor.action(replaceAll(compressedContent))
} else if (status === 'error') {
setTimeout(() => { compressState.value = 'idle' }, 3000)
}
})
if (compressState.value === 'queued') {
// Task already completed before polling started
}
}).catch(() => {
compressState.value = 'error'
setTimeout(() => { compressState.value = 'idle' }, 3000)
})
}
const typeLabel = computed(() => {
if (props.docType === 'docx') return 'DOCX'
@@ -84,6 +136,10 @@ const syncContent = () => {
if (!crepe) return
// Skip content sync when tab is hidden (energy saving for nested editors)
if (!isDocumentVisible()) return
// Don't sync during compression to avoid overwriting compressed content
if (compressState.value !== 'idle') return
if (syncTimer) clearTimeout(syncTimer)
const syncInterval = getRecommendedSyncInterval(120)
syncTimer = setTimeout(async () => {
@@ -96,11 +152,24 @@ const syncContent = () => {
const syncExternalContent = async (nextValue) => {
const value = nextValue || ''
// Reject empty content to prevent accidental document clearing (e.g., from failed compression)
if (!value || !value.trim()) {
return
}
if (!crepe) {
currentContent.value = value
return
}
if (value === currentContent.value) return
// Clear pending sync timer to prevent stale content from overwriting new content
if (syncTimer) {
clearTimeout(syncTimer)
syncTimer = null
}
syncingExternal = true
try {
crepe.editor.action(replaceAll(value))
@@ -173,6 +242,10 @@ onUnmounted(() => {
clearTimeout(syncTimer)
syncTimer = null
}
if (compressPoller) {
compressPoller.stop()
compressPoller = null
}
if (crepe) {
crepe.editor.action((ctx) => {
const view = ctx.get(editorViewCtx)
@@ -262,6 +335,26 @@ onUnmounted(() => {
.doc-card__actions {
display: flex;
gap: 4px;
align-items: center;
}
.doc-card__status-label {
font-size: 10px;
color: #f59e0b;
white-space: nowrap;
}
.doc-card__spinner {
width: 14px;
height: 14px;
border: 2px solid rgba(59, 130, 246, 0.2);
border-top-color: #3b82f6;
border-radius: 50%;
animation: doc-card-spin 0.6s linear infinite;
}
@keyframes doc-card-spin {
to { transform: rotate(360deg); }
}
.doc-card__btn {
+62 -9
View File
@@ -322,10 +322,10 @@ import { useTheme } from '../composables/useTheme.js'
import { OCR_URL, EXPORT_PDF_URL } from '../utils/config.js'
import TTSMenu from './TTSMenu.vue'
import TTSPlayer from './TTSPlayer.vue'
import { convertFileToMarkdown } from '../utils/convert.js'
import { convertFileToMarkdown, convertAudioToText } from '../utils/convert.js'
import { setOcrCache, clearOcrCache, clearAllOcrCache, IMAGE_SIZE_LIMIT, calculateImageHash, getOcrByHash, setOcrByHash } from '../utils/ocrCache.js'
import { isDocumentVisible, getRecommendedDebounce, getRecommendedSyncInterval } from '../composables/useVisibility.js'
import { DOC_BLOCK_NODE_TYPE, getDocTypeFromFilename, isSupportedDocFile, transformDocBlockMarkdownForClipboard, transformLegacyDocBlocksForExport, transformSpecialDocBlocksToLegacy } from '../utils/docBlock.js'
import { DOC_BLOCK_NODE_TYPE, getDocTypeFromFilename, isSupportedDocFile, transformDocBlockMarkdownForClipboard, transformLegacyDocBlocksForExport, transformSpecialDocBlocksToLegacy, isAudioFile } from '../utils/docBlock.js'
import { isUploadBlockTypeAllowed } from '../utils/uploadBlock.js'
const emit = defineEmits(['update:markdown'])
@@ -384,13 +384,14 @@ const acceptAll = computed(() => {
const types = [
'.txt', '.json', '.toml', '.yaml', '.yml',
'.docx', '.pptx', '.pdf',
'.wav', '.mp3', '.m4a', '.ogg', '.flac',
'.png', '.jpg', '.jpeg', '.gif', '.webp', '.bmp', '.svg', '.heic', '.heif', '.avif',
'text/plain', 'application/json',
'text/yaml', 'text/x-yaml', 'application/x-yaml',
'application/pdf',
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
'image/*'
'image/*', 'audio/*'
]
return types.join(',')
})
@@ -971,7 +972,7 @@ const warnUnsupportedUploadType = () => {
}
const warnUnsupportedInsertType = () => {
alert(t('uploadFileTypeWarning') || 'Unsupported file type. Supported: doc/docx/ppt/pptx/pdf/zip, images, txt/json.')
alert(t('uploadFileTypeWarning') || '不支持的文件类型。支持:doc/docx/ppt/pptx/pdf/wav/mp3/m4a/ogg/flac, 图片, txt/json.')
}
const warnUploadError = (message = '') => {
@@ -989,6 +990,60 @@ const warnImageTooLarge = () => {
alert(t('imgTooLarge') || `Image too large. Max ${limitMB}MB.`)
}
const consumeSseResult = async (res) => {
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
if (!res.body) {
throw new Error('流式响应不可用')
}
const reader = res.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
let boundary = buffer.indexOf('\n\n')
while (boundary >= 0) {
const chunk = buffer.slice(0, boundary)
buffer = buffer.slice(boundary + 2)
const lines = String(chunk).replace(/\r/g, '').split('\n')
let event = 'message'
const dataLines = []
for (const line of lines) {
if (!line) continue
if (line.startsWith('event:')) {
event = line.slice(6).trim() || 'message'
continue
}
if (line.startsWith('data:')) {
dataLines.push(line.slice(5).trimStart())
}
}
const data = dataLines.length ? JSON.parse(dataLines.join('\n')) : {}
if (event === 'done') {
return data.result || data
}
if (event === 'error') {
throw new Error(String(data.error || '请求失败'))
}
if (event === 'cancelled') {
throw new Error('请求已取消')
}
boundary = buffer.indexOf('\n\n')
}
}
throw new Error('未收到完成事件')
}
const performOCR = async (file, cacheKey, imageHash = '') => {
if (!aiEnabled.value) return
@@ -1012,11 +1067,7 @@ const performOCR = async (file, cacheKey, imageHash = '') => {
language: 'auto'
})
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
const data = await res.json()
const data = await consumeSseResult(res)
if (data.text) {
setOcrCache(cacheKey, data.text)
setOcrCache(file.name, data.text)
@@ -1135,6 +1186,8 @@ const parseDocFilesToBlocks = async (docFiles) => {
content = await file.text()
} else if (isConvertibleFile(file)) {
content = await convertFileToMarkdown(file)
} else if (isAudioFile(file)) {
content = await convertAudioToText(file)
} else {
throw new Error('不支持的文件类型')
}
+9 -9
View File
@@ -123,21 +123,21 @@ watch(
aspect-ratio: 1 / 1;
min-width: 108px;
border-radius: 22px;
border: 2px solid #2563eb;
background: linear-gradient(180deg, rgba(219, 234, 254, 0.72) 0%, rgba(191, 219, 254, 0.95) 100%);
border: 2px solid var(--upload-card-border);
background: var(--upload-card-bg);
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 18px;
cursor: pointer;
box-shadow: 0 16px 38px rgba(37, 99, 235, 0.16);
box-shadow: var(--upload-card-shadow);
transition: transform 0.18s ease, box-shadow 0.18s ease, border-color 0.18s ease;
}
.upload-block-card:hover {
transform: translateY(-2px);
box-shadow: 0 20px 42px rgba(37, 99, 235, 0.2);
box-shadow: 0 20px 42px rgba(249, 115, 22, 0.28);
}
.upload-block-card.is-uploading {
@@ -152,7 +152,7 @@ watch(
}
.upload-block-label {
color: #111827;
color: var(--app-text);
font-size: 15px;
font-weight: 700;
line-height: 1;
@@ -178,7 +178,7 @@ watch(
width: 100%;
height: 4px;
border-radius: 999px;
background: #2563eb;
background: var(--upload-card-border);
transform: translate(-50%, -50%);
}
@@ -193,7 +193,7 @@ watch(
padding: 0 18px 0 0;
border: none;
background: transparent;
color: #111827;
color: var(--app-text);
font-size: 13px;
font-weight: 600;
line-height: 1;
@@ -210,7 +210,7 @@ watch(
right: 0;
top: 50%;
transform: translateY(-50%);
color: #111827;
color: var(--app-text);
pointer-events: none;
}
@@ -225,7 +225,7 @@ watch(
:global(.upload-block-node-view.ProseMirror-selectednode .upload-block-card) {
outline: none !important;
box-shadow: 0 16px 38px rgba(37, 99, 235, 0.16) !important;
box-shadow: var(--upload-card-shadow) !important;
}
:global(.upload-block-node-view.ProseMirror-selectednode)::after {
+2 -41
View File
@@ -5,14 +5,13 @@ import { Node as ProseNode, Slice } from '@milkdown/prose/model'
import type { Ctx } from '@milkdown/kit/core'
import { Decoration, DecorationSet, type EditorView } from '@milkdown/prose/view'
import { extractDocBlockContextFromMarkdown } from '../utils/docBlock.js'
import { getOcrCache, OCR_SIZE_LIMIT, extractTextFromOCR } from '../utils/ocrCache'
import { getOcrCache, OCR_SIZE_LIMIT, extractTextFromOCR, buildOcrContextForDoc } from '../utils/ocrCache'
import { isDocumentVisible } from '../composables/useVisibility.js'
const COPILOT_PLUGIN_KEY = new PluginKey('milkdown-copilot')
const DEBOUNCE_MS = 1000
const SIZE_LIMIT = OCR_SIZE_LIMIT
const DOC_SIZE_LIMIT = 32 * 1024 // 文档块32KB限制
const IMAGE_NODE_TYPES = new Set(['image', 'image-block', 'imageBlock'])
const FALLBACK_BLOCK_SEPARATOR = '\n\n'
const FALLBACK_LEAF_TEXT = '\n'
@@ -279,23 +278,6 @@ function insertPlainText(view: EditorView, suggestion: string, from: number, mar
view.dispatch(tr)
}
function getImageSrc(node: ProseNode): string {
const src = node.attrs?.src
return typeof src === 'string' ? src : ''
}
function isImageNodeWithSrc(node: ProseNode): boolean {
return IMAGE_NODE_TYPES.has(node.type.name) && Boolean(getImageSrc(node))
}
function getImageLabel(node: ProseNode): string {
const candidates = [node.attrs?.alt, node.attrs?.title, node.attrs?.caption]
for (const value of candidates) {
if (typeof value === 'string' && value.trim()) return value.trim()
}
return 'untitled'
}
function serializeRangeToMarkdown(
doc: ProseNode,
from: number,
@@ -310,27 +292,6 @@ function serializeRangeToMarkdown(
return sliceDoc ? serializer(sliceDoc) : doc.textBetween(from, to, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
}
function buildOcrContextForRequest(doc: ProseNode, cursorPos: number): string {
const lines: string[] = []
doc.nodesBetween(0, cursorPos, (node) => {
if (!isImageNodeWithSrc(node)) return true
const src = getImageSrc(node)
const ocrText = getOcrCache(src)
if (!ocrText) return true
const textOnly = extractTextFromOCR(ocrText, 100)
if (!textOnly) return true
const label = getImageLabel(node)
lines.push(`![${label}](${src}) <OCR:${textOnly}>`)
return true
})
if (lines.length === 0) return ''
return lines.join('\n')
}
function doFetchSuggestion(
view: EditorView,
runtime: CopilotRuntime,
@@ -404,7 +365,7 @@ function scheduleFetch(view: EditorView, runtime: CopilotRuntime, pos: number) {
}
// 构建上下文:OCR内容 + 上传文档内容
const ocrContext = buildOcrContextForRequest(doc, pos)
const ocrContext = buildOcrContextForDoc(doc, 100)
// 从markdown中提取文档块内容用于AI补全上下文
const docContext = extractDocBlockContextFromMarkdown(prefixMarkdown + suffixMarkdown, 500)
+46
View File
@@ -145,6 +145,7 @@ class DocBlockNodeView implements NodeView {
onUpdateContent: (content: string) => this.updateAttrs({ content }),
onUpdateCollapsed: (collapsed: boolean) => this.updateAttrs({ collapsed }),
onDelete: () => this.deleteNode(),
onCompress: (compressedContent: string) => this.compressWithNewContent(compressedContent),
resolveSuggestionRequest: (payload: { prefix: string; suffix: string; languageId: string }) => this.resolveSuggestionRequest(payload),
})
this.mount()
@@ -175,6 +176,33 @@ class DocBlockNodeView implements NodeView {
this.view.focus()
}
destroy(hasNewHeader: boolean, hasNewFooter: boolean) {
if (this.app) {
this.app.unmount()
this.app = null
}
}
compressWithNewContent(compressedContent: string) {
const pos = this.getPosValue()
if (pos === undefined) return
// Dispatch setNodeMarkup with compressed content. ProseMirror will destroy
// the old NodeView (destroy() unmounts Vue app) and create a new one.
// The new instance's constructor reads from node.attrs which now has the compressed content.
const nextAttrs = {
docType: this.node.attrs.docType,
docName: this.node.attrs.docName,
uploadTime: this.node.attrs.uploadTime,
content: compressedContent,
collapsed: false,
}
this.view.dispatch(
this.view.state.tr.setNodeMarkup(pos, undefined, nextAttrs)
)
}
resolveSuggestionRequest(payload: { prefix: string; suffix: string; languageId: string }) {
const pos = this.getPosValue()
if (pos === undefined) return payload
@@ -342,3 +370,21 @@ export const docBlockView = $view(docBlockNode, (ctx) => {
export function buildDocContextFromDoc(doc: ProseNode, excludePos?: number) {
return buildDocContext(doc, excludePos)
}
export function checkSizeLimit(view: EditorView): { size: number; overLimit: boolean } {
let visibleChars = 0
let hiddenChars = 0
view.state.doc.descendants((node) => {
if (node.type.name === 'doc_block' && node.attrs.content) {
// 文档块内容是隐藏的,单独统计
hiddenChars += String(node.attrs.content).length
} else if (node.text) {
// 普通文本节点
visibleChars += node.text.length
}
})
const total = visibleChars + hiddenChars
return { size: total, overLimit: total > SIZE_LIMIT }
}
+66 -34
View File
@@ -6,7 +6,8 @@ import { type Node as ProseNode, Slice, type Schema } from '@milkdown/prose/mode
import { Decoration, DecorationSet, type EditorView, type NodeView } from '@milkdown/prose/view'
import ProBlockCrepe from '../components/ProBlockCrepe.vue'
import { extractDocBlockContextFromMarkdown } from '../utils/docBlock.js'
import { extractTextFromOCR, getOcrCache } from '../utils/ocrCache'
import { buildOcrContextForDoc } from '../utils/ocrCache'
import { normalizeProAcceptMarkdown, splitPlainTextFallbackBlocks } from '../utils/proAccept.js'
import { PRO_BLOCK_NODE_TYPE, PRO_DISPLAY_LABEL, PRO_TRIGGER_TEXT, parseProBlockSyntax, serializeProBlockSyntax } from '../utils/proBlock.js'
const PRO_BLOCK_INPUT_PLUGIN_KEY = new PluginKey('milkdown-pro-block-input')
@@ -15,7 +16,6 @@ const PRO_BLOCK_INPUT_META = 'pro-block-input-meta'
const PRO_CONTEXT_LIMIT = 32 * 1024
const FALLBACK_BLOCK_SEPARATOR = '\n\n'
const FALLBACK_LEAF_TEXT = '\n'
const IMAGE_NODE_TYPES = new Set(['image', 'image-block', 'imageBlock'])
interface ProBlockConfig {
fetchSuggestionStream: (payload: {
@@ -114,26 +114,6 @@ function findProBlockReplacements(doc: ProseNode) {
return replacements
}
function buildOcrContext(doc: ProseNode) {
const lines: string[] = []
doc.descendants((node) => {
if (!IMAGE_NODE_TYPES.has(node.type.name)) return true
const src = typeof node.attrs?.src === 'string' ? node.attrs.src : ''
if (!src) return true
const ocrText = getOcrCache(src)
const preview = ocrText ? extractTextFromOCR(ocrText, 120) : ''
if (!preview) return true
const label = typeof node.attrs?.alt === 'string' && node.attrs.alt.trim() ? node.attrs.alt.trim() : 'image'
lines.push(`![${label}](${src}) <OCR:${preview}>`)
return true
})
return lines.join('\n')
}
function createHighlightDecorations(doc: ProseNode, from: number, to: number) {
if (to <= from) return DecorationSet.empty
@@ -154,6 +134,15 @@ function createHighlightDecorations(doc: ProseNode, from: number, to: number) {
return DecorationSet.create(doc, decorations)
}
function getTransactionInsertedRange(tr: any, from: number, to: number, beforeSize: number) {
const deleteSize = Math.max(0, to - from)
const insertedSize = Math.max(0, tr.doc.content.size - beforeSize + deleteSize)
const startPos = Math.max(0, Math.min(from, tr.doc.content.size))
const endPos = Math.max(startPos, Math.min(startPos + insertedSize, tr.doc.content.size))
if (endPos <= startPos) return null
return { from: startPos, to: endPos }
}
function replaceWithParsedMarkdownSlice(tr: any, from: number, to: number, parsedDoc: ProseNode) {
if (!parsedDoc || parsedDoc.content.size <= 0) return null
@@ -161,17 +150,45 @@ function replaceWithParsedMarkdownSlice(tr: any, from: number, to: number, parse
if (!parsedSlice || parsedSlice.size <= 0) return null
const beforeSize = tr.doc.content.size
const deleteSize = Math.max(0, to - from)
try {
tr.replaceRange(from, to, parsedSlice)
} catch {
return null
}
const insertedSize = Math.max(0, tr.doc.content.size - beforeSize + deleteSize)
const startPos = Math.max(0, Math.min(from, tr.doc.content.size))
const endPos = Math.max(startPos, Math.min(startPos + insertedSize, tr.doc.content.size))
if (endPos <= startPos) return null
return { from: startPos, to: endPos }
return getTransactionInsertedRange(tr, from, to, beforeSize)
}
function replaceWithTextFallback(
tr: any,
from: number,
to: number,
schema: Schema,
source: string
) {
const text = String(source || '')
if (!text.trim()) return null
const paragraphType = schema.nodes.paragraph
const codeBlockType = schema.nodes.code_block || schema.nodes.codeBlock
const beforeSize = tr.doc.content.size
try {
if (codeBlockType && text.includes('\n')) {
tr.replaceWith(from, to, codeBlockType.create(null, schema.text(text)))
return getTransactionInsertedRange(tr, from, to, beforeSize)
}
if (!paragraphType) return null
const paragraphs = splitPlainTextFallbackBlocks(text)
const nodes = paragraphs.map((block) => paragraphType.create(null, schema.text(block)))
if (nodes.length === 0) return null
tr.replaceWith(from, to, nodes)
return getTransactionInsertedRange(tr, from, to, beforeSize)
} catch {
return null
}
}
function insertProBlockNode(view: EditorView, autoStart: boolean) {
@@ -420,7 +437,7 @@ class ProBlockNodeView implements NodeView {
const suffixStart = Math.min(pos + this.node.nodeSize, doc.content.size)
const suffixMarkdown = serializeRangeToMarkdown(doc, suffixStart, doc.content.size, schema, this.serializer)
|| doc.textBetween(suffixStart, doc.content.size, FALLBACK_BLOCK_SEPARATOR, FALLBACK_LEAF_TEXT)
const ocrContext = buildOcrContext(doc)
const ocrContext = buildOcrContextForDoc(doc, 120)
const docContext = extractDocBlockContextFromMarkdown(`${prefixMarkdown}\n\n${suffixMarkdown}`, 1600)
const fullPrefix = [ocrContext, docContext, prefixMarkdown].filter(Boolean).join('\n\n')
@@ -515,8 +532,19 @@ class ProBlockNodeView implements NodeView {
this.setStage('thinking', this.props.previewContent || '')
return
}
if (event === 'chunk') {
// 'chunk' events are handled by onChunk callback
return
}
if (event === 'done') {
// When done, update stage to 'done' and clear any error messages
this.setStage('done', '')
this.props.errorMessage = ''
return
}
if (event === 'error') {
this.props.errorMessage = String(data?.error || this.config.t('proErrorActionable') || 'PRO 模式生成失败,请重试。')
this.setStage('error', this.props.previewContent || '')
}
},
})
@@ -558,7 +586,7 @@ class ProBlockNodeView implements NodeView {
async acceptResult() {
if (this.props.isBusy) return
const source = normalizeProMarkdown(this.props.activeContent || this.props.previewContent || '')
const source = normalizeProAcceptMarkdown(this.props.activeContent || this.props.previewContent || '')
if (!source) return
const pos = this.getPosValue()
@@ -572,14 +600,18 @@ class ProBlockNodeView implements NodeView {
try {
const parsedDoc = await this.parser(source)
insertedRange = replaceWithParsedMarkdownSlice(tr, from, to, parsedDoc)
} catch {
} catch (e) {
console.error('PRO block parse failed:', e)
insertedRange = null
}
if (!insertedRange) {
this.props.errorMessage = this.config.t('proAcceptParseError') || '无法把当前结果解析为 Markdown,请重试生成。'
this.setStage('error', source)
return
insertedRange = replaceWithTextFallback(tr, from, to, this.view.state.schema, source)
if (!insertedRange) {
this.props.errorMessage = this.config.t('proAcceptParseError') || '无法接受当前 PRO 结果,请重试生成。'
this.setStage('error', source)
return
}
}
const endPos = Math.min(insertedRange.to, tr.doc.content.size)
+14
View File
@@ -8,6 +8,10 @@
-moz-osx-font-smoothing: grayscale;
}
:root {
--accent-orange: #f97316;
}
:root,
:root[data-theme='light'] {
color-scheme: light;
@@ -70,6 +74,11 @@
--crepe-color-hover: #e0e0e0;
--crepe-color-selected: #d5d5d5;
--crepe-color-inline-area: #cacaca;
/* UploadBlock orange tokens (light) */
--upload-card-bg: linear-gradient(180deg, rgba(255, 237, 213, 0.8) 0%, rgba(254, 215, 170, 0.9) 100%);
--upload-card-border: var(--accent-orange, #f97316);
--upload-card-shadow: 0 16px 38px rgba(249, 115, 22, 0.16);
}
/* GitHub-like light tokens (used by DocsView.gitHub styled components) */
@@ -157,6 +166,11 @@
--crepe-color-hover: #232323;
--crepe-color-selected: #2f2f2f;
--crepe-color-inline-area: #2b2b2b;
/* UploadBlock orange tokens (dark) */
--upload-card-bg: linear-gradient(180deg, rgba(43, 26, 5, 0.9) 0%, rgba(78, 32, 6, 0.95) 100%);
--upload-card-border: var(--accent-orange, #f97316);
--upload-card-shadow: 0 16px 38px rgba(249, 115, 22, 0.3);
}
:root[data-theme='light'] .milkdown {
+290 -260
View File
@@ -1,4 +1,15 @@
import { API_URL, API_KEY, PRO_STREAM_URL, PRO_FRONTEND_TIMEOUT_MS, TTS_URL, TTS_STATUS_URL, TTS_CONFIG_URL } from './config.js'
import {
API_URL,
API_KEY,
PRO_URL,
PRO_FRONTEND_TIMEOUT_MS,
TTS_URL,
TTS_STATUS_URL,
TTS_CONFIG_URL,
COMPRESS_SUBMIT_URL,
COMPRESS_STATUS_URL,
JOB_LOAD_URL,
} from './config.js'
import { useSettingsStore } from '../stores/settings'
function generateRequestId() {
@@ -8,27 +19,6 @@ function generateRequestId() {
return `${Date.now()}-${Math.random().toString(16).slice(2)}`
}
function getCancelUrl(apiUrl) {
const normalized = String(apiUrl || '').replace(/\/+$/, '')
if (!normalized) return '/v1/completions/cancel'
if (/\/v1\/pro\/completions\/stream$/i.test(normalized)) {
return normalized.replace(/\/v1\/pro\/completions\/stream$/i, '/v1/completions/cancel')
}
if (normalized.endsWith('/v1/completions')) {
return `${normalized}/cancel`
}
return `${normalized}/cancel`
}
function getProCancelUrl(apiUrl) {
const normalized = String(apiUrl || '').replace(/\/+$/, '')
if (!normalized) return '/v1/completions/cancel'
if (/\/v1\/pro\/completions$/i.test(normalized)) {
return normalized.replace(/\/v1\/pro\/completions$/i, '/v1/completions/cancel')
}
return normalized.replace(/\/v1\/pro\/completions\/stream$/i, '/v1/completions/cancel')
}
function normalizeAbortReason(reason) {
if (typeof reason === 'string' && reason.trim()) {
return reason.trim().slice(0, 64)
@@ -36,46 +26,6 @@ function normalizeAbortReason(reason) {
return 'abort'
}
async function sendCancelRequest(cancelUrl, requestId, reason) {
try {
await fetch(cancelUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': API_KEY,
},
body: JSON.stringify({
request_id: requestId,
reason,
}),
})
} catch {
// Cancel request failed silently
}
}
function createAbortError(message = 'Request aborted') {
const error = new Error(message)
error.name = 'AbortError'
return error
}
function buildCompletionBody(settings, prefix, suffix, languageId, extra = {}) {
return {
prefix,
suffix,
languageId,
model_thinking: settings.modelThinking,
privacy_mode: settings.privacyMode,
user_preferences: {
language: settings.language,
currency: settings.currency,
timezone: settings.detectedTimezone,
},
...extra,
}
}
function parseSseEvent(rawEvent) {
const lines = String(rawEvent || '').replace(/\r/g, '').split('\n')
let event = 'message'
@@ -98,6 +48,156 @@ function parseSseEvent(rawEvent) {
}
}
function createAbortError(message = 'Request aborted') {
const error = new Error(message)
error.name = 'AbortError'
return error
}
async function sendCancelRequest(cancelUrl, requestId, reason) {
try {
await fetch(cancelUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': API_KEY,
},
body: JSON.stringify({
request_id: requestId,
reason,
}),
})
} catch {
// Best-effort cancel only
}
}
function getCancelUrl(apiUrl) {
const normalized = String(apiUrl || '').replace(/\/+$/, '')
if (/\/v1\/pro\/completions$/i.test(normalized)) {
return normalized.replace(/\/v1\/pro\/completions$/i, '/v1/pro/completions/cancel')
}
if (/\/v1\/completions$/i.test(normalized)) {
return normalized.replace(/\/v1\/completions$/i, '/v1/completions/cancel')
}
return `${normalized}/cancel`
}
function buildCompletionBody(settings, prefix, suffix, languageId, extra = {}) {
return {
prefix,
suffix,
languageId,
model_thinking: settings.modelThinking,
privacy_mode: settings.privacyMode,
user_preferences: {
language: settings.language,
currency: settings.currency,
timezone: settings.detectedTimezone,
},
...extra,
}
}
async function consumeSseJson({
url,
body,
requestId,
signal,
timeoutMs,
onChunk,
onEvent,
onDone,
}) {
const requestController = new AbortController()
const timeoutId = timeoutMs ? setTimeout(() => requestController.abort('timeout'), timeoutMs) : null
const cancelUrl = getCancelUrl(url)
const relayAbort = () => {
requestController.abort(signal?.reason || 'abort')
}
const onAbort = () => {
const reason = normalizeAbortReason(requestController.signal.reason)
void sendCancelRequest(cancelUrl, requestId, reason)
}
requestController.signal.addEventListener('abort', onAbort, { once: true })
if (signal) {
if (signal.aborted) {
relayAbort()
} else {
signal.addEventListener('abort', relayAbort, { once: true })
}
}
try {
const res = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Request-Id': requestId,
'X-API-Key': API_KEY,
},
body: JSON.stringify(body),
signal: requestController.signal,
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
if (!res.body) {
throw new Error('流式响应不可用')
}
const reader = res.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
let finalPayload = null
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
let boundary = buffer.indexOf('\n\n')
while (boundary >= 0) {
const chunk = buffer.slice(0, boundary)
buffer = buffer.slice(boundary + 2)
const parsed = parseSseEvent(chunk)
const data = parsed.data ? JSON.parse(parsed.data) : {}
onEvent?.(parsed.event, data)
if (parsed.event === 'result') {
onChunk?.(data)
} else if (parsed.event === 'done') {
finalPayload = data.result || data
return onDone ? onDone(finalPayload) : finalPayload
} else if (parsed.event === 'error') {
throw new Error(String(data.error || '请求失败'))
} else if (parsed.event === 'cancelled') {
throw createAbortError('请求已取消')
}
boundary = buffer.indexOf('\n\n')
}
}
if (requestController.signal.aborted) {
throw createAbortError('请求已中止')
}
return onDone ? onDone(finalPayload || {}) : finalPayload
} finally {
if (timeoutId) clearTimeout(timeoutId)
requestController.signal.removeEventListener('abort', onAbort)
if (signal) {
signal.removeEventListener('abort', relayAbort)
}
}
}
export async function fetchSuggestion(prefix, suffix, languageId, signal, apiUrl = API_URL) {
let normalizedLanguageId = 'markdown'
if (typeof languageId === 'string' && languageId.trim()) {
@@ -109,60 +209,32 @@ export async function fetchSuggestion(prefix, suffix, languageId, signal, apiUrl
apiUrl = signal
signal = undefined
}
const settings = useSettingsStore()
const requestId = generateRequestId()
const cancelUrl = getCancelUrl(apiUrl)
let finalContent = ''
const onAbort = () => {
const reason = normalizeAbortReason(signal?.reason)
void sendCancelRequest(cancelUrl, requestId, reason)
}
const result = await consumeSseJson({
url: apiUrl,
body: buildCompletionBody(settings, prefix, suffix, normalizedLanguageId),
requestId,
signal,
onChunk(data) {
if (typeof data.content === 'string') {
finalContent = data.content
} else if (typeof data.delta === 'string') {
finalContent += data.delta
}
},
onDone(data) {
return typeof data.content === 'string' ? data.content : finalContent
},
})
if (signal) {
if (signal.aborted) {
onAbort()
} else {
signal.addEventListener('abort', onAbort, { once: true })
}
}
try {
const settings = useSettingsStore()
const headers = {
'Content-Type': 'application/json',
'X-Request-Id': requestId,
'X-API-Key': API_KEY,
}
const body = buildCompletionBody(settings, prefix, suffix, normalizedLanguageId)
const res = await fetch(apiUrl, {
method: 'POST',
headers,
body: JSON.stringify(body),
signal,
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
const data = await res.json()
return data.content || ''
} catch (e) {
if (e.name === 'AbortError') {
// ignore abort
} else {
throw e
}
} finally {
if (signal) {
signal.removeEventListener('abort', onAbort)
}
}
return result || ''
}
export async function fetchProSuggestionStream(payload, apiUrl = PRO_STREAM_URL) {
export async function fetchProSuggestionStream(payload, apiUrl = PRO_URL) {
const {
prefix = '',
suffix = '',
@@ -176,168 +248,65 @@ export async function fetchProSuggestionStream(payload, apiUrl = PRO_STREAM_URL)
const settings = useSettingsStore()
const requestId = generateRequestId()
const cancelUrl = getProCancelUrl(apiUrl)
const requestController = new AbortController()
const timeoutId = setTimeout(() => {
requestController.abort('timeout')
}, timeoutMs)
let finalContent = ''
const relayAbort = () => {
requestController.abort(signal?.reason || 'abort')
}
const onAbort = () => {
const reason = normalizeAbortReason(requestController.signal.reason)
void sendCancelRequest(cancelUrl, requestId, reason)
}
requestController.signal.addEventListener('abort', onAbort, { once: true })
if (signal) {
if (signal.aborted) {
relayAbort()
} else {
signal.addEventListener('abort', relayAbort, { once: true })
}
}
try {
const res = await fetch(apiUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Request-Id': requestId,
'X-API-Key': API_KEY,
return consumeSseJson({
url: apiUrl,
requestId,
signal,
timeoutMs,
body: {
prefix,
suffix,
languageId: String(languageId || 'markdown').trim() || 'markdown',
instruction,
pro_thinking: settings.proThinking || 'medium',
privacy_mode: settings.privacyMode,
user_preferences: {
language: settings.language,
currency: settings.currency,
timezone: settings.detectedTimezone,
},
body: JSON.stringify(
{
prefix,
suffix,
languageId: String(languageId || 'markdown').trim() || 'markdown',
instruction,
pro_thinking: settings.proThinking || 'medium',
privacy_mode: settings.privacyMode,
user_preferences: {
language: settings.language,
currency: settings.currency,
timezone: settings.detectedTimezone,
},
}
),
signal: requestController.signal,
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
if (!res.body) {
throw new Error('PRO 模式流式响应不可用')
}
const reader = res.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
let finalContent = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
let boundary = buffer.indexOf('\n\n')
while (boundary >= 0) {
const chunk = buffer.slice(0, boundary)
buffer = buffer.slice(boundary + 2)
const parsed = parseSseEvent(chunk)
if (parsed.event && parsed.event !== 'message') {
let eventData = {}
if (parsed.data) {
try {
eventData = JSON.parse(parsed.data)
} catch {
eventData = {}
}
}
onEvent?.(parsed.event, eventData)
}
if (parsed.event === 'chunk' && parsed.data) {
const data = JSON.parse(parsed.data)
const delta = String(data.delta || '')
if (delta) {
finalContent += delta
onChunk?.(delta)
}
}
if (parsed.event === 'done' && parsed.data) {
const data = JSON.parse(parsed.data)
return String(data.content || finalContent || '')
}
if (parsed.event === 'error' && parsed.data) {
const data = JSON.parse(parsed.data)
throw new Error(String(data.error || 'PRO 模式请求失败'))
}
if (parsed.event === 'cancelled') {
throw createAbortError('PRO 模式请求已取消')
}
boundary = buffer.indexOf('\n\n')
},
onChunk(data) {
const delta = String(data.delta || data.content || '')
if (delta) {
finalContent += delta
onChunk?.(delta)
}
}
if (requestController.signal.aborted) {
throw createAbortError('PRO 模式请求已中止')
}
return finalContent
} catch (e) {
if (e?.name === 'AbortError') {
throw e
}
throw e
} finally {
clearTimeout(timeoutId)
requestController.signal.removeEventListener('abort', onAbort)
if (signal) {
signal.removeEventListener('abort', relayAbort)
}
}
},
onEvent(event, data) {
if (event === 'progress' && data.phase === 'thinking') {
onEvent?.('thinking', data)
return
}
if (event === 'queued' || event === 'started' || event === 'resource') {
onEvent?.(event, data)
}
},
onDone(data) {
return String(data.content || finalContent || '')
},
})
}
export async function fetchTTS(text, instruct = '', apiUrl = TTS_URL) {
const res = await fetch(apiUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': API_KEY,
const requestId = generateRequestId()
return consumeSseJson({
url: apiUrl,
requestId,
body: { text, instruct, speaker: 'Vivian', format: 'wav' },
onDone(data) {
return data
},
body: JSON.stringify({ text, instruct, speaker: 'Vivian', format: 'wav' }),
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`TTS HTTP ${res.status}: ${errorText}`)
}
return res.json()
}
export async function fetchTTSStatus(apiUrl = TTS_STATUS_URL) {
const res = await fetch(apiUrl, {
headers: { 'X-API-Key': API_KEY },
})
if (!res.ok) {
throw new Error(`TTS Status HTTP ${res.status}`)
}
if (!res.ok) throw new Error(`TTS Status HTTP ${res.status}`)
return res.json()
}
@@ -345,10 +314,71 @@ export async function fetchTTSConfig(apiUrl = TTS_CONFIG_URL) {
const res = await fetch(apiUrl, {
headers: { 'X-API-Key': API_KEY },
})
if (!res.ok) throw new Error(`TTS Config HTTP ${res.status}`)
return res.json()
}
export async function submitCompress(content, docType = 'txt', apiUrl = COMPRESS_SUBMIT_URL) {
const res = await fetch(apiUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': API_KEY,
},
body: JSON.stringify({ content, docType }),
})
if (!res.ok) {
throw new Error(`TTS Config HTTP ${res.status}`)
const errorText = await res.text()
throw new Error(`压缩提交失败 HTTP ${res.status}: ${errorText}`)
}
return res.json()
}
export function pollCompressStatus(taskId, onStateChange, apiUrl = COMPRESS_STATUS_URL) {
let consecutiveErrors = 0
const interval = setInterval(async () => {
try {
const res = await fetch(`${apiUrl}?task_id=${encodeURIComponent(taskId)}`, {
headers: { 'X-API-Key': API_KEY },
})
if (!res.ok) {
consecutiveErrors++
if (consecutiveErrors >= 5) {
clearInterval(interval)
onStateChange('error', '', `请求失败 HTTP ${res.status}`)
}
return
}
consecutiveErrors = 0
const data = await res.json()
onStateChange(data.status, data.content || '', data.message, data)
if (['completed', 'error', 'cancelled'].includes(data.status)) {
clearInterval(interval)
}
} catch (err) {
consecutiveErrors++
if (consecutiveErrors >= 5) {
clearInterval(interval)
onStateChange('error', '', `网络异常: ${err.message || err}`)
}
}
}, 1000)
return { stop: () => clearInterval(interval) }
}
export async function fetchJobLoad(apiUrl = JOB_LOAD_URL) {
const res = await fetch(apiUrl, {
headers: { 'X-API-Key': API_KEY },
})
if (!res.ok) {
throw new Error(`Job Load HTTP ${res.status}`)
}
return res.json()
}
+8 -1
View File
@@ -4,7 +4,7 @@ const DEFAULT_API_BASE_URL = import.meta.env.DEV ? '' : 'https://api.imageteach.
const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || DEFAULT_API_BASE_URL
export const API_URL = import.meta.env.VITE_API_URL || `${API_BASE_URL}/v1/completions`
export const PRO_STREAM_URL = import.meta.env.VITE_PRO_STREAM_URL || `${API_BASE_URL}/v1/pro/completions/stream`
export const PRO_URL = import.meta.env.VITE_PRO_URL || `${API_BASE_URL}/v1/pro/completions`
export const PRO_FRONTEND_TIMEOUT_MS = Number(import.meta.env.VITE_PRO_FRONTEND_TIMEOUT_MS || 3660000)
export const OCR_URL = import.meta.env.VITE_OCR_URL || `${API_BASE_URL}/v1/ocr`
export const CONVERT_URL = import.meta.env.VITE_CONVERT_URL || `${API_BASE_URL}/v1/convert`
@@ -12,4 +12,11 @@ export const EXPORT_PDF_URL = import.meta.env.VITE_EXPORT_PDF_URL || '/v1/export
export const TTS_URL = import.meta.env.VITE_TTS_URL || `${API_BASE_URL}/v1/tts-asr/tts`
export const TTS_STATUS_URL = import.meta.env.VITE_TTS_STATUS_URL || `${API_BASE_URL}/v1/tts-asr/status`
export const TTS_CONFIG_URL = import.meta.env.VITE_TTS_CONFIG_URL || `${API_BASE_URL}/v1/tts-asr/config`
export const ASR_URL = import.meta.env.VITE_ASR_URL || `${API_BASE_URL}/v1/tts-asr/asr`
export const JOB_LOAD_URL = import.meta.env.VITE_JOB_LOAD_URL || `${API_BASE_URL}/v1/jobs/load`
export const API_KEY = import.meta.env.VITE_API_KEY || 'your-secret-key-here'
// Compression always goes to local backend (not through reverse proxy)
const COMPRESS_BASE_URL = import.meta.env.VITE_COMPRESS_BACKEND || 'http://localhost:8001'
export const COMPRESS_SUBMIT_URL = `${COMPRESS_BASE_URL}/v1/compress/submit`
export const COMPRESS_STATUS_URL = `${COMPRESS_BASE_URL}/v1/compress/status`
+272 -7
View File
@@ -1,4 +1,71 @@
import { CONVERT_URL } from './config.js'
import { CONVERT_URL, ASR_URL } from './config.js'
function parseSseEvent(rawEvent) {
const lines = String(rawEvent || '').replace(/\r/g, '').split('\n')
let event = 'message'
const dataLines = []
for (const line of lines) {
if (!line) continue
if (line.startsWith('event:')) {
event = line.slice(6).trim() || 'message'
continue
}
if (line.startsWith('data:')) {
dataLines.push(line.slice(5).trimStart())
}
}
return {
event,
data: dataLines.join('\n'),
}
}
async function consumeSseResult(res) {
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
if (!res.body) {
throw new Error('流式响应不可用')
}
const reader = res.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
let finalResult = null
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
let boundary = buffer.indexOf('\n\n')
while (boundary >= 0) {
const chunk = buffer.slice(0, boundary)
buffer = buffer.slice(boundary + 2)
const parsed = parseSseEvent(chunk)
const data = parsed.data ? JSON.parse(parsed.data) : {}
if (parsed.event === 'done') {
finalResult = data.result || data
return finalResult
}
if (parsed.event === 'error') {
throw new Error(String(data.error || '请求失败'))
}
if (parsed.event === 'cancelled') {
throw new Error('请求已取消')
}
boundary = buffer.indexOf('\n\n')
}
}
return finalResult
}
function readFileAsBase64(file) {
return new Promise((resolve, reject) => {
@@ -31,14 +98,212 @@ export async function convertFileToMarkdown(file) {
}),
})
if (!res.ok) {
const errorText = await res.text()
throw new Error(`HTTP ${res.status}: ${errorText}`)
}
const data = await res.json()
const data = await consumeSseResult(res)
if (!data || typeof data.markdown !== 'string') {
throw new Error('No markdown returned')
}
return data.markdown
}
/**
* Encode AudioBuffer as WAV (16kHz mono, 16-bit PCM) and return base64 string
*/
function audioBufferToWavBase64(audioBuffer) {
// Resample to 16kHz if needed using OfflineAudioContext
const targetSampleRate = 16000
if (audioBuffer.sampleRate === targetSampleRate) {
// No resampling needed, just convert to mono and encode WAV
} else {
const offlineCtx = new OfflineAudioContext(1, audioBuffer.length * (targetSampleRate / audioBuffer.sampleRate), targetSampleRate)
const source = offlineCtx.createBufferSource()
source.buffer = audioBuffer
source.connect(offlineCtx.destination)
// We need to wait for the offline context to finish rendering
}
return new Promise((resolve, reject) => {
const processBuffer = async (buffer) => {
// Convert to mono if stereo/multi-channel
let channels = buffer.numberOfChannels
const length = buffer.length
if (channels === 1) {
// Already mono, use directly
const channelData = buffer.getChannelData(0)
} else {
// Mix down to mono by averaging channels
const channelData = new Float32Array(length)
for (let i = 0; i < length; i++) {
let sum = 0
for (let ch = 0; ch < channels; ch++) {
sum += buffer.getChannelData(ch)[i]
}
channelData[i] = sum / channels
}
}
// Encode as 16-bit PCM WAV (simplified - we'll use the actual channel data)
const sampleRate = buffer.sampleRate
const numSamples = buffer.length
// Get mono data properly
let samples
if (buffer.numberOfChannels === 1) {
samples = buffer.getChannelData(0)
} else {
const monoSamples = new Float32Array(numSamples)
for (let i = 0; i < numSamples; i++) {
let sum = 0
for (let ch = 0; ch < buffer.numberOfChannels; ch++) {
sum += buffer.getChannelData(ch)[i]
}
monoSamples[i] = sum / buffer.numberOfChannels
}
}
// Convert float32 [-1, 1] to int16 PCM
const pcmData = new Int16Array(numSamples)
for (let i = 0; i < numSamples; i++) {
const s = Math.max(-1, Math.min(1, samples[i]))
pcmData[i] = s < 0 ? s * 32768 : s * 32767
}
// Build WAV file (RIFF format)
const wavBuffer = new ArrayBuffer(44 + numSamples * 2)
const view = new DataView(wavBuffer)
// RIFF header
writeString(view, 0, 'RIFF')
view.setUint32(4, 36 + numSamples * 2, true)
writeString(view, 8, 'WAVE')
// fmt chunk
writeString(view, 12, 'fmt ')
view.setUint32(16, 16, true) // chunk size
view.setUint16(20, 1, true) // PCM format
view.setUint16(22, 1, true) // mono channels
view.setUint32(24, sampleRate, true) // sample rate
view.setUint32(28, sampleRate * 2, true) // byte rate
view.setUint16(32, 2, true) // block align
view.setUint16(34, 16, true) // bits per sample
// data chunk
writeString(view, 36, 'data')
view.setUint32(40, numSamples * 2, true)
// Write PCM data
let offset = 44
for (let i = 0; i < numSamples; i++) {
view.setInt16(offset, pcmData[i], true)
offset += 2
}
// Convert to base64
const bytes = new Uint8Array(wavBuffer)
let binary = ''
for (let i = 0; i < bytes.length; i++) {
binary += String.fromCharCode(bytes[i])
}
resolve(btoa(binary))
}
// Handle resampling if needed
const targetSampleRate = 16000
if (audioBuffer.sampleRate === targetSampleRate) {
processBuffer(audioBuffer).catch(reject)
} else {
const offlineCtx = new OfflineAudioContext(1, Math.ceil(audioBuffer.duration * targetSampleRate), targetSampleRate)
const source = offlineCtx.createBufferSource()
source.buffer = audioBuffer
source.connect(offlineCtx.destination)
offlineCtx.oncomplete = (e) => {
processBuffer(e.renderedBuffer).catch(reject)
}
offlineCtx.startRendering()
}
})
}
function writeString(view, offset, string) {
for (let i = 0; i < string.length; i++) {
view.setUint8(offset + i, string.charCodeAt(i))
}
}
/**
* Convert audio file to WAV base64 (16kHz mono, 16-bit PCM)
* Uses Web Audio API to decode and resample if needed.
*/
export async function audioToWavBase64(file) {
return new Promise((resolve, reject) => {
const reader = new FileReader()
reader.onload = async () => {
try {
const arrayBuffer = reader.result
// Decode audio data using Web Audio API
const audioContext = new (window.AudioContext || window.webkitAudioContext)()
// Use a short timeout to avoid hanging on unsupported formats
const decodePromise = audioContext.decodeAudioData(arrayBuffer.slice(0))
// Set a timeout (10 seconds)
const timeoutPromise = new Promise((_, rej) => {
setTimeout(() => rej(new Error('音频解码超时,格式可能不支持')), 10000)
})
const audioBuffer = await Promise.race([decodePromise, timeoutPromise])
// Close the context
audioContext.close()
const wavBase64 = await audioBufferToWavBase64(audioBuffer)
resolve(wavBase64)
} catch (err) {
reject(err)
}
}
reader.onerror = () => reject(reader.error || new Error('Failed to read audio file'))
reader.readAsArrayBuffer(file)
})
}
/**
* Convert audio file to text using ASR endpoint.
* Returns the recognized text string.
*/
export async function convertAudioToText(file, language = 'zh-CN') {
// Step 1: Convert to WAV base64 (handles format conversion)
const wavBase64 = await audioToWavBase64(file)
// Step 2: Send to ASR endpoint
const res = await fetch(ASR_URL, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-API-Key': 'your-secret-key-here',
},
body: JSON.stringify({
audio_base64: wavBase64,
language: language || 'zh-CN',
}),
})
if (res.status === 501) {
throw new Error('ASR 功能不可用,当前环境不支持语音识别')
}
const data = await consumeSseResult(res)
if (!data || typeof data.text !== 'string') {
throw new Error('ASR 返回结果为空')
}
return data.text
}
+24 -1
View File
@@ -17,6 +17,8 @@ function clipDocContext(content = '', limit = 0) {
return `${content.slice(0, limit)}...`
}
const AUDIO_EXT_RE = /\.(wav|mp3|m4a|ogg|flac)$/i
export function normalizeDocType(value = '') {
const lower = String(value || '').trim().toLowerCase()
if (lower === 'txt' || lower === 'text' || lower === 'plain') return 'txt'
@@ -26,6 +28,12 @@ export function normalizeDocType(value = '') {
if (lower === 'doc' || lower === 'docx' || lower === 'word') return 'docx'
if (lower === 'ppt' || lower === 'pptx' || lower === 'powerpoint') return 'pptx'
if (lower === 'pdf') return 'pdf'
// Audio types - map to their extensions for doc block display
if (lower === 'wav' || lower === 'wave') return 'wav'
if (lower === 'mp3' || lower === 'mpeg') return 'mp3'
if (lower === 'm4a' || lower === 'aac') return 'm4a'
if (lower === 'ogg' || lower === 'opus') return 'ogg'
if (lower === 'flac') return 'flac'
return 'txt'
}
@@ -37,9 +45,22 @@ export function getDocTypeFromFilename(name = '') {
if (lower.endsWith('.json')) return 'json'
if (lower.endsWith('.toml')) return 'toml'
if (lower.endsWith('.yaml') || lower.endsWith('.yml')) return 'yaml'
// Audio types - preserve the actual extension for display
if (lower.endsWith('.wav')) return 'wav'
if (lower.endsWith('.mp3') || lower.endsWith('.mpeg')) return 'mp3'
if (lower.endsWith('.m4a') || lower.endsWith('.aac')) return 'm4a'
if (lower.endsWith('.ogg') || lower.endsWith('.opus')) return 'ogg'
if (lower.endsWith('.flac')) return 'flac'
return 'txt'
}
export function isAudioFile(file) {
if (!file) return false
const name = String(file.name || '').toLowerCase()
const type = String(file.type || '').toLowerCase()
return AUDIO_EXT_RE.test(name) || type.startsWith('audio/')
}
export function isSupportedDocFile(file) {
if (!file) return false
const name = String(file.name || '').toLowerCase()
@@ -53,6 +74,7 @@ export function isSupportedDocFile(file) {
name.endsWith('.docx') ||
name.endsWith('.pptx') ||
name.endsWith('.pdf') ||
AUDIO_EXT_RE.test(name) ||
type === 'text/plain' ||
type === 'application/json' ||
type === 'text/yaml' ||
@@ -60,7 +82,8 @@ export function isSupportedDocFile(file) {
type === 'application/x-yaml' ||
type === 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' ||
type === 'application/vnd.openxmlformats-officedocument.presentationml.presentation' ||
type === 'application/pdf'
type === 'application/pdf' ||
type.startsWith('audio/')
)
}
+30
View File
@@ -67,3 +67,33 @@ export function extractTextFromOCR(ocrText, maxLen = 100) {
if (text.toLowerCase() === '(none)') return ''
return text.length > maxLen ? text.substring(0, maxLen) + '...' : text
}
const IMAGE_NODE_TYPES = new Set(['image', 'image-block', 'imageBlock'])
/**
* ProseMirror doc 中提取 OCR 上下文 AI 补全使用
* @param {ProseNode} doc - ProseMirror document node
* @param {number} maxLen - OCR 文本最大长度
* @returns {string}
*/
export function buildOcrContextForDoc(doc, maxLen = 120) {
const lines = []
doc.descendants((node) => {
if (!IMAGE_NODE_TYPES.has(node.type.name)) return true
const src = typeof node.attrs?.src === 'string' ? node.attrs.src : ''
if (!src) return true
const ocrText = getOcrCache(src)
const preview = ocrText ? extractTextFromOCR(ocrText, maxLen) : ''
if (!preview) return true
const label = typeof node.attrs?.alt === 'string' && node.attrs.alt.trim()
? node.attrs.alt.trim()
: 'image'
lines.push(`![${label}](${src}) <OCR:${preview}>`)
return true
})
return lines.join('\n')
}
+36
View File
@@ -0,0 +1,36 @@
const MARKDOWN_FENCE_RE = /^(`{3,}|~{3,})[ \t]*(markdown|md|mdown|text|plain|plaintext)[^\n]*\n([\s\S]*?)\n\1[ \t]*$/i
function normalizeNewlines(value = '') {
return String(value || '').replace(/\r\n?/g, '\n')
}
export function normalizeProAcceptMarkdown(value = '') {
let text = normalizeNewlines(value)
const trimmed = text.trim()
if (!trimmed) return ''
if (trimmed.startsWith('"') && trimmed.endsWith('"')) {
try {
const parsed = JSON.parse(trimmed)
if (typeof parsed === 'string') {
text = normalizeNewlines(parsed)
}
} catch {
// Keep the original response when it is not a JSON string literal.
}
}
const fenceMatch = text.trim().match(MARKDOWN_FENCE_RE)
if (fenceMatch) {
return normalizeNewlines(fenceMatch[3]).trim()
}
return text.trim()
}
export function splitPlainTextFallbackBlocks(value = '') {
const text = normalizeNewlines(value).trim()
if (!text) return []
return text.split(/\n{2,}/).map((block) => block.trim()).filter(Boolean)
}
+18
View File
@@ -0,0 +1,18 @@
这是一份用于测试压缩功能的文档。
人工智能(Artificial Intelligence,简称 AI)是计算机科学的一个分支,它试图理解智能的本质,并生产出一种新的能以人类智能相似的方式做出反应的智能机器。
人工智能的研究包括机器人、语言识别、图像识别、自然语言处理和专家系统等。人工智能从诞生以来,理论和技术日益成熟,应用领域也不断扩大,可以设想未来人工智能带来的科技产品将会是人类智慧的容器。
人工智能可以对人的意识、思维的信息过程的模拟。人工智能不是人的智能,但能像人那样思考、也可能超过人的智能。
机器学习是人工智能的核心领域之一。它使用算法来解析数据、从中学习,然后对真实世界中的事件做出决策和预测。
深度学习是机器学习的一个子集,它使用多层神经网络来分析各种因素。深度学习的出现使得人工智能在许多领域取得了突破性进展,包括计算机视觉、语音识别和自然语言处理。
大型语言模型(LLM)是深度学习在自然语言处理领域的最新成果。它们通过在海量的文本数据上进行训练,学习到了语言的复杂模式和规律。
这些模型能够生成流畅的、符合语法的文本,回答问题,进行翻译,甚至创作诗歌和故事。
然而,人工智能的发展也带来了一些伦理和社会问题,比如隐私保护、算法偏见和就业影响等。
+23
View File
@@ -0,0 +1,23 @@
```llm-file
doc_type: txt
doc_name: test_compress_doc.txt
这是一份用于测试压缩功能的文档。
人工智能(Artificial Intelligence,简称 AI)是计算机科学的一个分支,它试图理解智能的本质,并生产出一种新的能以人类智能相似的方式做出反应的智能机器。
人工智能的研究包括机器人、语言识别、图像识别、自然语言处理和专家系统等。人工智能从诞生以来,理论和技术日益成熟,应用领域也不断扩大,可以设想未来人工智能带来的科技产品将会是人类智慧的容器。
人工智能可以对人的意识、思维的信息过程的模拟。人工智能不是人的智能,但能像人那样思考、也可能超过人的智能。
机器学习是人工智能的核心领域之一。它使用算法来解析数据、从中学习,然后对真实世界中的事件做出决策和预测。
深度学习是机器学习的一个子集,它使用多层神经网络来分析各种因素。深度学习的出现使得人工智能在许多领域取得了突破性进展,包括计算机视觉、语音识别和自然语言处理。
大型语言模型(LLM)是深度学习在自然语言处理领域的最新成果。它们通过在海量的文本数据上进行训练,学习到了语言的复杂模式和规律。
这些模型能够生成流畅的、符合语法的文本,回答问题,进行翻译,甚至创作诗歌和故事。
然而,人工智能的发展也带来了一些伦理和社会问题,比如隐私保护、算法偏见和就业影响等。
```