feat: add Docker support and update backend dependencies
- Introduced `requirements.docker.txt` for Docker-specific dependencies. - Updated `requirements.txt` to include `psycopg[binary]` and `python-multipart`. - Enhanced test suite in `test_main_endpoints.py` to cover document CRUD operations. - Modified `docker-compose.yml` to include PostgreSQL and frontend services. - Added Nginx configuration for reverse proxying API requests. - Refactored file handling in Vue components to support new document storage backend. - Created new utility functions in `docsApi.js` for document management. - Updated configuration to support new API endpoints for document operations. - Adjusted Vite configuration to proxy API requests to the local backend.
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
node_modules
|
||||
dist
|
||||
htmlcov
|
||||
.pytest_cache
|
||||
.git
|
||||
docker-data
|
||||
backend/__pycache__
|
||||
backend/tests/__pycache__
|
||||
@@ -8,6 +8,12 @@ VITE_TTS_STATUS_URL=
|
||||
VITE_TTS_CONFIG_URL=
|
||||
VITE_ASR_URL=
|
||||
VITE_JOB_LOAD_URL=
|
||||
VITE_DOCS_NODES_URL=
|
||||
VITE_DOCS_FOLDERS_URL=
|
||||
VITE_DOCS_TEXT_FILES_URL=
|
||||
VITE_DOCS_UPLOAD_URL=
|
||||
VITE_DOCS_BLOB_BASE_URL=
|
||||
VITE_DOCS_NODES_BASE_URL=
|
||||
VITE_PRO_FRONTEND_TIMEOUT_MS=3660000
|
||||
|
||||
# Document block compression context limit (characters)
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
- 这是一个智能 Markdown 编辑器,前端负责编辑器 UI、上传导出、补全交互和设置状态,后端负责 LLM、OCR、文件转换和 TTS 接口。
|
||||
- 前端技术栈:Vue 3 + Vite + Milkdown/Crepe + Pinia + Vue Router。
|
||||
- 后端技术栈:FastAPI + Python + Ollama。
|
||||
- 后端技术栈:FastAPI + Python + OpenAI-compatible LLM endpoint。
|
||||
|
||||
## 功能块系统(核心概念)
|
||||
|
||||
@@ -73,6 +73,21 @@
|
||||
- pytest backend/tests/test_prompt.py -v
|
||||
- pytest backend/tests/test_llm.py -v
|
||||
|
||||
## Docker 部署约定
|
||||
|
||||
- 本机部署目录固定在 `/Volumes/New Volume/lit/` 下,不在仓库外再散落数据库或 Docker 持久化目录。
|
||||
- 当前推荐的部署工作目录是 `/Volumes/New Volume/lit/llm-in-text/`;把仓库同步到该目录后,从该目录执行 `docker compose up -d --build`。
|
||||
- Docker 持久化数据统一落在部署目录内的 `docker-data/`,包括 PostgreSQL、Redis 和任务共享临时目录。
|
||||
- 容器内访问宿主机模型服务时,不要继续使用 `localhost`;应改成 `host.docker.internal` 之类的容器可达地址。
|
||||
- 当前 Docker 部署默认使用轻量后端依赖集(`backend/requirements.docker.txt`),覆盖补全、OCR、转换、文档空间和队列,不默认包含本地 `torch` / TTS / ASR 模型栈。
|
||||
- 修改 Docker 相关文件时,除了代码本身,还要同步检查:
|
||||
- `docker-compose.yml`
|
||||
- `backend/Dockerfile`
|
||||
- `backend/requirements.docker.txt`
|
||||
- `Dockerfile.frontend`
|
||||
- `docker/nginx.conf`
|
||||
- `backend/.env.example` 与实际部署用 `backend/.env`
|
||||
|
||||
## 代码约定
|
||||
|
||||
- 不要把整个仓库当成“全小写+短横线命名”项目。当前实际情况是:
|
||||
@@ -120,4 +135,3 @@
|
||||
- README.md 对产品功能有参考价值,但其中补全、TTS/ASR 和部分接口说明已经比代码旧。
|
||||
- backend/TTS_ASR_MACOS_FIX.md 和 backend/tests/TESTING_GUIDE.md 更适合作为历史背景,不应在与代码冲突时被当成事实来源。
|
||||
- 修改行为时,优先参考实现代码和对应测试,再决定是否同步普通文档。
|
||||
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
ARG DOCKER_REGISTRY_PREFIX=
|
||||
FROM ${DOCKER_REGISTRY_PREFIX}node:22-alpine AS build
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY package.json package-lock.json ./
|
||||
RUN npm ci
|
||||
|
||||
COPY index.html vite.config.js ./
|
||||
COPY public ./public
|
||||
COPY src ./src
|
||||
|
||||
RUN npm run build
|
||||
|
||||
FROM ${DOCKER_REGISTRY_PREFIX}nginx:1.27-alpine
|
||||
|
||||
COPY docker/nginx.conf /etc/nginx/conf.d/default.conf
|
||||
COPY --from=build /app/dist /usr/share/nginx/html
|
||||
|
||||
EXPOSE 80
|
||||
@@ -68,12 +68,44 @@
|
||||
- 后端: python backend/main.py (端口8001)
|
||||
- 前端: npm run dev (端口5173)
|
||||
|
||||
## Docker 部署
|
||||
|
||||
将整个项目目录放进 New Volume 的 `lit` 文件夹后,在项目根目录执行:
|
||||
|
||||
```bash
|
||||
cp backend/.env.example backend/.env
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
默认对外端口:
|
||||
- 前端: `http://localhost:8080`
|
||||
- 后端: `http://localhost:8001`
|
||||
|
||||
持久化目录全部位于当前项目下的 `docker-data/`:
|
||||
- PostgreSQL: `docker-data/postgres`
|
||||
- Redis: `docker-data/redis`
|
||||
- 任务共享临时目录: `docker-data/jobs`
|
||||
|
||||
部署前至少需要修改这些环境变量:
|
||||
- `backend/.env` 中的 `LLM_BASE_URL`
|
||||
- `backend/.env` 中的 `LLM_API_KEY`
|
||||
- `backend/.env` 或 shell 环境中的 `DATABASE_URL`
|
||||
- `backend/.env` 中的 `API_KEY`
|
||||
|
||||
## API接口
|
||||
|
||||
- POST /v1/completions 流式补全建议
|
||||
- POST /v1/ocr 图片文字识别
|
||||
- POST /v1/convert 文档转换
|
||||
- POST /v1/completions/cancel 取消请求
|
||||
- GET /v1/docs/nodes 文档空间节点列表
|
||||
- POST /v1/docs/folders 创建文件夹
|
||||
- POST /v1/docs/files/text 创建文本文件
|
||||
- POST /v1/docs/files/upload 上传文件到文档空间
|
||||
- PATCH /v1/docs/nodes/{id} 更新节点
|
||||
- PUT /v1/docs/files/{id}/blob 替换文件二进制内容
|
||||
- DELETE /v1/docs/nodes/{id} 删除节点
|
||||
- GET /v1/docs/files/{id}/blob 下载或预览原文件
|
||||
- GET /v1/tts-asr/status TTS/ASR模型状态
|
||||
- GET /v1/tts-asr/config TTS/ASR配置信息
|
||||
- POST /v1/tts-asr/warmup 模型预热
|
||||
|
||||
+10
-9
@@ -1,16 +1,15 @@
|
||||
# LLM provider (OpenAI-compatible endpoint)
|
||||
LLM_BASE_URL=http://localhost:11434/v1/
|
||||
# For Ollama, API key is not required but a placeholder is needed.
|
||||
LLM_API_KEY=ollama
|
||||
# OpenAI-compatible endpoint
|
||||
LLM_BASE_URL=https://api.openai.com/v1/
|
||||
LLM_API_KEY=sk-your-key
|
||||
|
||||
# Default model for inline completions (e.g., gpt-oss:20b, qwen3:8b)
|
||||
LLM_MODEL=gpt-oss:20b
|
||||
# Default model for inline completions
|
||||
LLM_MODEL=gpt-4.1-mini
|
||||
|
||||
# Pro-tier model (defaults to LLM_MODEL if unset)
|
||||
PRO_LLM_MODEL=gpt-oss:20b
|
||||
PRO_LLM_MODEL=gpt-4.1
|
||||
|
||||
# Vision model for OCR (e.g., qwen3-vl:30b, llava)
|
||||
VLM_MODEL=qwen3-vl:30b
|
||||
# Vision model for OCR
|
||||
VLM_MODEL=gpt-4.1-mini
|
||||
|
||||
# API key for the FastAPI app (change in production)
|
||||
API_KEY=your-secret-key-here
|
||||
@@ -18,6 +17,8 @@ API_KEY=your-secret-key-here
|
||||
# Job backend
|
||||
JOB_BACKEND=redis
|
||||
REDIS_URL=redis://localhost:6379/0
|
||||
DATABASE_URL=postgresql://llm_in_text:llm_in_text_change_me@localhost:5432/llm_in_text
|
||||
DOCS_BACKEND=postgres
|
||||
JOB_REDIS_PREFIX=llmtext:jobs
|
||||
JOB_CONSUMER_NAME=
|
||||
JOB_SHARED_TEMP_DIR=/tmp/llm-in-text-jobs
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
ARG DOCKER_REGISTRY_PREFIX=
|
||||
FROM ${DOCKER_REGISTRY_PREFIX}python:3.11-slim
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
WORKDIR /app/backend
|
||||
|
||||
COPY backend/requirements.docker.txt /tmp/requirements.docker.txt
|
||||
RUN pip install --no-cache-dir -r /tmp/requirements.docker.txt
|
||||
|
||||
COPY backend /app/backend
|
||||
|
||||
EXPOSE 8001
|
||||
|
||||
CMD ["python", "main.py"]
|
||||
@@ -0,0 +1,668 @@
|
||||
import mimetypes
|
||||
import os
|
||||
import threading
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
try: # pragma: no cover - optional in some test paths
|
||||
import psycopg
|
||||
from psycopg.rows import dict_row
|
||||
except Exception: # pragma: no cover
|
||||
psycopg = None
|
||||
dict_row = None
|
||||
|
||||
|
||||
MAX_TEXT_SIZE = 8 * 1024 * 1024
|
||||
PREVIEW_TEXT_SIZE = 2 * 1024 * 1024
|
||||
|
||||
TEXT_EXTENSIONS = {
|
||||
"md", "markdown", "txt", "json", "js", "jsx", "ts", "tsx",
|
||||
"css", "scss", "less", "html", "htm", "py", "vue", "xml",
|
||||
"yaml", "yml", "csv", "log", "sql", "toml", "ini", "cfg",
|
||||
"conf", "sh", "bat", "ps1", "java", "c", "cpp", "h", "hpp",
|
||||
"go", "rs", "swift", "kt", "rb", "php", "pl", "r", "scala",
|
||||
"gradle", "properties", "env", "gitignore", "dockerfile",
|
||||
}
|
||||
|
||||
BINARY_EXTENSIONS = {
|
||||
"exe", "dll", "so", "dylib", "bin", "dat", "obj", "o", "a",
|
||||
"doc", "docx", "xls", "xlsx", "ppt", "pptx", "odt", "ods", "odp",
|
||||
"pdf", "zip", "rar", "7z", "tar", "gz", "bz2", "xz",
|
||||
"png", "jpg", "jpeg", "gif", "bmp", "ico", "webp", "svg",
|
||||
"mp3", "mp4", "wav", "avi", "mov", "mkv", "flv", "wmv",
|
||||
"ttf", "otf", "woff", "woff2", "eot",
|
||||
"class", "pyc", "pyo", "jar", "war", "ear",
|
||||
"db", "sqlite", "mdb", "accdb",
|
||||
"pem", "key", "crt", "cer", "p12", "pfx", "jks",
|
||||
"msg", "eml", "pst", "ost",
|
||||
"dwg", "dxf", "step", "stl", "fbx", "3ds", "blend",
|
||||
}
|
||||
|
||||
DEFAULT_MIME_TYPES = {
|
||||
"avi": "video/x-msvideo",
|
||||
"md": "text/markdown",
|
||||
"markdown": "text/markdown",
|
||||
"mkv": "video/x-matroska",
|
||||
"mov": "video/quicktime",
|
||||
"mp4": "video/mp4",
|
||||
"txt": "text/plain",
|
||||
"json": "application/json",
|
||||
"js": "text/javascript",
|
||||
"jsx": "text/javascript",
|
||||
"ts": "text/typescript",
|
||||
"tsx": "text/typescript",
|
||||
"css": "text/css",
|
||||
"html": "text/html",
|
||||
"htm": "text/html",
|
||||
"py": "text/x-python",
|
||||
"vue": "text/plain",
|
||||
"xml": "application/xml",
|
||||
"yaml": "text/yaml",
|
||||
"yml": "text/yaml",
|
||||
"csv": "text/csv",
|
||||
"log": "text/plain",
|
||||
"sql": "text/plain",
|
||||
"toml": "text/plain",
|
||||
"ini": "text/plain",
|
||||
"cfg": "text/plain",
|
||||
"conf": "text/plain",
|
||||
"sh": "text/plain",
|
||||
"bat": "text/plain",
|
||||
"ps1": "text/plain",
|
||||
"jpg": "image/jpeg",
|
||||
"jpeg": "image/jpeg",
|
||||
"flv": "video/x-flv",
|
||||
"m4v": "video/x-m4v",
|
||||
"png": "image/png",
|
||||
"gif": "image/gif",
|
||||
"webp": "image/webp",
|
||||
"svg": "image/svg+xml",
|
||||
"pdf": "application/pdf",
|
||||
"ogg": "video/ogg",
|
||||
"ogv": "video/ogg",
|
||||
"webm": "video/webm",
|
||||
"wmv": "video/x-ms-wmv",
|
||||
}
|
||||
|
||||
|
||||
def _now_sql() -> str:
|
||||
return "CURRENT_TIMESTAMP"
|
||||
|
||||
|
||||
def get_extension(name: str = "") -> str:
|
||||
value = str(name or "")
|
||||
if "." not in value:
|
||||
return value.lower() if value.lower() == "dockerfile" else ""
|
||||
return value.rsplit(".", 1)[-1].lower()
|
||||
|
||||
|
||||
def infer_mime_type(name: str, fallback: str = "") -> str:
|
||||
if fallback:
|
||||
return fallback
|
||||
ext = get_extension(name)
|
||||
guessed = DEFAULT_MIME_TYPES.get(ext)
|
||||
if guessed:
|
||||
return guessed
|
||||
guessed, _ = mimetypes.guess_type(name)
|
||||
return guessed or "application/octet-stream"
|
||||
|
||||
|
||||
def is_text_file(name: str, mime_type: str = "") -> bool:
|
||||
ext = get_extension(name)
|
||||
if ext in BINARY_EXTENSIONS:
|
||||
return False
|
||||
mime_value = str(mime_type or "").lower()
|
||||
return ext in TEXT_EXTENSIONS or mime_value.startswith("text/") or "json" in mime_value or "xml" in mime_value
|
||||
|
||||
|
||||
def _text_payload(text: str) -> dict[str, Any]:
|
||||
encoded = text.encode("utf-8")
|
||||
preview = encoded[:PREVIEW_TEXT_SIZE].decode("utf-8", errors="ignore")
|
||||
return {
|
||||
"storage_kind": "text",
|
||||
"size": len(encoded),
|
||||
"content_text": text,
|
||||
"preview_text": preview if len(encoded) > PREVIEW_TEXT_SIZE else text,
|
||||
"is_truncated_preview": len(encoded) > PREVIEW_TEXT_SIZE,
|
||||
"blob_data": None,
|
||||
}
|
||||
|
||||
|
||||
def prepare_file_payload(name: str, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
resolved_mime = infer_mime_type(name, mime_type)
|
||||
if is_text_file(name, resolved_mime):
|
||||
text = raw_bytes.decode("utf-8", errors="ignore")
|
||||
return {
|
||||
"mime_type": resolved_mime,
|
||||
**_text_payload(text),
|
||||
}
|
||||
return {
|
||||
"mime_type": resolved_mime,
|
||||
"storage_kind": "blob",
|
||||
"size": len(raw_bytes),
|
||||
"content_text": "",
|
||||
"preview_text": "",
|
||||
"is_truncated_preview": False,
|
||||
"blob_data": raw_bytes,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class BlobPayload:
|
||||
content: bytes
|
||||
mime_type: str
|
||||
filename: str
|
||||
|
||||
|
||||
class BaseDocumentStore:
|
||||
_UNSET = object()
|
||||
|
||||
def list_nodes(self) -> list[dict[str, Any]]:
|
||||
raise NotImplementedError
|
||||
|
||||
def create_folder(self, name: str, parent_id: str | None) -> dict[str, Any]:
|
||||
raise NotImplementedError
|
||||
|
||||
def create_text_file(self, name: str, parent_id: str | None, content: str = "") -> dict[str, Any]:
|
||||
raise NotImplementedError
|
||||
|
||||
def upload_file(self, name: str, parent_id: str | None, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
raise NotImplementedError
|
||||
|
||||
def update_node(self, node_id: str, *, name: str | None | object = _UNSET, parent_id: str | None | object = _UNSET, content: str | None | object = _UNSET) -> dict[str, Any]:
|
||||
raise NotImplementedError
|
||||
|
||||
def replace_blob(self, node_id: str, filename: str, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
raise NotImplementedError
|
||||
|
||||
def delete_node(self, node_id: str) -> None:
|
||||
raise NotImplementedError
|
||||
|
||||
def get_blob(self, node_id: str) -> BlobPayload:
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class InMemoryDocumentStore(BaseDocumentStore):
|
||||
def __init__(self) -> None:
|
||||
self.nodes: dict[str, dict[str, Any]] = {}
|
||||
|
||||
def _serialize(self, node: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"id": node["id"],
|
||||
"parentId": node["parent_id"],
|
||||
"type": node["type"],
|
||||
"name": node["name"],
|
||||
"mimeType": node.get("mime_type") or "",
|
||||
"storageKind": node.get("storage_kind") or "text",
|
||||
"size": int(node.get("size") or 0),
|
||||
"content": node.get("content_text") or "",
|
||||
"previewText": node.get("preview_text") or "",
|
||||
"isTruncatedPreview": bool(node.get("is_truncated_preview")),
|
||||
"createdAt": int(node["created_at"]),
|
||||
"updatedAt": int(node["updated_at"]),
|
||||
}
|
||||
|
||||
def _timestamp(self) -> int:
|
||||
import time
|
||||
return int(time.time() * 1000)
|
||||
|
||||
def _get(self, node_id: str) -> dict[str, Any]:
|
||||
node = self.nodes.get(node_id)
|
||||
if node is None:
|
||||
raise KeyError(node_id)
|
||||
return node
|
||||
|
||||
def list_nodes(self) -> list[dict[str, Any]]:
|
||||
ordered = sorted(
|
||||
self.nodes.values(),
|
||||
key=lambda item: (item["type"] != "folder", item["name"].lower(), item["created_at"]),
|
||||
)
|
||||
return [self._serialize(node) for node in ordered]
|
||||
|
||||
def create_folder(self, name: str, parent_id: str | None) -> dict[str, Any]:
|
||||
now = self._timestamp()
|
||||
node = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"parent_id": parent_id,
|
||||
"type": "folder",
|
||||
"name": name,
|
||||
"mime_type": "",
|
||||
"storage_kind": "text",
|
||||
"size": 0,
|
||||
"content_text": "",
|
||||
"preview_text": "",
|
||||
"is_truncated_preview": False,
|
||||
"blob_data": None,
|
||||
"created_at": now,
|
||||
"updated_at": now,
|
||||
}
|
||||
self.nodes[node["id"]] = node
|
||||
return self._serialize(node)
|
||||
|
||||
def create_text_file(self, name: str, parent_id: str | None, content: str = "") -> dict[str, Any]:
|
||||
now = self._timestamp()
|
||||
payload = _text_payload(content)
|
||||
node = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"parent_id": parent_id,
|
||||
"type": "file",
|
||||
"name": name,
|
||||
"mime_type": infer_mime_type(name),
|
||||
**payload,
|
||||
"created_at": now,
|
||||
"updated_at": now,
|
||||
}
|
||||
self.nodes[node["id"]] = node
|
||||
return self._serialize(node)
|
||||
|
||||
def upload_file(self, name: str, parent_id: str | None, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
now = self._timestamp()
|
||||
payload = prepare_file_payload(name, raw_bytes, mime_type)
|
||||
node = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"parent_id": parent_id,
|
||||
"type": "file",
|
||||
"name": name,
|
||||
**payload,
|
||||
"created_at": now,
|
||||
"updated_at": now,
|
||||
}
|
||||
self.nodes[node["id"]] = node
|
||||
return self._serialize(node)
|
||||
|
||||
def update_node(self, node_id: str, *, name: str | None | object = BaseDocumentStore._UNSET, parent_id: str | None | object = BaseDocumentStore._UNSET, content: str | None | object = BaseDocumentStore._UNSET) -> dict[str, Any]:
|
||||
node = self._get(node_id)
|
||||
if name is not BaseDocumentStore._UNSET:
|
||||
node["name"] = name
|
||||
if node["type"] == "file":
|
||||
node["mime_type"] = infer_mime_type(name, node.get("mime_type") or "")
|
||||
if parent_id is not BaseDocumentStore._UNSET:
|
||||
node["parent_id"] = parent_id
|
||||
if content is not BaseDocumentStore._UNSET:
|
||||
payload = _text_payload(content)
|
||||
node.update(payload)
|
||||
node["updated_at"] = self._timestamp()
|
||||
return self._serialize(node)
|
||||
|
||||
def replace_blob(self, node_id: str, filename: str, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
node = self._get(node_id)
|
||||
payload = prepare_file_payload(filename, raw_bytes, mime_type)
|
||||
node.update(payload)
|
||||
node["name"] = filename
|
||||
node["mime_type"] = payload["mime_type"]
|
||||
node["updated_at"] = self._timestamp()
|
||||
return self._serialize(node)
|
||||
|
||||
def delete_node(self, node_id: str) -> None:
|
||||
descendants = {node_id}
|
||||
changed = True
|
||||
while changed:
|
||||
changed = False
|
||||
for node in list(self.nodes.values()):
|
||||
if node.get("parent_id") in descendants and node["id"] not in descendants:
|
||||
descendants.add(node["id"])
|
||||
changed = True
|
||||
for current_id in descendants:
|
||||
self.nodes.pop(current_id, None)
|
||||
|
||||
def get_blob(self, node_id: str) -> BlobPayload:
|
||||
node = self._get(node_id)
|
||||
if node["type"] != "file":
|
||||
raise FileNotFoundError(node_id)
|
||||
if node.get("blob_data") is not None:
|
||||
content = node["blob_data"]
|
||||
else:
|
||||
content = (node.get("content_text") or "").encode("utf-8")
|
||||
return BlobPayload(
|
||||
content=content,
|
||||
mime_type=node.get("mime_type") or infer_mime_type(node.get("name") or ""),
|
||||
filename=node["name"],
|
||||
)
|
||||
|
||||
|
||||
class PostgresDocumentStore(BaseDocumentStore):
|
||||
def __init__(self, database_url: str) -> None:
|
||||
if psycopg is None or dict_row is None:
|
||||
raise RuntimeError("psycopg 未安装,无法使用 PostgreSQL 文档存储")
|
||||
self.database_url = database_url
|
||||
self._init_lock = threading.Lock()
|
||||
self._initialized = False
|
||||
|
||||
def _connect(self):
|
||||
return psycopg.connect(self.database_url, autocommit=True, row_factory=dict_row)
|
||||
|
||||
def _ensure_initialized(self) -> None:
|
||||
if self._initialized:
|
||||
return
|
||||
with self._init_lock:
|
||||
if self._initialized:
|
||||
return
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS document_nodes (
|
||||
id TEXT PRIMARY KEY,
|
||||
parent_id TEXT REFERENCES document_nodes(id) ON DELETE CASCADE,
|
||||
type TEXT NOT NULL CHECK (type IN ('folder', 'file')),
|
||||
name TEXT NOT NULL,
|
||||
mime_type TEXT NOT NULL DEFAULT '',
|
||||
storage_kind TEXT NOT NULL DEFAULT 'text',
|
||||
size BIGINT NOT NULL DEFAULT 0,
|
||||
content_text TEXT NOT NULL DEFAULT '',
|
||||
preview_text TEXT NOT NULL DEFAULT '',
|
||||
is_truncated_preview BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
blob_data BYTEA,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
)
|
||||
"""
|
||||
)
|
||||
cur.execute(
|
||||
"CREATE INDEX IF NOT EXISTS document_nodes_parent_idx ON document_nodes(parent_id)"
|
||||
)
|
||||
self._initialized = True
|
||||
|
||||
def _serialize(self, row: dict[str, Any]) -> dict[str, Any]:
|
||||
created = row["created_at"]
|
||||
updated = row["updated_at"]
|
||||
return {
|
||||
"id": row["id"],
|
||||
"parentId": row["parent_id"],
|
||||
"type": row["type"],
|
||||
"name": row["name"],
|
||||
"mimeType": row.get("mime_type") or "",
|
||||
"storageKind": row.get("storage_kind") or "text",
|
||||
"size": int(row.get("size") or 0),
|
||||
"content": row.get("content_text") or "",
|
||||
"previewText": row.get("preview_text") or "",
|
||||
"isTruncatedPreview": bool(row.get("is_truncated_preview")),
|
||||
"createdAt": int(created.timestamp() * 1000),
|
||||
"updatedAt": int(updated.timestamp() * 1000),
|
||||
}
|
||||
|
||||
def _fetch_node(self, node_id: str) -> dict[str, Any]:
|
||||
self._ensure_initialized()
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("SELECT * FROM document_nodes WHERE id = %s", (node_id,))
|
||||
row = cur.fetchone()
|
||||
if row is None:
|
||||
raise KeyError(node_id)
|
||||
return row
|
||||
|
||||
def list_nodes(self) -> list[dict[str, Any]]:
|
||||
self._ensure_initialized()
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT
|
||||
id,
|
||||
parent_id,
|
||||
type,
|
||||
name,
|
||||
mime_type,
|
||||
storage_kind,
|
||||
size,
|
||||
CASE
|
||||
WHEN storage_kind = 'text' AND size <= %s THEN content_text
|
||||
ELSE ''
|
||||
END AS content_text,
|
||||
preview_text,
|
||||
is_truncated_preview,
|
||||
created_at,
|
||||
updated_at
|
||||
FROM document_nodes
|
||||
ORDER BY
|
||||
CASE WHEN type = 'folder' THEN 0 ELSE 1 END,
|
||||
LOWER(name),
|
||||
created_at
|
||||
""",
|
||||
(MAX_TEXT_SIZE,),
|
||||
)
|
||||
return [self._serialize(row) for row in cur.fetchall()]
|
||||
|
||||
def create_folder(self, name: str, parent_id: str | None) -> dict[str, Any]:
|
||||
self._ensure_initialized()
|
||||
node_id = str(uuid.uuid4())
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO document_nodes (id, parent_id, type, name, updated_at)
|
||||
VALUES (%s, %s, 'folder', %s, CURRENT_TIMESTAMP)
|
||||
RETURNING *
|
||||
""",
|
||||
(node_id, parent_id, name),
|
||||
)
|
||||
return self._serialize(cur.fetchone())
|
||||
|
||||
def create_text_file(self, name: str, parent_id: str | None, content: str = "") -> dict[str, Any]:
|
||||
self._ensure_initialized()
|
||||
node_id = str(uuid.uuid4())
|
||||
payload = _text_payload(content)
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO document_nodes (
|
||||
id, parent_id, type, name, mime_type, storage_kind, size,
|
||||
content_text, preview_text, is_truncated_preview, blob_data, updated_at
|
||||
)
|
||||
VALUES (%s, %s, 'file', %s, %s, %s, %s, %s, %s, %s, %s, CURRENT_TIMESTAMP)
|
||||
RETURNING *
|
||||
""",
|
||||
(
|
||||
node_id,
|
||||
parent_id,
|
||||
name,
|
||||
infer_mime_type(name),
|
||||
payload["storage_kind"],
|
||||
payload["size"],
|
||||
payload["content_text"],
|
||||
payload["preview_text"],
|
||||
payload["is_truncated_preview"],
|
||||
None,
|
||||
),
|
||||
)
|
||||
return self._serialize(cur.fetchone())
|
||||
|
||||
def upload_file(self, name: str, parent_id: str | None, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
self._ensure_initialized()
|
||||
node_id = str(uuid.uuid4())
|
||||
payload = prepare_file_payload(name, raw_bytes, mime_type)
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO document_nodes (
|
||||
id, parent_id, type, name, mime_type, storage_kind, size,
|
||||
content_text, preview_text, is_truncated_preview, blob_data, updated_at
|
||||
)
|
||||
VALUES (%s, %s, 'file', %s, %s, %s, %s, %s, %s, %s, %s, CURRENT_TIMESTAMP)
|
||||
RETURNING *
|
||||
""",
|
||||
(
|
||||
node_id,
|
||||
parent_id,
|
||||
name,
|
||||
payload["mime_type"],
|
||||
payload["storage_kind"],
|
||||
payload["size"],
|
||||
payload["content_text"],
|
||||
payload["preview_text"],
|
||||
payload["is_truncated_preview"],
|
||||
payload["blob_data"],
|
||||
),
|
||||
)
|
||||
return self._serialize(cur.fetchone())
|
||||
|
||||
def update_node(self, node_id: str, *, name: str | None | object = BaseDocumentStore._UNSET, parent_id: str | None | object = BaseDocumentStore._UNSET, content: str | None | object = BaseDocumentStore._UNSET) -> dict[str, Any]:
|
||||
self._ensure_initialized()
|
||||
row = self._fetch_node(node_id)
|
||||
next_name = row["name"] if name is BaseDocumentStore._UNSET else name
|
||||
next_parent_id = row["parent_id"] if parent_id is BaseDocumentStore._UNSET else parent_id
|
||||
next_mime_type = row.get("mime_type") or ""
|
||||
next_storage_kind = row.get("storage_kind") or "text"
|
||||
next_size = row.get("size") or 0
|
||||
next_content_text = row.get("content_text") or ""
|
||||
next_preview_text = row.get("preview_text") or ""
|
||||
next_is_truncated = bool(row.get("is_truncated_preview"))
|
||||
next_blob_data = row.get("blob_data")
|
||||
|
||||
if row["type"] == "file" and name is not BaseDocumentStore._UNSET:
|
||||
next_mime_type = infer_mime_type(next_name, next_mime_type)
|
||||
if content is not BaseDocumentStore._UNSET:
|
||||
payload = _text_payload(content)
|
||||
next_storage_kind = payload["storage_kind"]
|
||||
next_size = payload["size"]
|
||||
next_content_text = payload["content_text"]
|
||||
next_preview_text = payload["preview_text"]
|
||||
next_is_truncated = payload["is_truncated_preview"]
|
||||
next_blob_data = None
|
||||
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
UPDATE document_nodes
|
||||
SET
|
||||
name = %s,
|
||||
parent_id = %s,
|
||||
mime_type = %s,
|
||||
storage_kind = %s,
|
||||
size = %s,
|
||||
content_text = %s,
|
||||
preview_text = %s,
|
||||
is_truncated_preview = %s,
|
||||
blob_data = %s,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = %s
|
||||
RETURNING *
|
||||
""",
|
||||
(
|
||||
next_name,
|
||||
next_parent_id,
|
||||
next_mime_type,
|
||||
next_storage_kind,
|
||||
next_size,
|
||||
next_content_text,
|
||||
next_preview_text,
|
||||
next_is_truncated,
|
||||
next_blob_data,
|
||||
node_id,
|
||||
),
|
||||
)
|
||||
updated = cur.fetchone()
|
||||
if updated is None:
|
||||
raise KeyError(node_id)
|
||||
return self._serialize(updated)
|
||||
|
||||
def replace_blob(self, node_id: str, filename: str, raw_bytes: bytes, mime_type: str = "") -> dict[str, Any]:
|
||||
self._ensure_initialized()
|
||||
payload = prepare_file_payload(filename, raw_bytes, mime_type)
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
UPDATE document_nodes
|
||||
SET
|
||||
name = %s,
|
||||
mime_type = %s,
|
||||
storage_kind = %s,
|
||||
size = %s,
|
||||
content_text = %s,
|
||||
preview_text = %s,
|
||||
is_truncated_preview = %s,
|
||||
blob_data = %s,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = %s AND type = 'file'
|
||||
RETURNING *
|
||||
""",
|
||||
(
|
||||
filename,
|
||||
payload["mime_type"],
|
||||
payload["storage_kind"],
|
||||
payload["size"],
|
||||
payload["content_text"],
|
||||
payload["preview_text"],
|
||||
payload["is_truncated_preview"],
|
||||
payload["blob_data"],
|
||||
node_id,
|
||||
),
|
||||
)
|
||||
updated = cur.fetchone()
|
||||
if updated is None:
|
||||
raise KeyError(node_id)
|
||||
return self._serialize(updated)
|
||||
|
||||
def delete_node(self, node_id: str) -> None:
|
||||
self._ensure_initialized()
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
WITH RECURSIVE descendants AS (
|
||||
SELECT id FROM document_nodes WHERE id = %s
|
||||
UNION ALL
|
||||
SELECT child.id
|
||||
FROM document_nodes child
|
||||
INNER JOIN descendants parent ON child.parent_id = parent.id
|
||||
)
|
||||
DELETE FROM document_nodes
|
||||
WHERE id IN (SELECT id FROM descendants)
|
||||
""",
|
||||
(node_id,),
|
||||
)
|
||||
|
||||
def get_blob(self, node_id: str) -> BlobPayload:
|
||||
self._ensure_initialized()
|
||||
with self._connect() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT id, name, mime_type, content_text, blob_data
|
||||
FROM document_nodes
|
||||
WHERE id = %s AND type = 'file'
|
||||
""",
|
||||
(node_id,),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
if row is None:
|
||||
raise FileNotFoundError(node_id)
|
||||
blob_data = row.get("blob_data")
|
||||
if blob_data is None:
|
||||
content = (row.get("content_text") or "").encode("utf-8")
|
||||
else:
|
||||
content = bytes(blob_data)
|
||||
return BlobPayload(
|
||||
content=content,
|
||||
mime_type=row.get("mime_type") or infer_mime_type(row.get("name") or ""),
|
||||
filename=row["name"],
|
||||
)
|
||||
|
||||
|
||||
_document_store: BaseDocumentStore | None = None
|
||||
|
||||
|
||||
def get_document_store() -> BaseDocumentStore:
|
||||
global _document_store
|
||||
if _document_store is not None:
|
||||
return _document_store
|
||||
backend = (os.getenv("DOCS_BACKEND") or "postgres").strip().lower()
|
||||
if backend == "memory":
|
||||
_document_store = InMemoryDocumentStore()
|
||||
return _document_store
|
||||
|
||||
database_url = os.getenv("DATABASE_URL", "").strip()
|
||||
if not database_url:
|
||||
raise RuntimeError("缺少 DATABASE_URL,无法初始化 PostgreSQL 文档存储")
|
||||
_document_store = PostgresDocumentStore(database_url)
|
||||
return _document_store
|
||||
|
||||
|
||||
def reset_document_store() -> None:
|
||||
global _document_store
|
||||
_document_store = None
|
||||
+147
-1
@@ -1,3 +1,4 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
@@ -5,12 +6,13 @@ import os
|
||||
import uuid
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import FastAPI, HTTPException, Request, Security
|
||||
from fastapi import FastAPI, File, Form, HTTPException, Request, Response, Security, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse, StreamingResponse
|
||||
from fastapi.security import APIKeyHeader
|
||||
from pydantic import BaseModel
|
||||
|
||||
from docs_store import get_document_store
|
||||
from geoip import get_ip_location_text
|
||||
from job_handlers import (
|
||||
_sanitize_converted_markdown,
|
||||
@@ -110,6 +112,23 @@ class ASRJobRequest(BaseModel):
|
||||
language: Optional[str] = "zh-CN"
|
||||
|
||||
|
||||
class CreateFolderRequest(BaseModel):
|
||||
name: str
|
||||
parentId: Optional[str] = None
|
||||
|
||||
|
||||
class CreateTextFileRequest(BaseModel):
|
||||
name: str
|
||||
parentId: Optional[str] = None
|
||||
content: str = ""
|
||||
|
||||
|
||||
class UpdateNodeRequest(BaseModel):
|
||||
name: Optional[str] = None
|
||||
parentId: Optional[str] = None
|
||||
content: Optional[str] = None
|
||||
|
||||
|
||||
def _preview(text: str, limit: int = 80) -> str:
|
||||
value = (text or "").replace("\n", "\\n")
|
||||
if len(value) <= limit:
|
||||
@@ -219,6 +238,12 @@ async def _queue_load_snapshot() -> dict:
|
||||
return {}
|
||||
|
||||
|
||||
async def _docs_store_call(method_name: str, *args, **kwargs):
|
||||
store = get_document_store()
|
||||
method = getattr(store, method_name)
|
||||
return await asyncio.to_thread(method, *args, **kwargs)
|
||||
|
||||
|
||||
@app.post("/v1/completions")
|
||||
async def create_completion(
|
||||
request: Request,
|
||||
@@ -452,6 +477,127 @@ async def get_job_load(api_key: str = Security(get_api_key)):
|
||||
return {"queues": await _queue_load_snapshot()}
|
||||
|
||||
|
||||
@app.get("/v1/docs/nodes")
|
||||
async def list_docs_nodes(api_key: str = Security(get_api_key)):
|
||||
del api_key
|
||||
try:
|
||||
return {"nodes": await _docs_store_call("list_nodes")}
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/v1/docs/folders")
|
||||
async def create_docs_folder(req: CreateFolderRequest, api_key: str = Security(get_api_key)):
|
||||
del api_key
|
||||
if not (req.name or "").strip():
|
||||
raise HTTPException(status_code=400, detail="文件夹名称不能为空")
|
||||
try:
|
||||
node = await _docs_store_call("create_folder", req.name.strip(), req.parentId)
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return {"node": node}
|
||||
|
||||
|
||||
@app.post("/v1/docs/files/text")
|
||||
async def create_docs_text_file(req: CreateTextFileRequest, api_key: str = Security(get_api_key)):
|
||||
del api_key
|
||||
if not (req.name or "").strip():
|
||||
raise HTTPException(status_code=400, detail="文件名称不能为空")
|
||||
try:
|
||||
node = await _docs_store_call("create_text_file", req.name.strip(), req.parentId, req.content or "")
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return {"node": node}
|
||||
|
||||
|
||||
@app.post("/v1/docs/files/upload")
|
||||
async def upload_docs_file(
|
||||
file: UploadFile = File(...),
|
||||
parent_id: Optional[str] = Form(default=None),
|
||||
api_key: str = Security(get_api_key),
|
||||
):
|
||||
del api_key
|
||||
filename = (file.filename or "").strip()
|
||||
if not filename:
|
||||
raise HTTPException(status_code=400, detail="文件名称不能为空")
|
||||
raw_bytes = await file.read()
|
||||
try:
|
||||
node = await _docs_store_call("upload_file", filename, parent_id, raw_bytes, file.content_type or "")
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return {"node": node}
|
||||
|
||||
|
||||
@app.patch("/v1/docs/nodes/{node_id}")
|
||||
async def update_docs_node(node_id: str, req: UpdateNodeRequest, api_key: str = Security(get_api_key)):
|
||||
del api_key
|
||||
fields_set = req.model_fields_set if hasattr(req, "model_fields_set") else getattr(req, "__fields_set__", set())
|
||||
if not fields_set:
|
||||
raise HTTPException(status_code=400, detail="缺少更新内容")
|
||||
update_kwargs = {}
|
||||
if "name" in fields_set:
|
||||
next_name = req.name.strip() if isinstance(req.name, str) else ""
|
||||
if not next_name:
|
||||
raise HTTPException(status_code=400, detail="名称不能为空")
|
||||
update_kwargs["name"] = next_name
|
||||
if "parentId" in fields_set:
|
||||
update_kwargs["parent_id"] = req.parentId
|
||||
if "content" in fields_set:
|
||||
update_kwargs["content"] = req.content or ""
|
||||
try:
|
||||
node = await _docs_store_call("update_node", node_id, **update_kwargs)
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="节点不存在") from exc
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return {"node": node}
|
||||
|
||||
|
||||
@app.put("/v1/docs/files/{node_id}/blob")
|
||||
async def replace_docs_blob(
|
||||
node_id: str,
|
||||
file: UploadFile = File(...),
|
||||
api_key: str = Security(get_api_key),
|
||||
):
|
||||
del api_key
|
||||
filename = (file.filename or "").strip()
|
||||
if not filename:
|
||||
raise HTTPException(status_code=400, detail="文件名称不能为空")
|
||||
raw_bytes = await file.read()
|
||||
try:
|
||||
node = await _docs_store_call("replace_blob", node_id, filename, raw_bytes, file.content_type or "")
|
||||
except KeyError as exc:
|
||||
raise HTTPException(status_code=404, detail="节点不存在") from exc
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return {"node": node}
|
||||
|
||||
|
||||
@app.delete("/v1/docs/nodes/{node_id}")
|
||||
async def delete_docs_node(node_id: str, api_key: str = Security(get_api_key)):
|
||||
del api_key
|
||||
try:
|
||||
await _docs_store_call("delete_node", node_id)
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@app.get("/v1/docs/files/{node_id}/blob")
|
||||
async def download_docs_blob(node_id: str, api_key: str = Security(get_api_key)):
|
||||
del api_key
|
||||
try:
|
||||
payload = await _docs_store_call("get_blob", node_id)
|
||||
except FileNotFoundError as exc:
|
||||
raise HTTPException(status_code=404, detail="文件不存在") from exc
|
||||
except RuntimeError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
headers = {
|
||||
"Content-Disposition": f'inline; filename="{payload.filename}"',
|
||||
}
|
||||
return Response(content=payload.content, media_type=payload.mime_type, headers=headers)
|
||||
|
||||
|
||||
def _register_tts_asr_routes():
|
||||
try:
|
||||
from tts_asr import register_tts_asr_routes
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
fastapi>=0.95.0
|
||||
uvicorn[standard]>=0.23.0
|
||||
pydantic>=1.10.0
|
||||
httpx>=0.24.0
|
||||
redis>=5.0.0
|
||||
psycopg[binary]>=3.2.0
|
||||
python-multipart>=0.0.9
|
||||
python-dotenv>=1.0.0
|
||||
markitdown>=0.1.1
|
||||
geoip2>=4.8.0
|
||||
@@ -3,6 +3,8 @@ uvicorn[standard]>=0.23.0
|
||||
pydantic>=1.10.0
|
||||
httpx>=0.24.0
|
||||
redis>=5.0.0
|
||||
psycopg[binary]>=3.2.0
|
||||
python-multipart>=0.0.9
|
||||
python-dotenv>=1.0.0
|
||||
|
||||
numpy>=1.23.0
|
||||
|
||||
@@ -7,6 +7,7 @@ from pathlib import Path
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
os.environ["JOB_BACKEND"] = "memory"
|
||||
os.environ["DOCS_BACKEND"] = "memory"
|
||||
|
||||
CURRENT_DIR = Path(__file__).resolve().parent
|
||||
BACKEND_DIR = CURRENT_DIR.parent
|
||||
@@ -15,6 +16,7 @@ if str(BACKEND_DIR) not in sys.path:
|
||||
|
||||
import job_handlers # type: ignore
|
||||
import job_system # type: ignore
|
||||
import docs_store # type: ignore
|
||||
|
||||
main = importlib.import_module("main")
|
||||
|
||||
@@ -23,6 +25,7 @@ HEADERS = {"X-API-Key": main.API_KEY}
|
||||
|
||||
def setup_function():
|
||||
job_system.reset_job_manager()
|
||||
docs_store.reset_document_store()
|
||||
main._handlers_registered = False
|
||||
|
||||
|
||||
@@ -119,3 +122,69 @@ def test_post_convert_unsupported_extension_returns_500():
|
||||
})
|
||||
assert resp.status_code == 500
|
||||
assert "仅支持" in resp.json()["error"]
|
||||
|
||||
|
||||
def test_docs_nodes_crud_round_trip():
|
||||
with TestClient(main.app) as client:
|
||||
folder_resp = client.post("/v1/docs/folders", headers=HEADERS, json={
|
||||
"name": "项目资料",
|
||||
"parentId": None,
|
||||
})
|
||||
assert folder_resp.status_code == 200
|
||||
folder = folder_resp.json()["node"]
|
||||
|
||||
file_resp = client.post("/v1/docs/files/text", headers=HEADERS, json={
|
||||
"name": "notes.md",
|
||||
"parentId": folder["id"],
|
||||
"content": "# hello",
|
||||
})
|
||||
assert file_resp.status_code == 200
|
||||
file_node = file_resp.json()["node"]
|
||||
assert file_node["previewText"] == "# hello"
|
||||
|
||||
list_resp = client.get("/v1/docs/nodes", headers=HEADERS)
|
||||
assert list_resp.status_code == 200
|
||||
nodes = list_resp.json()["nodes"]
|
||||
assert len(nodes) == 2
|
||||
|
||||
rename_resp = client.patch(f"/v1/docs/nodes/{file_node['id']}", headers=HEADERS, json={
|
||||
"name": "renamed.md",
|
||||
})
|
||||
assert rename_resp.status_code == 200
|
||||
assert rename_resp.json()["node"]["name"] == "renamed.md"
|
||||
|
||||
blob_resp = client.get(f"/v1/docs/files/{file_node['id']}/blob", headers=HEADERS)
|
||||
assert blob_resp.status_code == 200
|
||||
assert blob_resp.content == b"# hello"
|
||||
|
||||
delete_resp = client.delete(f"/v1/docs/nodes/{folder['id']}", headers=HEADERS)
|
||||
assert delete_resp.status_code == 200
|
||||
|
||||
final_list = client.get("/v1/docs/nodes", headers=HEADERS)
|
||||
assert final_list.status_code == 200
|
||||
assert final_list.json()["nodes"] == []
|
||||
|
||||
|
||||
def test_docs_file_upload_and_blob_replace():
|
||||
with TestClient(main.app) as client:
|
||||
upload_resp = client.post(
|
||||
"/v1/docs/files/upload",
|
||||
headers=HEADERS,
|
||||
files={"file": ("image.png", b"png-bytes", "image/png")},
|
||||
data={"parent_id": ""},
|
||||
)
|
||||
assert upload_resp.status_code == 200
|
||||
node = upload_resp.json()["node"]
|
||||
assert node["storageKind"] == "blob"
|
||||
|
||||
replace_resp = client.put(
|
||||
f"/v1/docs/files/{node['id']}/blob",
|
||||
headers=HEADERS,
|
||||
files={"file": ("photo.jpg", b"jpeg-bytes", "image/jpeg")},
|
||||
)
|
||||
assert replace_resp.status_code == 200
|
||||
assert replace_resp.json()["node"]["name"] == "photo.jpg"
|
||||
|
||||
blob_resp = client.get(f"/v1/docs/files/{node['id']}/blob", headers=HEADERS)
|
||||
assert blob_resp.status_code == 200
|
||||
assert blob_resp.content == b"jpeg-bytes"
|
||||
|
||||
+43
-24
@@ -1,48 +1,67 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
frontend:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.frontend
|
||||
args:
|
||||
DOCKER_REGISTRY_PREFIX: ${DOCKER_REGISTRY_PREFIX:-}
|
||||
depends_on:
|
||||
- api
|
||||
ports:
|
||||
- "6379:6379"
|
||||
- "8080:80"
|
||||
|
||||
postgres:
|
||||
image: ${DOCKER_REGISTRY_PREFIX:-}postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-llm_in_text}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-llm_in_text}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-llm_in_text_change_me}
|
||||
volumes:
|
||||
- ./docker-data/postgres:/var/lib/postgresql/data
|
||||
|
||||
redis:
|
||||
image: ${DOCKER_REGISTRY_PREFIX:-}redis:7-alpine
|
||||
command: ["redis-server", "--appendonly", "yes"]
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
- ./docker-data/redis:/data
|
||||
|
||||
api:
|
||||
image: python:3.11-slim
|
||||
working_dir: /app
|
||||
command: sh -c "pip install -r backend/requirements.txt && python backend/main.py"
|
||||
build:
|
||||
context: .
|
||||
dockerfile: backend/Dockerfile
|
||||
args:
|
||||
DOCKER_REGISTRY_PREFIX: ${DOCKER_REGISTRY_PREFIX:-}
|
||||
env_file:
|
||||
- backend/.env
|
||||
environment:
|
||||
JOB_BACKEND: redis
|
||||
REDIS_URL: redis://redis:6379/0
|
||||
DATABASE_URL: ${DATABASE_URL:-postgresql://llm_in_text:llm_in_text_change_me@postgres:5432/llm_in_text}
|
||||
DOCS_BACKEND: postgres
|
||||
JOB_SHARED_TEMP_DIR: /shared-jobs
|
||||
volumes:
|
||||
- .:/app
|
||||
- job-shared:/shared-jobs
|
||||
depends_on:
|
||||
- postgres
|
||||
- redis
|
||||
ports:
|
||||
- "8001:8001"
|
||||
volumes:
|
||||
- ./docker-data/jobs:/shared-jobs
|
||||
|
||||
worker:
|
||||
image: python:3.11-slim
|
||||
working_dir: /app
|
||||
command: sh -c "pip install -r backend/requirements.txt && python backend/worker.py"
|
||||
build:
|
||||
context: .
|
||||
dockerfile: backend/Dockerfile
|
||||
args:
|
||||
DOCKER_REGISTRY_PREFIX: ${DOCKER_REGISTRY_PREFIX:-}
|
||||
command: ["python", "worker.py"]
|
||||
env_file:
|
||||
- backend/.env
|
||||
environment:
|
||||
JOB_BACKEND: redis
|
||||
REDIS_URL: redis://redis:6379/0
|
||||
DATABASE_URL: ${DATABASE_URL:-postgresql://llm_in_text:llm_in_text_change_me@postgres:5432/llm_in_text}
|
||||
DOCS_BACKEND: postgres
|
||||
JOB_SHARED_TEMP_DIR: /shared-jobs
|
||||
volumes:
|
||||
- .:/app
|
||||
- job-shared:/shared-jobs
|
||||
depends_on:
|
||||
- postgres
|
||||
- redis
|
||||
|
||||
volumes:
|
||||
redis-data:
|
||||
job-shared:
|
||||
volumes:
|
||||
- ./docker-data/jobs:/shared-jobs
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
location /v1/ {
|
||||
proxy_pass http://api:8001/v1/;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_buffering off;
|
||||
}
|
||||
}
|
||||
@@ -35,6 +35,8 @@ const imageEditorError = ref('')
|
||||
const isEditingImage = ref(false)
|
||||
const isSavingImage = ref(false)
|
||||
const videoPreviewError = ref('')
|
||||
const resolvedFileBlob = ref(null)
|
||||
let blobRequestToken = 0
|
||||
|
||||
const isRoot = computed(() => !props.node)
|
||||
const isFolder = computed(() => props.node?.type === 'folder')
|
||||
@@ -74,7 +76,7 @@ const folderItems = computed(() => {
|
||||
if (!isRoot.value && !isFolder.value) return []
|
||||
return isRoot.value ? props.rootNodes : props.node.children || []
|
||||
})
|
||||
const fileBlob = computed(() => props.getFileBlob(props.node))
|
||||
const fileBlob = computed(() => resolvedFileBlob.value)
|
||||
const previewText = computed(() => props.node?.content || props.node?.previewText || '')
|
||||
const lineCount = computed(() => {
|
||||
if (!previewText.value) return 0
|
||||
@@ -105,8 +107,8 @@ function assignBasePreviewUrl(url = '') {
|
||||
}
|
||||
|
||||
watch(
|
||||
[fileBlob, () => props.node?.id],
|
||||
([blob]) => {
|
||||
() => [props.node?.id, resolvedFileBlob.value],
|
||||
([, blob]) => {
|
||||
clearBasePreview()
|
||||
videoPreviewError.value = ''
|
||||
|
||||
@@ -117,6 +119,25 @@ watch(
|
||||
{ immediate: true }
|
||||
)
|
||||
|
||||
watch(
|
||||
() => props.node,
|
||||
async (node) => {
|
||||
const requestToken = ++blobRequestToken
|
||||
resolvedFileBlob.value = null
|
||||
if (!node || node.type !== 'file') return
|
||||
|
||||
try {
|
||||
const result = await props.getFileBlob(node)
|
||||
if (requestToken !== blobRequestToken) return
|
||||
resolvedFileBlob.value = result instanceof Blob ? result : null
|
||||
} catch {
|
||||
if (requestToken !== blobRequestToken) return
|
||||
resolvedFileBlob.value = null
|
||||
}
|
||||
},
|
||||
{ immediate: true }
|
||||
)
|
||||
|
||||
watch(
|
||||
() => props.node?.id,
|
||||
() => {
|
||||
@@ -127,6 +148,7 @@ watch(
|
||||
)
|
||||
|
||||
onBeforeUnmount(() => {
|
||||
blobRequestToken += 1
|
||||
clearBasePreview()
|
||||
})
|
||||
|
||||
@@ -253,8 +275,8 @@ function downloadFile() {
|
||||
<div class="directory-card">
|
||||
<div class="directory-card-header">
|
||||
<div>
|
||||
<h3>{{ isRoot ? '本地文件空间' : node.name }}</h3>
|
||||
<p>{{ isRoot ? '所有文件都保存在当前浏览器本地,不会上传到服务器。' : '像 GitHub 一样浏览当前目录内容。' }}</p>
|
||||
<h3>{{ isRoot ? '文档空间' : node.name }}</h3>
|
||||
<p>{{ isRoot ? '所有文件都保存在当前服务端文档空间中。' : '像 GitHub 一样浏览当前目录内容。' }}</p>
|
||||
</div>
|
||||
<div class="directory-header-meta">
|
||||
<span>{{ folderItems.length }} 项</span>
|
||||
@@ -387,7 +409,7 @@ function downloadFile() {
|
||||
<div v-if="videoPreviewError" class="video-state-card video-state-error">
|
||||
<h3>视频预览暂时不可用</h3>
|
||||
<p>{{ videoPreviewError }}</p>
|
||||
<p>原始文件仍保存在浏览器本地,你可以继续下载后使用本地播放器打开。</p>
|
||||
<p>原始文件仍保存在服务端文档空间中,你可以继续下载后使用本地播放器打开。</p>
|
||||
</div>
|
||||
|
||||
<video
|
||||
@@ -417,7 +439,7 @@ function downloadFile() {
|
||||
<div v-else class="content-unsupported">
|
||||
<div class="unsupported-card">
|
||||
<h3>暂不支持在线预览此文件</h3>
|
||||
<p>文件已保存在浏览器本地,你仍然可以点击右上角“下载”获取原文件。</p>
|
||||
<p>文件已保存在服务端文档空间中,你仍然可以点击右上角“下载”获取原文件。</p>
|
||||
<div class="unsupported-meta">
|
||||
<span>{{ node.name }}</span>
|
||||
<span>{{ fileSizeLabel }}</span>
|
||||
|
||||
+176
-349
@@ -1,66 +1,18 @@
|
||||
import { computed, ref } from 'vue'
|
||||
import {
|
||||
createDocFolder,
|
||||
createDocTextFile,
|
||||
deleteDocNode,
|
||||
fetchDocBlob,
|
||||
fetchDocNodes,
|
||||
replaceDocBlob,
|
||||
updateDocNode,
|
||||
uploadDocFile,
|
||||
} from '../utils/docsApi'
|
||||
|
||||
const DB_NAME = 'llm-in-text-docs'
|
||||
const DB_VERSION = 1
|
||||
const STORE_NAME = 'nodes'
|
||||
const MAX_FILE_SIZE = 1024 * 1024 * 1024
|
||||
const MAX_TEXT_SIZE = 8 * 1024 * 1024
|
||||
const PREVIEW_TEXT_SIZE = 2 * 1024 * 1024
|
||||
const MAX_NODES = 5000
|
||||
|
||||
let dbPromise = null
|
||||
|
||||
function generateId() {
|
||||
return `${Date.now().toString(36)}${Math.random().toString(36).slice(2, 10)}`
|
||||
}
|
||||
|
||||
function openDatabase() {
|
||||
if (dbPromise) return dbPromise
|
||||
dbPromise = new Promise((resolve, reject) => {
|
||||
const request = indexedDB.open(DB_NAME, DB_VERSION)
|
||||
request.onupgradeneeded = () => {
|
||||
const db = request.result
|
||||
if (!db.objectStoreNames.contains(STORE_NAME)) {
|
||||
db.createObjectStore(STORE_NAME, { keyPath: 'id' })
|
||||
}
|
||||
}
|
||||
request.onsuccess = () => resolve(request.result)
|
||||
request.onerror = () => reject(request.error || new Error('打开本地数据库失败'))
|
||||
})
|
||||
return dbPromise
|
||||
}
|
||||
|
||||
async function withStore(mode, handler) {
|
||||
const db = await openDatabase()
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction(STORE_NAME, mode)
|
||||
const store = transaction.objectStore(STORE_NAME)
|
||||
let request
|
||||
try {
|
||||
request = handler(store)
|
||||
} catch (error) {
|
||||
reject(error)
|
||||
return
|
||||
}
|
||||
if (request && typeof request.onsuccess === 'function') {
|
||||
request.onsuccess = () => resolve(request.result)
|
||||
request.onerror = () => reject(request.error || new Error('本地数据库操作失败'))
|
||||
} else {
|
||||
transaction.oncomplete = () => resolve(request)
|
||||
transaction.onerror = () => reject(transaction.error || new Error('本地数据库操作失败'))
|
||||
}
|
||||
transaction.onabort = () => reject(transaction.error || new Error('本地数据库操作已取消'))
|
||||
})
|
||||
}
|
||||
|
||||
function cloneRecord(record) {
|
||||
if (!record) return record
|
||||
return {
|
||||
...record,
|
||||
children: undefined
|
||||
}
|
||||
}
|
||||
|
||||
function getExtension(name = '') {
|
||||
const parts = String(name).split('.')
|
||||
return parts.length > 1 ? parts.pop().toLowerCase() : ''
|
||||
@@ -147,7 +99,6 @@ function inferMimeType(name, fallback = '') {
|
||||
|
||||
function isTextFile(record) {
|
||||
const ext = getExtension(record?.name)
|
||||
// 二进制文件不提供预览
|
||||
if (isBinaryExtension(ext)) return false
|
||||
const mime = String(record?.mimeType || '')
|
||||
return isTextExtension(ext) || mime.startsWith('text/') || mime.includes('json') || mime.includes('xml')
|
||||
@@ -169,11 +120,8 @@ function buildTree(records) {
|
||||
continue
|
||||
}
|
||||
const parent = map.get(record.parentId)
|
||||
if (parent?.type === 'folder') {
|
||||
parent.children.push(current)
|
||||
} else {
|
||||
roots.push(current)
|
||||
}
|
||||
if (parent?.type === 'folder') parent.children.push(current)
|
||||
else roots.push(current)
|
||||
}
|
||||
const sorter = (a, b) => {
|
||||
if (a.type !== b.type) return a.type === 'folder' ? -1 : 1
|
||||
@@ -182,9 +130,7 @@ function buildTree(records) {
|
||||
const sortChildren = (nodes) => {
|
||||
nodes.sort(sorter)
|
||||
for (const node of nodes) {
|
||||
if (node.type === 'folder' && Array.isArray(node.children)) {
|
||||
sortChildren(node.children)
|
||||
}
|
||||
if (node.type === 'folder' && Array.isArray(node.children)) sortChildren(node.children)
|
||||
}
|
||||
}
|
||||
sortChildren(roots)
|
||||
@@ -220,94 +166,15 @@ function estimateRecordSize(record) {
|
||||
return 0
|
||||
}
|
||||
|
||||
async function readFilePayload(file) {
|
||||
const mimeType = inferMimeType(file.name, file.type)
|
||||
const ext = getExtension(file.name)
|
||||
const textFile = isTextExtension(ext) || mimeType.startsWith('text/') || mimeType.includes('json') || mimeType.includes('xml')
|
||||
|
||||
if (!textFile) {
|
||||
return {
|
||||
mimeType,
|
||||
size: file.size,
|
||||
storageKind: 'blob',
|
||||
blob: file
|
||||
}
|
||||
}
|
||||
|
||||
// 二进制扩展名文件不尝试读取内容,避免长时间等待
|
||||
if (isBinaryExtension(ext)) {
|
||||
return {
|
||||
mimeType,
|
||||
size: file.size,
|
||||
storageKind: 'blob',
|
||||
blob: file
|
||||
}
|
||||
}
|
||||
|
||||
if (file.size <= MAX_TEXT_SIZE) {
|
||||
const content = await file.text()
|
||||
return {
|
||||
mimeType,
|
||||
size: file.size,
|
||||
storageKind: 'text',
|
||||
content,
|
||||
previewText: content,
|
||||
isTruncatedPreview: false
|
||||
}
|
||||
}
|
||||
const previewText = await file.slice(0, PREVIEW_TEXT_SIZE).text()
|
||||
return {
|
||||
mimeType,
|
||||
size: file.size,
|
||||
storageKind: 'blob',
|
||||
blob: file,
|
||||
previewText,
|
||||
isTruncatedPreview: true
|
||||
}
|
||||
function cloneNode(node) {
|
||||
return JSON.parse(JSON.stringify(node))
|
||||
}
|
||||
|
||||
function createWelcomeRecords() {
|
||||
const folderId = generateId()
|
||||
const fileId = generateId()
|
||||
const now = Date.now()
|
||||
return [
|
||||
{
|
||||
id: folderId,
|
||||
name: '示例文件夹',
|
||||
type: 'folder',
|
||||
parentId: null,
|
||||
createdAt: now,
|
||||
updatedAt: now
|
||||
},
|
||||
{
|
||||
id: fileId,
|
||||
name: '欢迎使用.md',
|
||||
type: 'file',
|
||||
parentId: null,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
mimeType: 'text/markdown',
|
||||
storageKind: 'text',
|
||||
size: 258,
|
||||
content: [
|
||||
'# 欢迎使用文档模式',
|
||||
'',
|
||||
'这里已经切换为更接近 GitHub 的文件浏览体验。',
|
||||
'',
|
||||
'## 现在支持',
|
||||
'',
|
||||
'- 左侧文件树与快速上传',
|
||||
'- 浏览器本地持久化存储',
|
||||
'- 文本、Markdown、图片、PDF 预览',
|
||||
'- 大文件保留原始文件并显示截断预览'
|
||||
].join('\n'),
|
||||
previewText: '',
|
||||
isTruncatedPreview: false
|
||||
}
|
||||
]
|
||||
}
|
||||
let singleton = null
|
||||
|
||||
export function useFileSystem() {
|
||||
if (singleton) return singleton
|
||||
|
||||
const records = ref([])
|
||||
const selectedId = ref(null)
|
||||
const expandedIds = ref(new Set())
|
||||
@@ -315,6 +182,7 @@ export function useFileSystem() {
|
||||
const contextMenu = ref(null)
|
||||
const error = ref(null)
|
||||
const loading = ref(false)
|
||||
const blobCache = new Map()
|
||||
|
||||
const tree = computed(() => buildTree(records.value))
|
||||
const stats = computed(() => {
|
||||
@@ -329,161 +197,123 @@ export function useFileSystem() {
|
||||
return { fileCount, folderCount, usedBytes }
|
||||
})
|
||||
|
||||
function clearBlobCache(id = null) {
|
||||
if (id) {
|
||||
blobCache.delete(id)
|
||||
return
|
||||
}
|
||||
blobCache.clear()
|
||||
}
|
||||
|
||||
function upsertRecord(nextRecord) {
|
||||
const index = records.value.findIndex((item) => item.id === nextRecord.id)
|
||||
if (index === -1) records.value = [...records.value, nextRecord]
|
||||
else {
|
||||
const next = [...records.value]
|
||||
next[index] = nextRecord
|
||||
records.value = next
|
||||
}
|
||||
return nextRecord
|
||||
}
|
||||
|
||||
async function load() {
|
||||
loading.value = true
|
||||
try {
|
||||
const nextRecords = await withStore('readonly', (store) => store.getAll())
|
||||
if (!Array.isArray(nextRecords) || nextRecords.length === 0) {
|
||||
const seed = createWelcomeRecords()
|
||||
await Promise.all(seed.map((record) => persistRecord(record)))
|
||||
records.value = seed
|
||||
} else {
|
||||
records.value = nextRecords
|
||||
}
|
||||
records.value = await fetchDocNodes()
|
||||
error.value = null
|
||||
} catch {
|
||||
error.value = '读取本地文件失败,请刷新页面后重试'
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '读取文档空间失败,请稍后重试'
|
||||
records.value = []
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
async function persistRecord(record) {
|
||||
return withStore('readwrite', (store) => store.put(cloneRecord(record)))
|
||||
}
|
||||
|
||||
async function deleteRecord(id) {
|
||||
return withStore('readwrite', (store) => store.delete(id))
|
||||
}
|
||||
|
||||
function touchParent(parentId) {
|
||||
if (!parentId) return
|
||||
const parent = records.value.find((item) => item.id === parentId)
|
||||
if (!parent) return
|
||||
parent.updatedAt = Date.now()
|
||||
persistRecord(parent).catch(() => {
|
||||
error.value = '更新目录时间失败'
|
||||
})
|
||||
}
|
||||
|
||||
function createFile(parentId, name, content = '', options = {}) {
|
||||
async function createFile(parentId, name, content = '') {
|
||||
if (records.value.length >= MAX_NODES) {
|
||||
error.value = `文件数量不能超过 ${MAX_NODES} 个`
|
||||
return false
|
||||
}
|
||||
const size = typeof options.size === 'number' ? options.size : new Blob([content]).size
|
||||
if (size > MAX_FILE_SIZE) {
|
||||
error.value = '单个文件不能超过 1GB'
|
||||
try {
|
||||
const node = await createDocTextFile(name, parentId || null, content)
|
||||
upsertRecord(node)
|
||||
if (parentId) {
|
||||
const next = new Set(expandedIds.value)
|
||||
next.add(parentId)
|
||||
expandedIds.value = next
|
||||
}
|
||||
selectedId.value = node.id
|
||||
error.value = null
|
||||
return true
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '创建文件失败'
|
||||
return false
|
||||
}
|
||||
const now = Date.now()
|
||||
const file = {
|
||||
id: generateId(),
|
||||
name,
|
||||
type: 'file',
|
||||
parentId: parentId || null,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
mimeType: inferMimeType(name, options.mimeType),
|
||||
storageKind: options.storageKind || 'text',
|
||||
size,
|
||||
content: options.content ?? content,
|
||||
previewText: options.previewText ?? '',
|
||||
isTruncatedPreview: Boolean(options.isTruncatedPreview),
|
||||
blob: options.blob || null
|
||||
}
|
||||
records.value = [...records.value, file]
|
||||
if (parentId) {
|
||||
const next = new Set(expandedIds.value)
|
||||
next.add(parentId)
|
||||
expandedIds.value = next
|
||||
}
|
||||
selectedId.value = file.id
|
||||
error.value = null
|
||||
persistRecord(file).catch(() => {
|
||||
error.value = '保存文件失败,可能是浏览器存储空间不足'
|
||||
})
|
||||
touchParent(parentId)
|
||||
return true
|
||||
}
|
||||
|
||||
function updateFile(id, nextValue, options = {}) {
|
||||
async function updateFile(id, nextValue, options = {}) {
|
||||
const file = records.value.find((item) => item.id === id && item.type === 'file')
|
||||
if (!file) return false
|
||||
|
||||
const nextName = options.name || file.name
|
||||
const isBlobValue = nextValue instanceof Blob
|
||||
const nextContent = isBlobValue ? (options.content ?? '') : String(options.content ?? nextValue ?? '')
|
||||
const nextSize = typeof options.size === 'number'
|
||||
? options.size
|
||||
: isBlobValue
|
||||
? nextValue.size
|
||||
: new Blob([nextContent]).size
|
||||
|
||||
if (nextSize > MAX_FILE_SIZE) {
|
||||
error.value = '单个文件不能超过 1GB'
|
||||
try {
|
||||
let node
|
||||
if (nextValue instanceof Blob) {
|
||||
const filename = options.name || file.name
|
||||
const upload = nextValue instanceof File
|
||||
? nextValue
|
||||
: new File([nextValue], filename, { type: options.mimeType || nextValue.type || file.mimeType || '' })
|
||||
node = await replaceDocBlob(id, upload)
|
||||
} else {
|
||||
const content = String(options.content ?? nextValue ?? '')
|
||||
node = await updateDocNode(id, {
|
||||
name: options.name || file.name,
|
||||
content,
|
||||
})
|
||||
}
|
||||
upsertRecord(node)
|
||||
clearBlobCache(id)
|
||||
error.value = null
|
||||
return true
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '保存文件失败'
|
||||
return false
|
||||
}
|
||||
|
||||
file.name = nextName
|
||||
file.updatedAt = Date.now()
|
||||
file.mimeType = inferMimeType(nextName, options.mimeType || (isBlobValue ? nextValue.type : file.mimeType))
|
||||
file.size = nextSize
|
||||
file.storageKind = options.storageKind || (isBlobValue ? 'blob' : 'text')
|
||||
file.content = nextContent
|
||||
file.previewText = options.previewText ?? (file.storageKind === 'text' ? nextContent : '')
|
||||
file.isTruncatedPreview = Boolean(options.isTruncatedPreview)
|
||||
file.blob = isBlobValue ? nextValue : null
|
||||
records.value = [...records.value]
|
||||
error.value = null
|
||||
|
||||
persistRecord(file).catch(() => {
|
||||
error.value = '保存文件失败,可能是浏览器存储空间不足'
|
||||
})
|
||||
touchParent(file.parentId)
|
||||
return true
|
||||
}
|
||||
|
||||
function createFolder(parentId, name) {
|
||||
async function createFolder(parentId, name) {
|
||||
if (records.value.length >= MAX_NODES) {
|
||||
error.value = `目录项数量不能超过 ${MAX_NODES} 个`
|
||||
return false
|
||||
}
|
||||
const now = Date.now()
|
||||
const folder = {
|
||||
id: generateId(),
|
||||
name,
|
||||
type: 'folder',
|
||||
parentId: parentId || null,
|
||||
createdAt: now,
|
||||
updatedAt: now
|
||||
try {
|
||||
const node = await createDocFolder(name, parentId || null)
|
||||
upsertRecord(node)
|
||||
if (parentId) {
|
||||
const next = new Set(expandedIds.value)
|
||||
next.add(parentId)
|
||||
expandedIds.value = next
|
||||
}
|
||||
selectedId.value = node.id
|
||||
error.value = null
|
||||
return true
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '创建文件夹失败'
|
||||
return false
|
||||
}
|
||||
records.value = [...records.value, folder]
|
||||
if (parentId) {
|
||||
const next = new Set(expandedIds.value)
|
||||
next.add(parentId)
|
||||
expandedIds.value = next
|
||||
}
|
||||
selectedId.value = folder.id
|
||||
error.value = null
|
||||
persistRecord(folder).catch(() => {
|
||||
error.value = '保存文件夹失败'
|
||||
})
|
||||
touchParent(parentId)
|
||||
return true
|
||||
}
|
||||
|
||||
function rename(id, newName) {
|
||||
async function rename(id, newName) {
|
||||
const node = records.value.find((item) => item.id === id)
|
||||
if (!node) return false
|
||||
node.name = newName
|
||||
node.updatedAt = Date.now()
|
||||
error.value = null
|
||||
persistRecord(node).catch(() => {
|
||||
error.value = '重命名失败'
|
||||
})
|
||||
return true
|
||||
try {
|
||||
const updated = await updateDocNode(id, { name: newName })
|
||||
upsertRecord(updated)
|
||||
clearBlobCache(id)
|
||||
error.value = null
|
||||
return true
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '重命名失败'
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function collectDescendantIds(id) {
|
||||
@@ -501,23 +331,21 @@ export function useFileSystem() {
|
||||
return [...ids]
|
||||
}
|
||||
|
||||
function remove(id) {
|
||||
async function remove(id) {
|
||||
const ids = new Set(collectDescendantIds(id))
|
||||
const deletingSelected = selectedId.value && ids.has(selectedId.value)
|
||||
records.value = records.value.filter((item) => !ids.has(item.id))
|
||||
if (deletingSelected) selectedId.value = null
|
||||
if (clipboard.value?.nodeId && ids.has(clipboard.value.nodeId)) {
|
||||
clipboard.value = null
|
||||
try {
|
||||
await deleteDocNode(id)
|
||||
records.value = records.value.filter((item) => !ids.has(item.id))
|
||||
if (selectedId.value && ids.has(selectedId.value)) selectedId.value = null
|
||||
if (clipboard.value?.nodeId && ids.has(clipboard.value.nodeId)) clipboard.value = null
|
||||
if (clipboard.value?.node?.id && ids.has(clipboard.value.node.id)) clipboard.value = null
|
||||
ids.forEach((currentId) => clearBlobCache(currentId))
|
||||
error.value = null
|
||||
return true
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '删除文件失败'
|
||||
return false
|
||||
}
|
||||
if (clipboard.value?.node?.id && ids.has(clipboard.value.node.id)) {
|
||||
clipboard.value = null
|
||||
}
|
||||
error.value = null
|
||||
ids.forEach((currentId) => {
|
||||
deleteRecord(currentId).catch(() => {
|
||||
error.value = '删除文件失败'
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
function select(id) {
|
||||
@@ -536,19 +364,13 @@ export function useFileSystem() {
|
||||
function copy(id) {
|
||||
const node = findNode(tree.value, id)
|
||||
if (!node) return
|
||||
clipboard.value = {
|
||||
mode: 'copy',
|
||||
node: JSON.parse(JSON.stringify(node))
|
||||
}
|
||||
clipboard.value = { mode: 'copy', node: cloneNode(node) }
|
||||
}
|
||||
|
||||
function cut(id) {
|
||||
const node = records.value.find((item) => item.id === id)
|
||||
if (!node) return
|
||||
clipboard.value = {
|
||||
mode: 'cut',
|
||||
nodeId: node.id
|
||||
}
|
||||
clipboard.value = { mode: 'cut', nodeId: node.id }
|
||||
}
|
||||
|
||||
function isDescendantOf(sourceId, targetParentId) {
|
||||
@@ -560,58 +382,58 @@ export function useFileSystem() {
|
||||
return false
|
||||
}
|
||||
|
||||
function duplicateNode(node, targetParentId) {
|
||||
const now = Date.now()
|
||||
const clonedId = generateId()
|
||||
const record = {
|
||||
...cloneRecord(node),
|
||||
id: clonedId,
|
||||
parentId: targetParentId,
|
||||
createdAt: now,
|
||||
updatedAt: now
|
||||
}
|
||||
records.value = [...records.value, record]
|
||||
persistRecord(record).catch(() => {
|
||||
error.value = '复制文件失败'
|
||||
})
|
||||
async function duplicateNode(node, targetParentId) {
|
||||
let created
|
||||
if (node.type === 'folder') {
|
||||
created = await createDocFolder(node.name, targetParentId)
|
||||
upsertRecord(created)
|
||||
for (const child of node.children || []) {
|
||||
duplicateNode(child, clonedId)
|
||||
await duplicateNode(child, created.id)
|
||||
}
|
||||
return created
|
||||
}
|
||||
|
||||
if (node.storageKind === 'blob') {
|
||||
const blob = await getFileBlob(node)
|
||||
const upload = new File([blob], node.name, { type: node.mimeType || blob.type || inferMimeType(node.name) })
|
||||
created = await uploadDocFile(upload, targetParentId)
|
||||
} else {
|
||||
created = await createDocTextFile(node.name, targetParentId, node.content || node.previewText || '')
|
||||
}
|
||||
upsertRecord(created)
|
||||
return created
|
||||
}
|
||||
|
||||
function paste(targetParentId) {
|
||||
async function paste(targetParentId) {
|
||||
if (!clipboard.value) return
|
||||
if (clipboard.value.mode === 'cut') {
|
||||
const node = records.value.find((item) => item.id === clipboard.value.nodeId)
|
||||
if (!node) {
|
||||
try {
|
||||
if (clipboard.value.mode === 'cut') {
|
||||
const node = records.value.find((item) => item.id === clipboard.value.nodeId)
|
||||
if (!node) {
|
||||
clipboard.value = null
|
||||
return
|
||||
}
|
||||
if (node.id === targetParentId || (targetParentId && isDescendantOf(node.id, targetParentId))) {
|
||||
error.value = '不能移动到自身或子目录中'
|
||||
return
|
||||
}
|
||||
const updated = await updateDocNode(node.id, { parentId: targetParentId || null })
|
||||
upsertRecord(updated)
|
||||
clipboard.value = null
|
||||
error.value = null
|
||||
return
|
||||
}
|
||||
if (node.id === targetParentId || (targetParentId && isDescendantOf(node.id, targetParentId))) {
|
||||
error.value = '不能移动到自身或子目录中'
|
||||
const source = clipboard.value.node
|
||||
if (!source) return
|
||||
if (records.value.length >= MAX_NODES) {
|
||||
error.value = `目录项数量不能超过 ${MAX_NODES} 个`
|
||||
return
|
||||
}
|
||||
node.parentId = targetParentId || null
|
||||
node.updatedAt = Date.now()
|
||||
persistRecord(node).catch(() => {
|
||||
error.value = '移动文件失败'
|
||||
})
|
||||
touchParent(targetParentId)
|
||||
clipboard.value = null
|
||||
await duplicateNode(source, targetParentId || null)
|
||||
error.value = null
|
||||
return
|
||||
} catch (err) {
|
||||
error.value = err instanceof Error && err.message ? err.message : '粘贴失败'
|
||||
}
|
||||
const source = clipboard.value.node
|
||||
if (!source) return
|
||||
if (records.value.length >= MAX_NODES) {
|
||||
error.value = `目录项数量不能超过 ${MAX_NODES} 个`
|
||||
return
|
||||
}
|
||||
duplicateNode(source, targetParentId || null)
|
||||
touchParent(targetParentId)
|
||||
error.value = null
|
||||
}
|
||||
|
||||
function canPaste() {
|
||||
@@ -699,12 +521,14 @@ export function useFileSystem() {
|
||||
continue
|
||||
}
|
||||
try {
|
||||
const payload = await readFilePayload(file)
|
||||
const created = createFile(parentId, file.name, payload.content || '', payload)
|
||||
if (created) success += 1
|
||||
else failed.push({ name: file.name, reason: error.value || '创建文件失败' })
|
||||
} catch {
|
||||
failed.push({ name: file.name, reason: '读取文件失败' })
|
||||
const node = await uploadDocFile(file, parentId)
|
||||
upsertRecord(node)
|
||||
success += 1
|
||||
} catch (err) {
|
||||
failed.push({
|
||||
name: file.name,
|
||||
reason: err instanceof Error && err.message ? err.message : '上传文件失败',
|
||||
})
|
||||
}
|
||||
}
|
||||
if (success > 0 && parentId) {
|
||||
@@ -715,19 +539,20 @@ export function useFileSystem() {
|
||||
return { success, failed }
|
||||
}
|
||||
|
||||
function getFileBlob(node) {
|
||||
async function getFileBlob(node) {
|
||||
if (!node || node.type !== 'file') return null
|
||||
if (node.blob instanceof Blob) return node.blob
|
||||
if (typeof node.content === 'string') {
|
||||
return new Blob([node.content], { type: inferMimeType(node.name, node.mimeType) })
|
||||
if (blobCache.has(node.id)) return blobCache.get(node.id)
|
||||
if (node.storageKind === 'text' && typeof node.content === 'string' && node.content && !node.isTruncatedPreview) {
|
||||
const blob = new Blob([node.content], { type: inferMimeType(node.name, node.mimeType) })
|
||||
blobCache.set(node.id, blob)
|
||||
return blob
|
||||
}
|
||||
if (typeof node.previewText === 'string' && node.previewText) {
|
||||
return new Blob([node.previewText], { type: inferMimeType(node.name, node.mimeType) })
|
||||
}
|
||||
return null
|
||||
const blob = await fetchDocBlob(node.id)
|
||||
blobCache.set(node.id, blob)
|
||||
return blob
|
||||
}
|
||||
|
||||
return {
|
||||
singleton = {
|
||||
tree,
|
||||
selectedId,
|
||||
expandedIds,
|
||||
@@ -761,4 +586,6 @@ export function useFileSystem() {
|
||||
MAX_FILE_SIZE,
|
||||
MAX_NODES
|
||||
}
|
||||
|
||||
return singleton
|
||||
}
|
||||
|
||||
+8
-3
@@ -1,6 +1,6 @@
|
||||
export const DEBUG = import.meta.env.DEV
|
||||
|
||||
const DEFAULT_API_BASE_URL = import.meta.env.DEV ? '' : 'https://api.imageteach.tech:8002'
|
||||
const DEFAULT_API_BASE_URL = ''
|
||||
const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || DEFAULT_API_BASE_URL
|
||||
|
||||
export const API_URL = import.meta.env.VITE_API_URL || `${API_BASE_URL}/v1/completions`
|
||||
@@ -15,8 +15,13 @@ export const TTS_CONFIG_URL = import.meta.env.VITE_TTS_CONFIG_URL || `${API_BASE
|
||||
export const ASR_URL = import.meta.env.VITE_ASR_URL || `${API_BASE_URL}/v1/tts-asr/asr`
|
||||
export const JOB_LOAD_URL = import.meta.env.VITE_JOB_LOAD_URL || `${API_BASE_URL}/v1/jobs/load`
|
||||
export const API_KEY = import.meta.env.VITE_API_KEY || 'your-secret-key-here'
|
||||
export const DOCS_NODES_URL = import.meta.env.VITE_DOCS_NODES_URL || `${API_BASE_URL}/v1/docs/nodes`
|
||||
export const DOCS_FOLDERS_URL = import.meta.env.VITE_DOCS_FOLDERS_URL || `${API_BASE_URL}/v1/docs/folders`
|
||||
export const DOCS_TEXT_FILES_URL = import.meta.env.VITE_DOCS_TEXT_FILES_URL || `${API_BASE_URL}/v1/docs/files/text`
|
||||
export const DOCS_UPLOAD_URL = import.meta.env.VITE_DOCS_UPLOAD_URL || `${API_BASE_URL}/v1/docs/files/upload`
|
||||
export const DOCS_BLOB_BASE_URL = import.meta.env.VITE_DOCS_BLOB_BASE_URL || `${API_BASE_URL}/v1/docs/files`
|
||||
export const DOCS_NODES_BASE_URL = import.meta.env.VITE_DOCS_NODES_BASE_URL || `${API_BASE_URL}/v1/docs/nodes`
|
||||
|
||||
// Compression always goes to local backend (not through reverse proxy)
|
||||
const COMPRESS_BASE_URL = import.meta.env.VITE_COMPRESS_BACKEND || 'http://localhost:8001'
|
||||
const COMPRESS_BASE_URL = import.meta.env.VITE_COMPRESS_BACKEND || API_BASE_URL
|
||||
export const COMPRESS_SUBMIT_URL = `${COMPRESS_BASE_URL}/v1/compress/submit`
|
||||
export const COMPRESS_STATUS_URL = `${COMPRESS_BASE_URL}/v1/compress/status`
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
import {
|
||||
API_KEY,
|
||||
DOCS_BLOB_BASE_URL,
|
||||
DOCS_FOLDERS_URL,
|
||||
DOCS_NODES_BASE_URL,
|
||||
DOCS_NODES_URL,
|
||||
DOCS_TEXT_FILES_URL,
|
||||
DOCS_UPLOAD_URL,
|
||||
} from './config.js'
|
||||
|
||||
function buildHeaders(extra = {}) {
|
||||
return {
|
||||
'X-API-Key': API_KEY,
|
||||
...extra,
|
||||
}
|
||||
}
|
||||
|
||||
async function parseJsonResponse(res) {
|
||||
if (!res.ok) {
|
||||
let message = `HTTP ${res.status}`
|
||||
try {
|
||||
const data = await res.json()
|
||||
message = data.detail || data.error || message
|
||||
} catch {
|
||||
const text = await res.text()
|
||||
if (text) message = text
|
||||
}
|
||||
throw new Error(message)
|
||||
}
|
||||
return res.json()
|
||||
}
|
||||
|
||||
export async function fetchDocNodes() {
|
||||
const res = await fetch(DOCS_NODES_URL, {
|
||||
headers: buildHeaders(),
|
||||
})
|
||||
const data = await parseJsonResponse(res)
|
||||
return Array.isArray(data.nodes) ? data.nodes : []
|
||||
}
|
||||
|
||||
export async function createDocFolder(name, parentId = null) {
|
||||
const res = await fetch(DOCS_FOLDERS_URL, {
|
||||
method: 'POST',
|
||||
headers: buildHeaders({ 'Content-Type': 'application/json' }),
|
||||
body: JSON.stringify({ name, parentId }),
|
||||
})
|
||||
const data = await parseJsonResponse(res)
|
||||
return data.node
|
||||
}
|
||||
|
||||
export async function createDocTextFile(name, parentId = null, content = '') {
|
||||
const res = await fetch(DOCS_TEXT_FILES_URL, {
|
||||
method: 'POST',
|
||||
headers: buildHeaders({ 'Content-Type': 'application/json' }),
|
||||
body: JSON.stringify({ name, parentId, content }),
|
||||
})
|
||||
const data = await parseJsonResponse(res)
|
||||
return data.node
|
||||
}
|
||||
|
||||
export async function uploadDocFile(file, parentId = null) {
|
||||
const formData = new FormData()
|
||||
formData.append('file', file)
|
||||
formData.append('parent_id', parentId || '')
|
||||
const res = await fetch(DOCS_UPLOAD_URL, {
|
||||
method: 'POST',
|
||||
headers: buildHeaders(),
|
||||
body: formData,
|
||||
})
|
||||
const data = await parseJsonResponse(res)
|
||||
return data.node
|
||||
}
|
||||
|
||||
export async function updateDocNode(nodeId, payload) {
|
||||
const res = await fetch(`${DOCS_NODES_BASE_URL}/${encodeURIComponent(nodeId)}`, {
|
||||
method: 'PATCH',
|
||||
headers: buildHeaders({ 'Content-Type': 'application/json' }),
|
||||
body: JSON.stringify(payload),
|
||||
})
|
||||
const data = await parseJsonResponse(res)
|
||||
return data.node
|
||||
}
|
||||
|
||||
export async function replaceDocBlob(nodeId, file) {
|
||||
const formData = new FormData()
|
||||
formData.append('file', file, file.name || 'file')
|
||||
const res = await fetch(`${DOCS_BLOB_BASE_URL}/${encodeURIComponent(nodeId)}/blob`, {
|
||||
method: 'PUT',
|
||||
headers: buildHeaders(),
|
||||
body: formData,
|
||||
})
|
||||
const data = await parseJsonResponse(res)
|
||||
return data.node
|
||||
}
|
||||
|
||||
export async function deleteDocNode(nodeId) {
|
||||
const res = await fetch(`${DOCS_NODES_BASE_URL}/${encodeURIComponent(nodeId)}`, {
|
||||
method: 'DELETE',
|
||||
headers: buildHeaders(),
|
||||
})
|
||||
return parseJsonResponse(res)
|
||||
}
|
||||
|
||||
export async function fetchDocBlob(nodeId) {
|
||||
const res = await fetch(`${DOCS_BLOB_BASE_URL}/${encodeURIComponent(nodeId)}/blob`, {
|
||||
headers: buildHeaders(),
|
||||
})
|
||||
if (!res.ok) {
|
||||
let message = `HTTP ${res.status}`
|
||||
try {
|
||||
const data = await res.json()
|
||||
message = data.detail || data.error || message
|
||||
} catch {
|
||||
const text = await res.text()
|
||||
if (text) message = text
|
||||
}
|
||||
throw new Error(message)
|
||||
}
|
||||
return res.blob()
|
||||
}
|
||||
+11
-11
@@ -28,16 +28,16 @@ const storageSummary = computed(() => {
|
||||
return `${value >= 100 || index === 0 ? value.toFixed(0) : value.toFixed(1)} ${units[index]}`
|
||||
})
|
||||
|
||||
function handleCreateFile(parentId, name = 'untitled.md') {
|
||||
fs.createFile(parentId, name)
|
||||
async function handleCreateFile(parentId, name = 'untitled.md') {
|
||||
await fs.createFile(parentId, name)
|
||||
}
|
||||
|
||||
function handleCreateFolder(parentId, name = '新建文件夹') {
|
||||
fs.createFolder(parentId, name)
|
||||
async function handleCreateFolder(parentId, name = '新建文件夹') {
|
||||
await fs.createFolder(parentId, name)
|
||||
}
|
||||
|
||||
function handleRename(id, newName) {
|
||||
fs.rename(id, newName)
|
||||
async function handleRename(id, newName) {
|
||||
await fs.rename(id, newName)
|
||||
}
|
||||
|
||||
function findNode(nodes, id) {
|
||||
@@ -61,9 +61,9 @@ function handleDelete(id) {
|
||||
}
|
||||
}
|
||||
|
||||
function confirmDelete() {
|
||||
async function confirmDelete() {
|
||||
if (!confirmDialog.value) return
|
||||
fs.remove(confirmDialog.value.id)
|
||||
await fs.remove(confirmDialog.value.id)
|
||||
confirmDialog.value = null
|
||||
}
|
||||
|
||||
@@ -71,10 +71,10 @@ function handleContextMenu(x, y, node) {
|
||||
fs.showContextMenu(x, y, node)
|
||||
}
|
||||
|
||||
function handleDrop(draggedId, targetParentId) {
|
||||
async function handleDrop(draggedId, targetParentId) {
|
||||
if (draggedId === targetParentId) return
|
||||
fs.cut(draggedId)
|
||||
fs.paste(targetParentId)
|
||||
await fs.paste(targetParentId)
|
||||
}
|
||||
|
||||
async function handleUploadFiles(files) {
|
||||
@@ -148,7 +148,7 @@ function closeConfirm() {
|
||||
</div>
|
||||
|
||||
<div class="toolbar-right">
|
||||
<span class="storage-pill">本地存储 {{ storageSummary }}</span>
|
||||
<span class="storage-pill">文档空间 {{ storageSummary }}</span>
|
||||
<button
|
||||
v-if="fs.canPaste()"
|
||||
class="toolbar-btn"
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ export default defineConfig({
|
||||
port: 5173,
|
||||
proxy: {
|
||||
'/v1': {
|
||||
target: 'https://api.imageteach.tech:8002',
|
||||
target: 'http://localhost:8001',
|
||||
changeOrigin: true
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user