- Video pipeline: video file OCR via VLM plus audio track ASR, integrated
into job_handlers with progress emit per phase. New media_utils.py for
audio extraction from video files.
- Document export: richExport.js replaces inline docx builder; DOCX and PDF
export buttons are now enabled in MilkdownEditor. File size limit raised to
100 MB.
- Input block: new InputBlockCrepe.vue component with inputBlockPlugin.ts
and inputBlock.js for custom user-input nodes in the editor.
- Risk config: added Vite dev server ports (5173) to CORS allowlist and
increased OCR max input from 10 MB to 100 MB.
- TTS/ASR refactor: simplified tts_asr.py model loading and warmup logic.
- Test coverage: updated tests for llm, main endpoints, pro completions and
web search modules.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Update WAV encoding logic to convert audio to a NumPy array, employ a
temporary file for safe write with soundfile, and ensure cleanup in a
finally block. This resolves the BytesIO limitation and improves the
reliability of the TTS endpoint.
Add support to download and load TTS model from ModelScope, with a fallback to the HuggingFace mirror.
Implement a `documentLoadStatus` property and helper functions in `office.js` to track file loading state.
Improve request cancellation logic in `api.js`, ensuring proper cancel URL resolution and request‑id handling.
These changes enhance robustness, reduce external dependencies, and provide better UX for office file handling.
- Add AGENTS.md knowledge base with project documentation
- Move UserPreferences model to separate models.py file
- Extract API_KEY to environment variable for security
- Enhance Univer Editor with PPTX support and improved UI
- Improve file system handling with binary file detection
- Add HF_ENDPOINT mirror for better China connectivity
- Clean up unused imports and code structure
Added a new `.coveragerc` file configuring coverage thresholds and exclusions.
Included `pytest.ini` to enable coverage reporting for multiple backend modules (`main`, `llm`, `prompt`, `geoip`, `tts_asr`) with a 90 % fail‑under requirement and detailed HTML output.
Implemented a suite of unit tests:
* `test_geoip.py` – validates geo‑location lookup logic.
* `test_llm_extended.py` – tests LLm response extraction and Ollama interactions.
* `test_main_endpoints.py` – covers API endpoints for completions, OCR, and TTS.
* `test_prompt_extended.py` – verifies language sanitization, timestamp generation, and prompt building.
* `test_tts_asr_coverage.py` – checks device detection, cache clearing, and model loading under various environment configurations.
* `test_tts_asr_extended.py` – further tests TTS/ASR device selection and time‑outs.
Updated `backend/requirements.txt` to use newer, compatible packages, removed obsolete testing dependencies, and added `qwen-tts`.
Modified `backend/tts_asr.py` to work with the new `Qwen3TTSModel`, simplified imports, and adjusted device mapping logic.
Additionally, frontend changes added a new `TreeNodeItem` component, updated Markdown rendering, added TTS instruction fields, and reworked context menu handling.
No breaking changes were introduced.
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
Adds a DocBlock component that renders embedded documents, new export buttons for DOCX
and PDF, and updates the file‑upload picker to accept *.txt, *.docx, *.pptx, and *.pdf.
Introduces a DOCX→PDF conversion bridge in the backend and new /tts and /asr
endpoints that expose TTS and speech‑recognition functionality. The README is
rewritten to describe the new features and clean up legacy documentation. All
changes are backward‑compatible and do not introduce breaking API changes.
“ydy0615”
ydy0615