feat: sync full-stack Docker runtime and UI
This commit is contained in:
“ydy0615”
@@ -1,453 +0,0 @@
|
||||
# TTS/ASR 测试指南
|
||||
|
||||
本文档提供完整的测试脚本使用说明，包括单元测试、集成测试和macOS环境模拟测试。
|
||||
|
||||
## 测试脚本概览
|
||||
|
||||
| 脚本 | 位置 | 用途 | 需要后端服务 |
|
||||
|------|------|------|--------------|
|
||||
| `test_tts_asr_unit.py` | `backend/tests/` | 单元测试（设备检测、模型选择、音频处理） | 否 |
|
||||
| `test_tts_asr_integration.py` | `backend/tests/` | 集成测试（API端点、完整流程） | 是 |
|
||||
| `simulate_macos.py` | `backend/tests/` | macOS环境模拟（在非Mac环境测试） | 否 |
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 单元测试（推荐首先运行）
|
||||
|
||||
单元测试不需要实际运行模型或后端服务，测试代码逻辑：
|
||||
|
||||
```bash
|
||||
# 使用pytest运行（推荐）
|
||||
pytest backend/tests/test_tts_asr_unit.py -v
|
||||
|
||||
# 直接运行
|
||||
python backend/tests/test_tts_asr_unit.py
|
||||
|
||||
# 运行特定测试类
|
||||
pytest backend/tests/test_tts_asr_unit.py::TestAppleSiliconDetection -v
|
||||
|
||||
# 运行特定测试方法
|
||||
pytest backend/tests/test_tts_asr_unit.py::TestAppleSiliconDetection::test_is_apple_silicon_on_darwin_arm64 -v
|
||||
```
|
||||
|
||||
### 2. macOS环境模拟测试
|
||||
|
||||
在非macOS环境下模拟Apple Silicon环境：
|
||||
|
||||
```bash
|
||||
# 运行完整模拟测试套件
|
||||
python backend/tests/simulate_macos.py --full-simulation
|
||||
|
||||
# 仅模拟Apple Silicon环境并进入交互模式
|
||||
python backend/tests/simulate_macos.py --apple-silicon
|
||||
|
||||
# 模拟特定设备
|
||||
python backend/tests/simulate_macos.py --device mps
|
||||
python backend/tests/simulate_macos.py --device cuda
|
||||
|
||||
# 运行特定测试
|
||||
python backend/tests/simulate_macos.py --test device # 设备检测
|
||||
python backend/tests/simulate_macos.py --test memory # 内存管理
|
||||
python backend/tests/simulate_macos.py --test model # 模型选择
|
||||
python backend/tests/simulate_macos.py --test audio # 音频处理
|
||||
python backend/tests/simulate_macos.py --test env # 环境变量
|
||||
```
|
||||
|
||||
### 3. 集成测试
|
||||
|
||||
集成测试需要运行后端服务：
|
||||
|
||||
```bash
|
||||
# 1. 启动后端服务（终端1）
|
||||
python backend/main.py
|
||||
|
||||
# 2. 运行集成测试（终端2）
|
||||
# 运行所有测试
|
||||
python backend/tests/test_tts_asr_integration.py
|
||||
|
||||
# 运行特定测试
|
||||
python backend/tests/test_tts_asr_integration.py --test config # 配置端点
|
||||
python backend/tests/test_tts_asr_integration.py --test status # 状态端点
|
||||
python backend/tests/test_tts_asr_integration.py --test warmup # 预热测试
|
||||
python backend/tests/test_tts_asr_integration.py --test tts # TTS测试
|
||||
python backend/tests/test_tts_asr_integration.py --test asr # ASR测试
|
||||
python backend/tests/test_tts_asr_integration.py --test perf # 性能测试
|
||||
|
||||
# 自定义API地址
|
||||
python backend/tests/test_tts_asr_integration.py --url http://localhost:8001 --key your-api-key
|
||||
```
|
||||
|
||||
## 详细测试说明
|
||||
|
||||
### 单元测试详解
|
||||
|
||||
#### TestAppleSiliconDetection
|
||||
|
||||
测试Apple Silicon检测功能：
|
||||
|
||||
- `test_is_apple_silicon_on_darwin_arm64`: 在Darwin/arm64环境检测
|
||||
- `test_is_apple_silicon_on_windows`: 在Windows环境不应检测到
|
||||
- `test_is_apple_silicon_on_linux`: 在Linux环境不应检测到
|
||||
|
||||
#### TestEnvironmentVariables
|
||||
|
||||
测试环境变量解析：
|
||||
|
||||
- `test_default_environment_values`: 验证默认值
|
||||
- `test_custom_environment_values`: 验证自定义值
|
||||
|
||||
#### TestModelSizeSelection
|
||||
|
||||
测试模型大小选择：
|
||||
|
||||
- `test_whisper_model_sizes_mapping`: 模型大小映射验证
|
||||
- `test_recommended_model_size_explicit`: 显式指定大小
|
||||
- `test_invalid_model_size_falls_back`: 无效大小回退
|
||||
|
||||
#### TestAudioValidation
|
||||
|
||||
测试音频验证：
|
||||
|
||||
- `test_validate_empty_audio`: 空音频验证
|
||||
- `test_validate_valid_wav_header`: 有效WAV头验证
|
||||
- `test_validate_invalid_audio`: 无效音频验证
|
||||
|
||||
#### TestAudioResampling
|
||||
|
||||
测试音频重采样：
|
||||
|
||||
- `test_resample_same_rate`: 相同采样率
|
||||
- `test_resample_different_rate`: 不同采样率重采样
|
||||
- `test_resample_downsample`: 下采样
|
||||
|
||||
#### TestDeviceCapabilities
|
||||
|
||||
测试设备能力检测：
|
||||
|
||||
- `test_device_capabilities_dataclass`: 数据类验证
|
||||
- `test_device_capabilities_with_mps`: MPS设备能力
|
||||
|
||||
#### TestModelCacheCheck
|
||||
|
||||
测试模型缓存检查：
|
||||
|
||||
- `test_cache_check_non_offline_mode`: 非离线模式
|
||||
- `test_cache_check_offline_mode_missing`: 离线模式缺失模型
|
||||
|
||||
#### TestRequestResponseModels
|
||||
|
||||
测试API模型：
|
||||
|
||||
- `test_tts_request_model`: TTS请求模型
|
||||
- `test_asr_request_model`: ASR请求模型
|
||||
- `test_model_status_model`: 状态模型
|
||||
|
||||
### 集成测试详解
|
||||
|
||||
#### TTSASRIntegrationTest
|
||||
|
||||
主要集成测试：
|
||||
|
||||
- `test_01_config_endpoint`: 配置端点测试
|
||||
- `test_02_status_endpoint`: 状态端点测试
|
||||
- `test_03_warmup_endpoint`: 预热端点测试
|
||||
- `test_04_tts_endpoint_basic`: TTS基本功能测试
|
||||
- `test_05_asr_endpoint_basic`: ASR基本功能测试
|
||||
- `test_06_api_key_validation`: API密钥验证测试
|
||||
- `test_07_tts_long_text`: TTS长文本测试
|
||||
|
||||
#### PerformanceTest
|
||||
|
||||
性能测试：
|
||||
|
||||
- `test_tts_latency`: TTS延迟测试
|
||||
|
||||
### macOS模拟测试详解
|
||||
|
||||
#### MacOSSimulator类
|
||||
|
||||
提供以下模拟功能：
|
||||
|
||||
- `simulate_apple_silicon()`: 模拟Darwin/arm64环境
|
||||
- `simulate_mps_device()`: 模拟MPS设备可用
|
||||
- `simulate_cuda_device()`: 模拟CUDA设备可用
|
||||
- `cleanup()`: 清理模拟环境
|
||||
|
||||
#### 独立测试函数
|
||||
|
||||
- `test_device_detection_on_apple_silicon()`: Apple Silicon设备检测
|
||||
- `test_memory_management()`: 内存管理测试
|
||||
- `test_model_size_selection()`: 模型大小选择测试
|
||||
- `test_audio_processing()`: 音频处理测试
|
||||
- `test_environment_variables()`: 环境变量测试
|
||||
|
||||
## 测试覆盖率
|
||||
|
||||
### 单元测试覆盖的功能
|
||||
|
||||
- [x] Apple Silicon检测逻辑
|
||||
- [x] 环境变量解析和默认值
|
||||
- [x] 模型大小选择和推荐
|
||||
- [x] 音频数据验证
|
||||
- [x] 音频重采样（多回退方案）
|
||||
- [x] 设备能力检测数据结构
|
||||
- [x] 模型缓存检查
|
||||
- [x] API请求/响应模型
|
||||
|
||||
### 集成测试覆盖的功能
|
||||
|
||||
- [x] 配置端点（`/v1/tts-asr/config`）
|
||||
- [x] 状态端点（`/v1/tts-asr/status`）
|
||||
- [x] 预热端点（`/v1/tts-asr/warmup`）
|
||||
- [x] TTS端点（`/v1/tts-asr/tts`）
|
||||
- [x] ASR端点（`/v1/tts-asr/asr`）
|
||||
- [x] API密钥验证
|
||||
- [x] 长文本处理
|
||||
- [x] 性能基准测试
|
||||
|
||||
### macOS模拟测试覆盖的场景
|
||||
|
||||
- [x] Apple Silicon环境模拟
|
||||
- [x] MPS设备模拟
|
||||
- [x] CUDA设备模拟
|
||||
- [x] 系统内存模拟
|
||||
- [x] 完整环境变量测试
|
||||
|
||||
## 常见测试场景
|
||||
|
||||
### 场景1: 开发时快速验证
|
||||
|
||||
```bash
|
||||
# 快速单元测试
|
||||
pytest backend/tests/test_tts_asr_unit.py -v --tb=short
|
||||
|
||||
# macOS模拟（完整）
|
||||
python backend/tests/simulate_macos.py --full-simulation
|
||||
```
|
||||
|
||||
### 场景2: 验证特定配置
|
||||
|
||||
```bash
|
||||
# 设置环境变量后测试
|
||||
export TTS_ASR_MODEL_SIZE=small
|
||||
export TTS_ASR_QUANTIZE=true
|
||||
|
||||
# 运行测试
|
||||
python backend/tests/simulate_macos.py --test model
|
||||
```
|
||||
|
||||
### 场景3: API功能验证
|
||||
|
||||
```bash
|
||||
# 启动服务
|
||||
python backend/main.py
|
||||
|
||||
# 测试配置端点
|
||||
python backend/tests/test_tts_asr_integration.py --test config
|
||||
|
||||
# 测试TTS功能
|
||||
python backend/tests/test_tts_asr_integration.py --test tts
|
||||
|
||||
# 测试ASR功能
|
||||
python backend/tests/test_tts_asr_integration.py --test asr
|
||||
```
|
||||
|
||||
### 场景4: 性能基准测试
|
||||
|
||||
```bash
|
||||
# 启动服务
|
||||
python backend/main.py
|
||||
|
||||
# 运行性能测试
|
||||
python backend/tests/test_tts_asr_integration.py --test perf
|
||||
```
|
||||
|
||||
## 测试输出解读
|
||||
|
||||
### 成功示例
|
||||
|
||||
```
|
||||
test_is_apple_silicon_on_darwin_arm64 ... ok
|
||||
test_is_apple_silicon_on_windows ... ok
|
||||
test_is_apple_silicon_on_linux ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 3 tests in 0.005s
|
||||
|
||||
OK
|
||||
```
|
||||
|
||||
### 失败示例
|
||||
|
||||
```
|
||||
test_device_detection_on_apple_silicon ... FAIL
|
||||
|
||||
======================================================================
|
||||
FAIL: test_device_detection_on_apple_silicon
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "test_tts_asr_unit.py", line 45, in test_is_apple_silicon_on_darwin_arm64
|
||||
self.assertTrue(_is_apple_silicon())
|
||||
AssertionError: False is not true
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 tests in 0.002s
|
||||
|
||||
FAILED (failures=1)
|
||||
```
|
||||
|
||||
## 持续集成配置
|
||||
|
||||
### GitHub Actions示例
|
||||
|
||||
```yaml
|
||||
name: TTS/ASR Tests
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
unit-tests:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.10'
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
pip install -r backend/requirements.txt
|
||||
pip install pytest
|
||||
- name: Run unit tests
|
||||
run: pytest backend/tests/test_tts_asr_unit.py -v
|
||||
- name: Run macOS simulation
|
||||
run: python backend/tests/simulate_macos.py --full-simulation
|
||||
```
|
||||
|
||||
### pytest配置
|
||||
|
||||
创建 `pytest.ini`:
|
||||
|
||||
```ini
|
||||
[pytest]
|
||||
testpaths = backend/tests
|
||||
python_files = test_*.py
|
||||
python_classes = Test*
|
||||
python_functions = test_*
|
||||
addopts = -v --tb=short
|
||||
```
|
||||
|
||||
## 故障排查
|
||||
|
||||
### 问题1: 导入错误
|
||||
|
||||
```
|
||||
ModuleNotFoundError: No module named 'backend'
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
|
||||
```bash
|
||||
# 确保在项目根目录运行
|
||||
cd /path/to/llm-in-text
|
||||
|
||||
# 或设置PYTHONPATH
|
||||
export PYTHONPATH="${PYTHONPATH}:$(pwd)"
|
||||
```
|
||||
|
||||
### 问题2: 后端服务连接失败
|
||||
|
||||
```
|
||||
✗ 无法连接到服务: [Errno 111] Connection refused
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
|
||||
```bash
|
||||
# 确保后端服务正在运行
|
||||
python backend/main.py
|
||||
|
||||
# 检查端口
|
||||
lsof -i :8001
|
||||
|
||||
# 或使用自定义URL
|
||||
python backend/tests/test_tts_asr_integration.py --url http://localhost:8001
|
||||
```
|
||||
|
||||
### 问题3: 模型未加载
|
||||
|
||||
```
|
||||
⚠ TTS失败（可能是模型未加载）
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
|
||||
这是预期行为，表示模型需要时间下载。可以：
|
||||
|
||||
1. 等待模型下载完成
|
||||
2. 使用预热端点: `POST /v1/tts-asr/warmup`
|
||||
3. 启用离线模式（如果模型已下载）
|
||||
|
||||
### 问题4: 测试超时
|
||||
|
||||
```
|
||||
httpx.ReadTimeout: timed out
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
|
||||
```bash
|
||||
# 增加超时时间
|
||||
export TEST_TIMEOUT=300.0
|
||||
|
||||
# 或在测试脚本中修改
|
||||
TEST_TIMEOUT = 300.0 # 5分钟
|
||||
```
|
||||
|
||||
## 最佳实践
|
||||
|
||||
1. **开发时**: 频繁运行单元测试
|
||||
```bash
|
||||
pytest backend/tests/test_tts_asr_unit.py -v --tb=short
|
||||
```
|
||||
|
||||
2. **提交前**: 运行完整测试套件
|
||||
```bash
|
||||
pytest backend/tests/test_tts_asr_unit.py -v
|
||||
python backend/tests/simulate_macos.py --full-simulation
|
||||
```
|
||||
|
||||
3. **部署前**: 运行集成测试
|
||||
```bash
|
||||
python backend/tests/test_tts_asr_integration.py
|
||||
```
|
||||
|
||||
4. **调试时**: 使用详细输出
|
||||
```bash
|
||||
pytest backend/tests/test_tts_asr_unit.py -v -s --tb=long
|
||||
```
|
||||
|
||||
## 测试报告
|
||||
|
||||
生成测试覆盖率报告：
|
||||
|
||||
```bash
|
||||
# 安装coverage
|
||||
pip install pytest-cov
|
||||
|
||||
# 运行并生成报告
|
||||
pytest backend/tests/test_tts_asr_unit.py --cov=backend.tts_asr --cov-report=html
|
||||
|
||||
# 查看报告
|
||||
open htmlcov/index.html
|
||||
```
|
||||
|
||||
## 相关文档
|
||||
|
||||
- [TTS/ASR修复说明](./TTS_ASR_MACOS_FIX.md)
|
||||
- [环境变量配置](../README.md#ttsasr环境变量配置)
|
||||
- [API文档](../README.md#api接口)
|
||||
|
||||
---
|
||||
|
||||
**更新日期**: 2026-04-06
|
||||
**维护者**: 项目开发团队
|
||||
@@ -0,0 +1,209 @@
|
||||
"""Lightweight benchmark for TTS/ASR queueing and API throughput.
|
||||
|
||||
This benchmark uses the FastAPI app with a mocked upstream speech API so it
|
||||
measures this project's queueing, request handling, and SSE delivery cost
|
||||
without requiring a real external model endpoint.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import statistics
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
BACKEND_DIR = Path(__file__).resolve().parents[1]
|
||||
if str(BACKEND_DIR) not in __import__("sys").path:
|
||||
__import__("sys").path.insert(0, str(BACKEND_DIR))
|
||||
|
||||
import main # noqa: E402
|
||||
import tts_asr # noqa: E402
|
||||
from job_system import reset_job_manager # noqa: E402
|
||||
|
||||
|
||||
def _wav_bytes(duration_ms: int = 320) -> bytes:
|
||||
sample_rate = 16000
|
||||
frames = max(1, int(sample_rate * duration_ms / 1000))
|
||||
data = b"".join((i % 32768).to_bytes(2, "little", signed=False) for i in range(frames))
|
||||
data_size = len(data)
|
||||
return (
|
||||
b"RIFF" + (36 + data_size).to_bytes(4, "little")
|
||||
+ b"WAVE"
|
||||
+ b"fmt " + (16).to_bytes(4, "little")
|
||||
+ (1).to_bytes(2, "little")
|
||||
+ (1).to_bytes(2, "little")
|
||||
+ sample_rate.to_bytes(4, "little")
|
||||
+ sample_rate.to_bytes(4, "little")
|
||||
+ (2).to_bytes(2, "little")
|
||||
+ (16).to_bytes(2, "little")
|
||||
+ b"data" + data_size.to_bytes(4, "little")
|
||||
+ data
|
||||
)
|
||||
|
||||
|
||||
def _parse_sse_done(text: str) -> dict:
|
||||
for chunk in reversed([item for item in text.split("\n\n") if item.strip()]):
|
||||
event = ""
|
||||
data = ""
|
||||
for line in chunk.splitlines():
|
||||
if line.startswith("event:"):
|
||||
event = line.split(":", 1)[1].strip()
|
||||
elif line.startswith("data:"):
|
||||
data = line.split(":", 1)[1].strip()
|
||||
if event == "done" and data:
|
||||
payload = json.loads(data)
|
||||
result = dict(payload.get("result") or {})
|
||||
for key in ("queue_ms", "run_ms", "total_ms", "queued_count", "running_count", "busy_level", "busy_ratio"):
|
||||
if key in payload:
|
||||
result[key] = payload[key]
|
||||
return result
|
||||
raise RuntimeError("done event not found")
|
||||
|
||||
|
||||
def _percentile(values: list[float], q: float) -> float:
|
||||
if not values:
|
||||
return 0.0
|
||||
if len(values) == 1:
|
||||
return values[0]
|
||||
index = (len(values) - 1) * q
|
||||
lower = int(index)
|
||||
upper = min(lower + 1, len(values) - 1)
|
||||
if lower == upper:
|
||||
return values[lower]
|
||||
weight = index - lower
|
||||
return values[lower] * (1 - weight) + values[upper] * weight
|
||||
|
||||
|
||||
async def _build_mock_client(tts_delay_ms: int, asr_delay_ms: int) -> httpx.AsyncClient:
|
||||
async def transport(request: httpx.Request):
|
||||
if request.url.path.endswith("/audio/speech"):
|
||||
await asyncio.sleep(tts_delay_ms / 1000.0)
|
||||
return httpx.Response(200, content=_wav_bytes(420), headers={"x-request-id": "bench-tts"}, request=request)
|
||||
await asyncio.sleep(asr_delay_ms / 1000.0)
|
||||
return httpx.Response(200, json={"text": "benchmark transcript", "language": "zh"}, headers={"x-request-id": "bench-asr"}, request=request)
|
||||
|
||||
return httpx.AsyncClient(
|
||||
base_url="https://benchmark.example/v1/",
|
||||
transport=httpx.MockTransport(transport),
|
||||
)
|
||||
|
||||
|
||||
async def _run_case(case_name: str, concurrency: int, request_count: int, audio_b64: str | None = None) -> dict:
|
||||
results: list[dict] = []
|
||||
latencies: list[float] = []
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=main.app),
|
||||
base_url="http://testserver",
|
||||
timeout=120.0,
|
||||
) as client:
|
||||
semaphore = asyncio.Semaphore(concurrency)
|
||||
|
||||
async def fire(index: int) -> None:
|
||||
async with semaphore:
|
||||
started = time.perf_counter()
|
||||
if case_name == "tts":
|
||||
response = await client.post(
|
||||
"/v1/tts-asr/tts",
|
||||
json={"text": f"第 {index} 条基准文本", "speaker": "Vivian", "format": "wav"},
|
||||
)
|
||||
else:
|
||||
response = await client.post(
|
||||
"/v1/tts-asr/asr",
|
||||
json={"audio_base64": audio_b64, "language": "zh-CN"},
|
||||
)
|
||||
response.raise_for_status()
|
||||
payload = _parse_sse_done(response.text)
|
||||
latencies.append((time.perf_counter() - started) * 1000.0)
|
||||
results.append(payload)
|
||||
|
||||
await asyncio.gather(*(fire(index) for index in range(request_count)))
|
||||
|
||||
queue_values = sorted(float(item.get("queue_ms", 0) or 0) for item in results)
|
||||
run_values = sorted(float(item.get("run_ms", 0) or 0) for item in results)
|
||||
total_values = sorted(float(item.get("total_ms", 0) or 0) for item in results)
|
||||
latency_values = sorted(latencies)
|
||||
elapsed_sum_ms = sum(latency_values)
|
||||
return {
|
||||
"case": case_name,
|
||||
"requests": request_count,
|
||||
"concurrency": concurrency,
|
||||
"avg_latency_ms": round(statistics.fmean(latency_values), 2),
|
||||
"p95_latency_ms": round(_percentile(latency_values, 0.95), 2),
|
||||
"avg_queue_ms": round(statistics.fmean(queue_values), 2),
|
||||
"p95_queue_ms": round(_percentile(queue_values, 0.95), 2),
|
||||
"avg_run_ms": round(statistics.fmean(run_values), 2),
|
||||
"p95_run_ms": round(_percentile(run_values, 0.95), 2),
|
||||
"avg_total_ms": round(statistics.fmean(total_values), 2),
|
||||
"p95_total_ms": round(_percentile(total_values, 0.95), 2),
|
||||
"throughput_rps_estimate": round((request_count * 1000.0) / max(latency_values[-1], elapsed_sum_ms / max(request_count, 1)), 2),
|
||||
}
|
||||
|
||||
|
||||
async def main_async(args) -> None:
|
||||
os.environ["JOB_BACKEND"] = "memory"
|
||||
os.environ["JOB_TTS_CONCURRENCY"] = str(args.tts_workers)
|
||||
os.environ["JOB_TTS_MAX_QUEUE"] = str(max(args.tts_requests, args.tts_workers))
|
||||
os.environ["JOB_ASR_CONCURRENCY"] = str(args.asr_workers)
|
||||
os.environ["JOB_ASR_MAX_QUEUE"] = str(max(args.asr_requests, args.asr_workers))
|
||||
reset_job_manager()
|
||||
|
||||
mock_client = await _build_mock_client(args.tts_delay_ms, args.asr_delay_ms)
|
||||
tts_asr._httpx_client = mock_client
|
||||
try:
|
||||
audio_b64 = base64.b64encode(_wav_bytes(args.audio_duration_ms)).decode("utf-8")
|
||||
tts_stats = await _run_case("tts", args.tts_concurrency, args.tts_requests)
|
||||
asr_stats = await _run_case("asr", args.asr_concurrency, args.asr_requests, audio_b64=audio_b64)
|
||||
finally:
|
||||
await mock_client.aclose()
|
||||
tts_asr._httpx_client = None
|
||||
reset_job_manager()
|
||||
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"benchmark_date": time.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"assumptions": {
|
||||
"upstream_tts_delay_ms": args.tts_delay_ms,
|
||||
"upstream_asr_delay_ms": args.asr_delay_ms,
|
||||
"job_backend": "memory",
|
||||
},
|
||||
"tts": tts_stats,
|
||||
"asr": asr_stats,
|
||||
"recommended_defaults": {
|
||||
"JOB_TTS_CONCURRENCY": args.tts_workers,
|
||||
"JOB_TTS_MAX_QUEUE": max(16, args.tts_workers * 4),
|
||||
"JOB_ASR_CONCURRENCY": args.asr_workers,
|
||||
"JOB_ASR_MAX_QUEUE": max(8, args.asr_workers * 4),
|
||||
"TTS_ASR_MAX_CONNECTIONS": max(24, (args.tts_workers + args.asr_workers) * 4),
|
||||
"TTS_ASR_MAX_KEEPALIVE_CONNECTIONS": max(12, (args.tts_workers + args.asr_workers) * 2),
|
||||
},
|
||||
},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--tts-delay-ms", type=int, default=120)
|
||||
parser.add_argument("--asr-delay-ms", type=int, default=280)
|
||||
parser.add_argument("--tts-workers", type=int, default=4)
|
||||
parser.add_argument("--asr-workers", type=int, default=2)
|
||||
parser.add_argument("--tts-concurrency", type=int, default=8)
|
||||
parser.add_argument("--asr-concurrency", type=int, default=4)
|
||||
parser.add_argument("--tts-requests", type=int, default=32)
|
||||
parser.add_argument("--asr-requests", type=int, default=16)
|
||||
parser.add_argument("--audio-duration-ms", type=int, default=320)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main_async(parse_args()))
|
||||
@@ -1,188 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
快速验证脚本
|
||||
验证TTS/ASR模块修复是否正确应用
|
||||
|
||||
运行方式:
|
||||
python backend/tests/quick_verify.py
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# 设置控制台编码
|
||||
if sys.platform == 'win32':
|
||||
import io
|
||||
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
|
||||
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8')
|
||||
|
||||
# 确保可以导入backend模块
|
||||
script_path = Path(__file__).resolve()
|
||||
project_root = script_path.parent.parent.parent
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
print(f"项目根目录: {project_root}")
|
||||
print(f"脚本路径: {script_path}")
|
||||
|
||||
|
||||
def check_file_exists(filepath: str, description: str) -> bool:
|
||||
"""检查文件是否存在"""
|
||||
full_path = project_root / filepath
|
||||
exists = full_path.exists()
|
||||
status = "[OK]" if exists else "[FAIL]"
|
||||
print(f"{status} {description}: {filepath} (完整路径: {full_path})")
|
||||
return exists
|
||||
|
||||
|
||||
def check_function_exists(module_name: str, function_name: str) -> bool:
|
||||
"""检查函数是否存在"""
|
||||
try:
|
||||
module = __import__(module_name, fromlist=[function_name])
|
||||
exists = hasattr(module, function_name)
|
||||
status = "[OK]" if exists else "[FAIL]"
|
||||
print(f"{status} 函数存在: {module_name}.{function_name}")
|
||||
return exists
|
||||
except Exception as e:
|
||||
print(f"[FAIL] 导入失败: {module_name} - {e}")
|
||||
return False
|
||||
|
||||
|
||||
def check_environment_variable(var_name: str, expected_default: str) -> bool:
|
||||
"""检查环境变量默认值"""
|
||||
try:
|
||||
# 清除可能存在的环境变量
|
||||
original_value = os.environ.get(var_name)
|
||||
if var_name in os.environ:
|
||||
del os.environ[var_name]
|
||||
|
||||
# 重新导入模块
|
||||
if 'backend.tts_asr' in sys.modules:
|
||||
del sys.modules['backend.tts_asr']
|
||||
|
||||
from backend.tts_asr import (
|
||||
TTS_ASR_DEVICE, TTS_ASR_MODEL_SIZE, TTS_ASR_QUANTIZE,
|
||||
TTS_ASR_OFFLINE_MODE, TTS_ASR_WARMUP, TTS_ASR_WARMUP_TIMEOUT,
|
||||
TTS_ASR_IDLE_TIMEOUT, TTS_ASR_MPS_MEMORY_LIMIT_MB
|
||||
)
|
||||
|
||||
var_map = {
|
||||
'TTS_ASR_DEVICE': TTS_ASR_DEVICE,
|
||||
'TTS_ASR_MODEL_SIZE': TTS_ASR_MODEL_SIZE,
|
||||
'TTS_ASR_QUANTIZE': TTS_ASR_QUANTIZE,
|
||||
'TTS_ASR_OFFLINE_MODE': TTS_ASR_OFFLINE_MODE,
|
||||
'TTS_ASR_WARMUP': TTS_ASR_WARMUP,
|
||||
'TTS_ASR_WARMUP_TIMEOUT': TTS_ASR_WARMUP_TIMEOUT,
|
||||
'TTS_ASR_IDLE_TIMEOUT': TTS_ASR_IDLE_TIMEOUT,
|
||||
'TTS_ASR_MPS_MEMORY_LIMIT_MB': TTS_ASR_MPS_MEMORY_LIMIT_MB,
|
||||
}
|
||||
|
||||
actual_value = var_map.get(var_name)
|
||||
if var_name == 'TTS_ASR_MODEL_SIZE':
|
||||
expected = 'auto'
|
||||
elif var_name == 'TTS_ASR_QUANTIZE':
|
||||
expected = False
|
||||
elif var_name == 'TTS_ASR_OFFLINE_MODE':
|
||||
expected = False
|
||||
elif var_name == 'TTS_ASR_WARMUP':
|
||||
expected = True
|
||||
elif var_name == 'TTS_ASR_WARMUP_TIMEOUT':
|
||||
expected = 120
|
||||
elif var_name == 'TTS_ASR_IDLE_TIMEOUT':
|
||||
expected = 0
|
||||
elif var_name == 'TTS_ASR_MPS_MEMORY_LIMIT_MB':
|
||||
expected = 8192
|
||||
else:
|
||||
expected = expected_default
|
||||
|
||||
matches = actual_value == expected
|
||||
status = "[OK]" if matches else "[FAIL]"
|
||||
print(f"{status} 环境变量默认值: {var_name} = {actual_value} (预期: {expected})")
|
||||
return matches
|
||||
|
||||
except Exception as e:
|
||||
print(f"[FAIL] 检查环境变量失败: {var_name} - {e}")
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
print("="*70)
|
||||
print("TTS/ASR模块快速验证")
|
||||
print("="*70)
|
||||
|
||||
checks = []
|
||||
|
||||
# 1. 检查文件
|
||||
print("\n[1] 文件检查")
|
||||
print("-"*70)
|
||||
checks.append(check_file_exists("backend/tts_asr.py", "主模块文件"))
|
||||
checks.append(check_file_exists("backend/tests/test_tts_asr_unit.py", "单元测试"))
|
||||
checks.append(check_file_exists("backend/tests/test_tts_asr_integration.py", "集成测试"))
|
||||
checks.append(check_file_exists("backend/tests/simulate_macos.py", "macOS模拟工具"))
|
||||
checks.append(check_file_exists("backend/tests/TESTING_GUIDE.md", "测试指南"))
|
||||
checks.append(check_file_exists("backend/TTS_ASR_MACOS_FIX.md", "修复文档"))
|
||||
|
||||
# 2. 检查核心函数
|
||||
print("\n[2] 核心函数检查")
|
||||
print("-"*70)
|
||||
checks.append(check_function_exists("backend.tts_asr", "_is_apple_silicon"))
|
||||
checks.append(check_function_exists("backend.tts_asr", "_detect_device_capabilities"))
|
||||
checks.append(check_function_exists("backend.tts_asr", "_get_recommended_model_size"))
|
||||
checks.append(check_function_exists("backend.tts_asr", "_validate_audio_data"))
|
||||
checks.append(check_function_exists("backend.tts_asr", "_resample_audio_robust"))
|
||||
checks.append(check_function_exists("backend.tts_asr", "_check_model_cached"))
|
||||
|
||||
# 3. 检查数据类
|
||||
print("\n[3] 数据类检查")
|
||||
print("-"*70)
|
||||
checks.append(check_function_exists("backend.tts_asr", "DeviceCapabilities"))
|
||||
checks.append(check_function_exists("backend.tts_asr", "ModelStatus"))
|
||||
|
||||
# 4. 检查环境变量
|
||||
print("\n[4] 环境变量默认值检查")
|
||||
print("-"*70)
|
||||
checks.append(check_environment_variable("TTS_ASR_DEVICE", "auto"))
|
||||
checks.append(check_environment_variable("TTS_ASR_MODEL_SIZE", "auto"))
|
||||
checks.append(check_environment_variable("TTS_ASR_QUANTIZE", "false"))
|
||||
checks.append(check_environment_variable("TTS_ASR_OFFLINE_MODE", "false"))
|
||||
|
||||
# 5. 检查常量
|
||||
print("\n[5] 常量检查")
|
||||
print("-"*70)
|
||||
try:
|
||||
from backend.tts_asr import WHISPER_MODEL_SIZES, APPLE_SILICON_DEFAULT_SIZE
|
||||
expected_sizes = ['tiny', 'base', 'small', 'medium', 'large', 'turbo']
|
||||
sizes_match = list(WHISPER_MODEL_SIZES.keys()) == expected_sizes
|
||||
status = "[OK]" if sizes_match else "[FAIL]"
|
||||
print(f"{status} WHISPER_MODEL_SIZES: {list(WHISPER_MODEL_SIZES.keys())}")
|
||||
checks.append(sizes_match)
|
||||
|
||||
size_match = APPLE_SILICON_DEFAULT_SIZE == 'small'
|
||||
status = "[OK]" if size_match else "[FAIL]"
|
||||
print(f"{status} APPLE_SILICON_DEFAULT_SIZE: {APPLE_SILICON_DEFAULT_SIZE}")
|
||||
checks.append(size_match)
|
||||
except Exception as e:
|
||||
print(f"[FAIL] 常量检查失败: {e}")
|
||||
checks.extend([False, False])
|
||||
|
||||
# 汇总结果
|
||||
print("\n" + "="*70)
|
||||
print("验证结果")
|
||||
print("="*70)
|
||||
|
||||
total = len(checks)
|
||||
passed = sum(checks)
|
||||
|
||||
print(f"通过: {passed}/{total}")
|
||||
|
||||
if all(checks):
|
||||
print("\n[SUCCESS] 所有验证通过！TTS/ASR模块修复已正确应用。")
|
||||
return 0
|
||||
else:
|
||||
print("\n[FAILED] 部分验证失败，请检查上述错误。")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
+47
-170
@@ -1,16 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
TTS/ASR测试运行器
|
||||
便捷地运行各种测试组合
|
||||
"""Speech test runner for the current API-based TTS/ASR stack."""
|
||||
|
||||
运行方式:
|
||||
python backend/tests/run_tests.py --help
|
||||
python backend/tests/run_tests.py unit
|
||||
python backend/tests/run_tests.py integration
|
||||
python backend/tests/run_tests.py simulate
|
||||
python backend/tests/run_tests.py all
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
@@ -19,186 +10,72 @@ import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def run_command(cmd: list, cwd: str = None) -> int:
|
||||
"""运行命令并返回退出码"""
|
||||
def run_command(cmd: list[str], cwd: str | None = None) -> int:
|
||||
print(f"\n执行: {' '.join(cmd)}")
|
||||
print("-" * 70)
|
||||
result = subprocess.run(cmd, cwd=cwd)
|
||||
return result.returncode
|
||||
return subprocess.run(cmd, cwd=cwd).returncode
|
||||
|
||||
|
||||
def run_unit_tests(verbose: bool = False) -> int:
|
||||
"""运行单元测试"""
|
||||
print("\n" + "="*70)
|
||||
print("运行单元测试")
|
||||
print("="*70)
|
||||
|
||||
cmd = ['pytest', 'backend/tests/test_tts_asr_unit.py']
|
||||
cmd = ["pytest", "backend/tests/test_tts_asr.py"]
|
||||
if verbose:
|
||||
cmd.append('-v')
|
||||
|
||||
cmd.append("-v")
|
||||
return run_command(cmd)
|
||||
|
||||
|
||||
def run_integration_tests(test_type: str = None, url: str = None, key: str = None) -> int:
|
||||
"""运行集成测试"""
|
||||
print("\n" + "="*70)
|
||||
print("运行集成测试")
|
||||
print("="*70)
|
||||
|
||||
cmd = ['python', 'backend/tests/test_tts_asr_integration.py']
|
||||
|
||||
if test_type:
|
||||
cmd.extend(['--test', test_type])
|
||||
|
||||
if url:
|
||||
cmd.extend(['--url', url])
|
||||
|
||||
if key:
|
||||
cmd.extend(['--key', key])
|
||||
|
||||
def run_benchmark(extra_args: list[str] | None = None) -> int:
|
||||
cmd = ["python", "backend/tests/benchmark_tts_asr.py"]
|
||||
if extra_args:
|
||||
cmd.extend(extra_args)
|
||||
return run_command(cmd)
|
||||
|
||||
|
||||
def run_simulation(test_type: str = None) -> int:
|
||||
"""运行macOS模拟测试"""
|
||||
print("\n" + "="*70)
|
||||
print("运行macOS环境模拟测试")
|
||||
print("="*70)
|
||||
|
||||
if test_type == 'full':
|
||||
cmd = ['python', 'backend/tests/simulate_macos.py', '--full-simulation']
|
||||
elif test_type:
|
||||
cmd = ['python', 'backend/tests/simulate_macos.py', '--test', test_type]
|
||||
else:
|
||||
cmd = ['python', 'backend/tests/simulate_macos.py', '--full-simulation']
|
||||
|
||||
return run_command(cmd)
|
||||
def run_all(verbose: bool = False) -> int:
|
||||
results = [
|
||||
("单元测试", run_unit_tests(verbose=verbose)),
|
||||
("基准测试", run_benchmark()),
|
||||
]
|
||||
|
||||
|
||||
def run_all_tests(url: str = None, key: str = None) -> int:
|
||||
"""运行所有测试"""
|
||||
print("\n" + "="*70)
|
||||
print("运行完整测试套件")
|
||||
print("="*70)
|
||||
|
||||
results = []
|
||||
|
||||
# 1. 单元测试
|
||||
print("\n[1/3] 单元测试")
|
||||
results.append(("单元测试", run_unit_tests(verbose=True)))
|
||||
|
||||
# 2. macOS模拟测试
|
||||
print("\n[2/3] macOS模拟测试")
|
||||
results.append(("macOS模拟", run_simulation(test_type='full')))
|
||||
|
||||
# 3. 集成测试（如果服务可用）
|
||||
print("\n[3/3] 集成测试")
|
||||
print("注意: 集成测试需要后端服务运行中")
|
||||
response = input("是否继续运行集成测试? [y/N]: ")
|
||||
|
||||
if response.lower() == 'y':
|
||||
results.append(("集成测试", run_integration_tests(url=url, key=key)))
|
||||
else:
|
||||
print("跳过集成测试")
|
||||
results.append(("集成测试", 0))
|
||||
|
||||
# 汇总结果
|
||||
print("\n" + "="*70)
|
||||
print("\n" + "=" * 70)
|
||||
print("测试结果汇总")
|
||||
print("="*70)
|
||||
|
||||
total_passed = 0
|
||||
print("=" * 70)
|
||||
passed = 0
|
||||
for name, code in results:
|
||||
status = "✓ 通过" if code == 0 else "✗ 失败"
|
||||
print(f"{name}: {status}")
|
||||
if code == 0:
|
||||
total_passed += 1
|
||||
|
||||
print("\n" + "-"*70)
|
||||
print(f"总计: {total_passed}/{len(results)} 测试套件通过")
|
||||
print("="*70)
|
||||
|
||||
return 0 if all(code == 0 for _, code in results) else 1
|
||||
ok = code == 0
|
||||
passed += int(ok)
|
||||
print(f"{name}: {'✓ 通过' if ok else '✗ 失败'}")
|
||||
print("-" * 70)
|
||||
print(f"总计: {passed}/{len(results)} 通过")
|
||||
return 0 if passed == len(results) else 1
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='TTS/ASR测试运行器',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
示例:
|
||||
# 运行单元测试
|
||||
python backend/tests/run_tests.py unit
|
||||
|
||||
# 运行集成测试
|
||||
python backend/tests/run_tests.py integration
|
||||
|
||||
# 运行macOS模拟测试
|
||||
python backend/tests/run_tests.py simulate
|
||||
|
||||
# 运行所有测试
|
||||
python backend/tests/run_tests.py all
|
||||
|
||||
# 运行特定集成测试
|
||||
python backend/tests/run_tests.py integration --test config
|
||||
|
||||
# 运行特定模拟测试
|
||||
python backend/tests/run_tests.py simulate --test device
|
||||
"""
|
||||
)
|
||||
|
||||
subparsers = parser.add_subparsers(dest='command', help='测试类型')
|
||||
|
||||
# 单元测试
|
||||
unit_parser = subparsers.add_parser('unit', help='运行单元测试')
|
||||
unit_parser.add_argument('-v', '--verbose', action='store_true', help='详细输出')
|
||||
|
||||
# 集成测试
|
||||
integration_parser = subparsers.add_parser('integration', help='运行集成测试')
|
||||
integration_parser.add_argument('--test', choices=[
|
||||
'config', 'status', 'warmup', 'tts', 'asr', 'perf'
|
||||
], help='运行特定测试')
|
||||
integration_parser.add_argument('--url', default='http://localhost:8001', help='API URL')
|
||||
integration_parser.add_argument('--key', default='your-secret-key-here', help='API密钥')
|
||||
|
||||
# macOS模拟测试
|
||||
simulate_parser = subparsers.add_parser('simulate', help='运行macOS模拟测试')
|
||||
simulate_parser.add_argument('--test', choices=[
|
||||
'device', 'memory', 'model', 'audio', 'env', 'full'
|
||||
], help='运行特定测试')
|
||||
|
||||
# 所有测试
|
||||
all_parser = subparsers.add_parser('all', help='运行所有测试')
|
||||
all_parser.add_argument('--url', default='http://localhost:8001', help='API URL')
|
||||
all_parser.add_argument('--key', default='your-secret-key-here', help='API密钥')
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="当前 API 化 TTS/ASR 测试运行器")
|
||||
subparsers = parser.add_subparsers(dest="command", help="测试类型")
|
||||
|
||||
unit_parser = subparsers.add_parser("unit", help="运行当前 TTS/ASR 单元测试")
|
||||
unit_parser.add_argument("-v", "--verbose", action="store_true", help="详细输出")
|
||||
|
||||
benchmark_parser = subparsers.add_parser("benchmark", help="运行当前 TTS/ASR benchmark")
|
||||
benchmark_parser.add_argument("benchmark_args", nargs="*", help="透传给 benchmark_tts_asr.py")
|
||||
|
||||
all_parser = subparsers.add_parser("all", help="运行当前 TTS/ASR 单元测试和 benchmark")
|
||||
all_parser.add_argument("-v", "--verbose", action="store_true", help="详细输出")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# 确保在项目根目录
|
||||
project_root = Path(__file__).parent.parent.parent
|
||||
os.chdir(project_root)
|
||||
|
||||
if args.command == 'unit':
|
||||
|
||||
if args.command == "unit":
|
||||
return run_unit_tests(verbose=args.verbose)
|
||||
|
||||
elif args.command == 'integration':
|
||||
return run_integration_tests(
|
||||
test_type=args.test,
|
||||
url=args.url,
|
||||
key=args.key
|
||||
)
|
||||
|
||||
elif args.command == 'simulate':
|
||||
return run_simulation(test_type=args.test)
|
||||
|
||||
elif args.command == 'all':
|
||||
return run_all_tests(url=args.url, key=args.key)
|
||||
|
||||
else:
|
||||
parser.print_help()
|
||||
return 0
|
||||
if args.command == "benchmark":
|
||||
return run_benchmark(extra_args=args.benchmark_args)
|
||||
if args.command == "all":
|
||||
return run_all(verbose=args.verbose)
|
||||
|
||||
parser.print_help()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
@@ -1,504 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
macOS环境模拟测试工具
|
||||
在非macOS环境下模拟Apple Silicon环境进行测试
|
||||
|
||||
运行方式:
|
||||
python backend/tests/simulate_macos.py --help
|
||||
python backend/tests/simulate_macos.py --device mps
|
||||
python backend/tests/simulate_macos.py --apple-silicon
|
||||
python backend/tests/simulate_macos.py --full-simulation
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import platform
|
||||
import sys
|
||||
from unittest.mock import patch
|
||||
import numpy as np
|
||||
|
||||
|
||||
class MacOSSimulator:
|
||||
"""macOS环境模拟器"""
|
||||
|
||||
def __init__(self):
|
||||
self.original_platform_system = platform.system
|
||||
self.original_platform_machine = platform.machine
|
||||
self.patches = []
|
||||
|
||||
def simulate_apple_silicon(self):
|
||||
"""模拟Apple Silicon环境"""
|
||||
print("\n" + "="*70)
|
||||
print("模拟 Apple Silicon 环境")
|
||||
print("="*70)
|
||||
|
||||
# 模拟Darwin系统和arm64架构
|
||||
self.patches.append(patch('platform.system', return_value='Darwin'))
|
||||
self.patches.append(patch('platform.machine', return_value='arm64'))
|
||||
|
||||
for p in self.patches:
|
||||
p.start()
|
||||
|
||||
print("✓ 平台: Darwin (macOS)")
|
||||
print("✓ 架构: arm64 (Apple Silicon)")
|
||||
|
||||
def simulate_mps_device(self):
|
||||
"""模拟MPS设备可用"""
|
||||
print("\n" + "="*70)
|
||||
print("模拟 MPS 设备")
|
||||
print("="*70)
|
||||
|
||||
# 创建模拟的torch.backends.mps
|
||||
mock_mps = type('MockMPS', (), {
|
||||
'is_available': lambda: True,
|
||||
'is_built': lambda: True,
|
||||
'empty_cache': lambda: None
|
||||
})()
|
||||
|
||||
mock_backends = type('MockBackends', (), {
|
||||
'mps': mock_mps
|
||||
})()
|
||||
|
||||
# 模拟torch模块
|
||||
mock_torch = type('MockTorch', (), {
|
||||
'backends': mock_backends,
|
||||
'mps': mock_mps,
|
||||
'randn': lambda *args, **kwargs: np.random.randn(*args),
|
||||
'mm': lambda a, b: np.dot(a, b),
|
||||
'empty_cache': lambda: None
|
||||
})()
|
||||
|
||||
self.patches.append(patch('torch', mock_torch))
|
||||
self.patches.append(patch('torch.backends.mps.is_available', return_value=True))
|
||||
self.patches.append(patch('torch.backends.mps.is_built', return_value=True))
|
||||
|
||||
for p in self.patches[-3:]:
|
||||
p.start()
|
||||
|
||||
print("✓ MPS 可用: True")
|
||||
print("✓ MPS 已编译: True")
|
||||
|
||||
def simulate_cuda_device(self):
|
||||
"""模拟CUDA设备可用"""
|
||||
print("\n" + "="*70)
|
||||
print("模拟 CUDA 设备")
|
||||
print("="*70)
|
||||
|
||||
mock_cuda = type('MockCUDA', (), {
|
||||
'is_available': lambda: True,
|
||||
'device_count': lambda: 1,
|
||||
'get_device_properties': lambda n: type('Props', (), {'total_memory': 8*1024*1024*1024})(),
|
||||
'empty_cache': lambda: None
|
||||
})()
|
||||
|
||||
self.patches.append(patch('torch.cuda', mock_cuda))
|
||||
self.patches.append(patch('torch.cuda.is_available', return_value=True))
|
||||
|
||||
for p in self.patches[-2:]:
|
||||
p.start()
|
||||
|
||||
print("✓ CUDA 可用: True")
|
||||
print("✓ GPU 数量: 1")
|
||||
print("✓ 显存: 8 GB")
|
||||
|
||||
def cleanup(self):
|
||||
"""清理所有补丁"""
|
||||
for p in self.patches:
|
||||
p.stop()
|
||||
self.patches.clear()
|
||||
print("\n✓ 已清理模拟环境")
|
||||
|
||||
|
||||
def test_device_detection_on_apple_silicon():
|
||||
"""测试Apple Silicon设备检测"""
|
||||
print("\n测试1: Apple Silicon 设备检测")
|
||||
print("-"*70)
|
||||
|
||||
simulator = MacOSSimulator()
|
||||
try:
|
||||
simulator.simulate_apple_silicon()
|
||||
simulator.simulate_mps_device()
|
||||
|
||||
# 设置环境变量
|
||||
os.environ['TTS_ASR_DEVICE'] = 'auto'
|
||||
os.environ['TTS_ASR_MODEL_SIZE'] = 'auto'
|
||||
|
||||
# 重新导入模块以应用模拟
|
||||
if 'backend.tts_asr' in sys.modules:
|
||||
del sys.modules['backend.tts_asr']
|
||||
|
||||
from backend.tts_asr import (
|
||||
_is_apple_silicon,
|
||||
_detect_device_capabilities,
|
||||
_get_recommended_model_size
|
||||
)
|
||||
|
||||
# 测试Apple Silicon检测
|
||||
assert _is_apple_silicon(), "应该检测到Apple Silicon"
|
||||
print("✓ Apple Silicon 检测: 通过")
|
||||
|
||||
# 测试设备能力检测
|
||||
caps = _detect_device_capabilities()
|
||||
print(f"✓ 设备: {caps.device}")
|
||||
print(f"✓ MPS 可用: {caps.mps_available}")
|
||||
print(f"✓ 推荐模型大小: {caps.recommended_model_size}")
|
||||
|
||||
# 测试模型大小推荐
|
||||
recommended_size = _get_recommended_model_size()
|
||||
assert recommended_size in ['small', 'tiny', 'base'], \
|
||||
f"Apple Silicon应推荐小模型，但推荐了 {recommended_size}"
|
||||
print(f"✓ 推荐模型大小: {recommended_size}")
|
||||
|
||||
print("\n✓ 测试通过")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n✗ 测试失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
finally:
|
||||
simulator.cleanup()
|
||||
|
||||
|
||||
def test_memory_management():
|
||||
"""测试内存管理"""
|
||||
print("\n测试2: 内存管理")
|
||||
print("-"*70)
|
||||
|
||||
simulator = MacOSSimulator()
|
||||
try:
|
||||
simulator.simulate_apple_silicon()
|
||||
simulator.simulate_mps_device()
|
||||
|
||||
# 模拟系统内存
|
||||
import psutil
|
||||
original_virtual_memory = psutil.virtual_memory
|
||||
|
||||
def mock_virtual_memory():
|
||||
mock_mem = type('MockMemory', (), {
|
||||
'total': 16 * 1024 * 1024 * 1024 # 16GB
|
||||
})()
|
||||
return mock_mem
|
||||
|
||||
self.patches.append(patch('psutil.virtual_memory', mock_virtual_memory))
|
||||
|
||||
from backend.tts_asr import _get_system_memory_mb, TTS_ASR_MPS_MEMORY_LIMIT_MB
|
||||
|
||||
mem_mb = _get_system_memory_mb()
|
||||
print(f"✓ 系统内存: {mem_mb} MB")
|
||||
|
||||
# 计算预期的MPS内存限制（60%）
|
||||
expected_limit = int(mem_mb * 0.6)
|
||||
print(f"✓ 预期MPS限制: {expected_limit} MB (60%)")
|
||||
print(f"✓ 配置MPS限制: {TTS_ASR_MPS_MEMORY_LIMIT_MB} MB")
|
||||
|
||||
print("\n✓ 测试通过")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n✗ 测试失败: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
finally:
|
||||
simulator.cleanup()
|
||||
|
||||
|
||||
def test_model_size_selection():
|
||||
"""测试模型大小选择"""
|
||||
print("\n测试3: 模型大小选择")
|
||||
print("-"*70)
|
||||
|
||||
test_cases = [
|
||||
('auto', 'Apple Silicon默认'),
|
||||
('tiny', '最小模型'),
|
||||
('small', '推荐模型'),
|
||||
('medium', '中等模型'),
|
||||
('large', '大模型'),
|
||||
('turbo', 'turbo模型'),
|
||||
]
|
||||
|
||||
from backend.tts_asr import WHISPER_MODEL_SIZES, _get_recommended_model_size
|
||||
|
||||
for size, desc in test_cases:
|
||||
os.environ['TTS_ASR_MODEL_SIZE'] = size
|
||||
|
||||
# 重新加载模块
|
||||
if 'backend.tts_asr' in sys.modules:
|
||||
del sys.modules['backend.tts_asr']
|
||||
|
||||
from backend.tts_asr import _get_recommended_model_size
|
||||
|
||||
if size == 'auto':
|
||||
# 自动选择
|
||||
recommended = _get_recommended_model_size()
|
||||
print(f"✓ {desc}: {recommended}")
|
||||
else:
|
||||
# 显式选择
|
||||
os.environ['TTS_ASR_MODEL_SIZE'] = size
|
||||
result = _get_recommended_model_size()
|
||||
assert result == size, f"应该返回 {size}，但返回了 {result}"
|
||||
print(f"✓ {desc}: {size} -> {WHISPER_MODEL_SIZES[size]}")
|
||||
|
||||
print("\n✓ 测试通过")
|
||||
return True
|
||||
|
||||
|
||||
def test_audio_processing():
|
||||
"""测试音频处理"""
|
||||
print("\n测试4: 音频处理")
|
||||
print("-"*70)
|
||||
|
||||
from backend.tts_asr import (
|
||||
_validate_audio_data,
|
||||
_resample_audio_robust
|
||||
)
|
||||
|
||||
# 测试音频验证
|
||||
test_cases = [
|
||||
(b'', False, "空数据"),
|
||||
(b'short', False, "太短"),
|
||||
(b'RIFF' + b'\x00' * 40, True, "有效WAV头"),
|
||||
]
|
||||
|
||||
for data, expected, desc in test_cases:
|
||||
result = _validate_audio_data(data)
|
||||
assert result == expected, f"{desc}: 预期 {expected}，得到 {result}"
|
||||
print(f"✓ 音频验证 ({desc}): {'通过' if result == expected else '失败'}")
|
||||
|
||||
# 测试重采样
|
||||
audio_16k = np.sin(np.linspace(0, 2*np.pi, 16000)).astype(np.float32)
|
||||
|
||||
# 16k -> 48k
|
||||
audio_48k = _resample_audio_robust(audio_16k, 16000, 48000)
|
||||
assert len(audio_48k) == 48000, f"48kHz音频长度错误: {len(audio_48k)}"
|
||||
print(f"✓ 重采样 (16k -> 48k): 长度 {len(audio_16k)} -> {len(audio_48k)}")
|
||||
|
||||
# 48k -> 16k
|
||||
audio_back = _resample_audio_robust(audio_48k, 48000, 16000)
|
||||
assert len(audio_back) == 16000, f"16kHz音频长度错误: {len(audio_back)}"
|
||||
print(f"✓ 重采样 (48k -> 16k): 长度 {len(audio_48k)} -> {len(audio_back)}")
|
||||
|
||||
print("\n✓ 测试通过")
|
||||
return True
|
||||
|
||||
|
||||
def test_environment_variables():
|
||||
"""测试环境变量"""
|
||||
print("\n测试5: 环境变量配置")
|
||||
print("-"*70)
|
||||
|
||||
# 清理环境变量
|
||||
env_vars = [
|
||||
'TTS_ASR_DEVICE', 'TTS_ASR_MODEL_SIZE', 'TTS_ASR_QUANTIZE',
|
||||
'TTS_ASR_OFFLINE_MODE', 'TTS_ASR_WARMUP', 'TTS_ASR_WARMUP_TIMEOUT',
|
||||
'TTS_ASR_IDLE_TIMEOUT', 'TTS_ASR_MPS_MEMORY_LIMIT_MB'
|
||||
]
|
||||
|
||||
original_values = {}
|
||||
for var in env_vars:
|
||||
original_values[var] = os.environ.get(var)
|
||||
if var in os.environ:
|
||||
del os.environ[var]
|
||||
|
||||
try:
|
||||
# 测试默认值
|
||||
from backend.tts_asr import (
|
||||
TTS_ASR_DEVICE, TTS_ASR_MODEL_SIZE, TTS_ASR_QUANTIZE,
|
||||
TTS_ASR_OFFLINE_MODE, TTS_ASR_WARMUP, TTS_ASR_WARMUP_TIMEOUT,
|
||||
TTS_ASR_IDLE_TIMEOUT, TTS_ASR_MPS_MEMORY_LIMIT_MB
|
||||
)
|
||||
|
||||
defaults = {
|
||||
'TTS_ASR_DEVICE': 'auto',
|
||||
'TTS_ASR_MODEL_SIZE': 'auto',
|
||||
'TTS_ASR_QUANTIZE': False,
|
||||
'TTS_ASR_OFFLINE_MODE': False,
|
||||
'TTS_ASR_WARMUP': True,
|
||||
'TTS_ASR_WARMUP_TIMEOUT': 120,
|
||||
'TTS_ASR_IDLE_TIMEOUT': 0,
|
||||
'TTS_ASR_MPS_MEMORY_LIMIT_MB': 8192,
|
||||
}
|
||||
|
||||
for var, expected in defaults.items():
|
||||
actual = locals()[var]
|
||||
assert actual == expected, f"{var}: 预期 {expected}，得到 {actual}"
|
||||
print(f"✓ {var} = {actual}")
|
||||
|
||||
# 测试自定义值
|
||||
print("\n自定义配置测试:")
|
||||
os.environ['TTS_ASR_MODEL_SIZE'] = 'small'
|
||||
os.environ['TTS_ASR_QUANTIZE'] = 'true'
|
||||
os.environ['TTS_ASR_OFFLINE_MODE'] = 'true'
|
||||
os.environ['TTS_ASR_MPS_MEMORY_LIMIT_MB'] = '4096'
|
||||
|
||||
# 重新加载
|
||||
if 'backend.tts_asr' in sys.modules:
|
||||
del sys.modules['backend.tts_asr']
|
||||
|
||||
from backend.tts_asr import (
|
||||
TTS_ASR_MODEL_SIZE, TTS_ASR_QUANTIZE,
|
||||
TTS_ASR_OFFLINE_MODE, TTS_ASR_MPS_MEMORY_LIMIT_MB
|
||||
)
|
||||
|
||||
assert TTS_ASR_MODEL_SIZE == 'small'
|
||||
assert TTS_ASR_QUANTIZE == True
|
||||
assert TTS_ASR_OFFLINE_MODE == True
|
||||
assert TTS_ASR_MPS_MEMORY_LIMIT_MB == 4096
|
||||
|
||||
print(f"✓ TTS_ASR_MODEL_SIZE = {TTS_ASR_MODEL_SIZE}")
|
||||
print(f"✓ TTS_ASR_QUANTIZE = {TTS_ASR_QUANTIZE}")
|
||||
print(f"✓ TTS_ASR_OFFLINE_MODE = {TTS_ASR_OFFLINE_MODE}")
|
||||
print(f"✓ TTS_ASR_MPS_MEMORY_LIMIT_MB = {TTS_ASR_MPS_MEMORY_LIMIT_MB}")
|
||||
|
||||
print("\n✓ 测试通过")
|
||||
return True
|
||||
|
||||
finally:
|
||||
# 恢复原始值
|
||||
for var, value in original_values.items():
|
||||
if value is not None:
|
||||
os.environ[var] = value
|
||||
elif var in os.environ:
|
||||
del os.environ[var]
|
||||
|
||||
|
||||
def run_full_simulation():
|
||||
"""运行完整模拟测试"""
|
||||
print("\n" + "="*70)
|
||||
print("完整macOS环境模拟测试")
|
||||
print("="*70)
|
||||
|
||||
results = []
|
||||
|
||||
# 运行所有测试
|
||||
results.append(("设备检测", test_device_detection_on_apple_silicon()))
|
||||
results.append(("内存管理", test_memory_management()))
|
||||
results.append(("模型选择", test_model_size_selection()))
|
||||
results.append(("音频处理", test_audio_processing()))
|
||||
results.append(("环境变量", test_environment_variables()))
|
||||
|
||||
# 汇总结果
|
||||
print("\n" + "="*70)
|
||||
print("测试结果汇总")
|
||||
print("="*70)
|
||||
|
||||
for name, passed in results:
|
||||
status = "✓ 通过" if passed else "✗ 失败"
|
||||
print(f"{name}: {status}")
|
||||
|
||||
total = len(results)
|
||||
passed = sum(1 for _, p in results if p)
|
||||
|
||||
print("\n" + "-"*70)
|
||||
print(f"总计: {passed}/{total} 测试通过")
|
||||
print("="*70)
|
||||
|
||||
return all(p for _, p in results)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='macOS环境模拟测试工具',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
示例:
|
||||
# 运行完整模拟测试
|
||||
python backend/tests/simulate_macos.py --full-simulation
|
||||
|
||||
# 仅模拟Apple Silicon环境
|
||||
python backend/tests/simulate_macos.py --apple-silicon
|
||||
|
||||
# 仅模拟MPS设备
|
||||
python backend/tests/simulate_macos.py --device mps
|
||||
|
||||
# 仅模拟CUDA设备
|
||||
python backend/tests/simulate_macos.py --device cuda
|
||||
"""
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
'--full-simulation',
|
||||
action='store_true',
|
||||
help='运行完整模拟测试'
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
'--apple-silicon',
|
||||
action='store_true',
|
||||
help='模拟Apple Silicon环境'
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
'--device',
|
||||
choices=['mps', 'cuda'],
|
||||
help='模拟特定设备'
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
'--test',
|
||||
choices=['device', 'memory', 'model', 'audio', 'env'],
|
||||
help='运行特定测试'
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# 确保可以导入backend模块
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..'))
|
||||
|
||||
if args.full_simulation:
|
||||
success = run_full_simulation()
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
if args.apple_silicon:
|
||||
simulator = MacOSSimulator()
|
||||
try:
|
||||
simulator.simulate_apple_silicon()
|
||||
simulator.simulate_mps_device()
|
||||
|
||||
print("\n环境已模拟，按Ctrl+D退出")
|
||||
print("在Python环境中可以使用:")
|
||||
print(" from backend.tts_asr import _is_apple_silicon")
|
||||
print(" print(_is_apple_silicon()) # 应该返回 True")
|
||||
|
||||
# 进入交互模式
|
||||
import code
|
||||
code.interact(local=locals())
|
||||
finally:
|
||||
simulator.cleanup()
|
||||
|
||||
if args.device:
|
||||
simulator = MacOSSimulator()
|
||||
try:
|
||||
if args.device == 'mps':
|
||||
simulator.simulate_mps_device()
|
||||
elif args.device == 'cuda':
|
||||
simulator.simulate_cuda_device()
|
||||
|
||||
print("\n设备已模拟")
|
||||
import code
|
||||
code.interact(local=locals())
|
||||
finally:
|
||||
simulator.cleanup()
|
||||
|
||||
if args.test:
|
||||
test_func = {
|
||||
'device': test_device_detection_on_apple_silicon,
|
||||
'memory': test_memory_management,
|
||||
'model': test_model_size_selection,
|
||||
'audio': test_audio_processing,
|
||||
'env': test_environment_variables,
|
||||
}
|
||||
|
||||
success = test_func[args.test]()
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
# 默认运行完整测试
|
||||
if not any([args.full_simulation, args.apple_silicon, args.device, args.test]):
|
||||
parser.print_help()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,75 @@
|
||||
"""Regression tests for PostgreSQL audit persistence."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
BACKEND_DIR = Path(__file__).resolve().parents[1]
|
||||
if str(BACKEND_DIR) not in __import__("sys").path:
|
||||
__import__("sys").path.insert(0, str(BACKEND_DIR))
|
||||
|
||||
import audit_store # noqa: E402
|
||||
from audit_store import PostgresAuditStore # noqa: E402
|
||||
|
||||
|
||||
class _RecordingCursor:
|
||||
def __init__(self) -> None:
|
||||
self.query = ""
|
||||
self.params = ()
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
def execute(self, query: str, params=()) -> None:
|
||||
self.query = query
|
||||
self.params = params or ()
|
||||
assert query.count("%s") == len(self.params)
|
||||
|
||||
|
||||
class _RecordingConnection:
|
||||
def __init__(self, cursor: _RecordingCursor) -> None:
|
||||
self._cursor = cursor
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc, tb):
|
||||
return False
|
||||
|
||||
def cursor(self) -> _RecordingCursor:
|
||||
return self._cursor
|
||||
|
||||
|
||||
def test_record_llm_call_keeps_columns_placeholders_and_params_aligned(monkeypatch):
|
||||
cursor = _RecordingCursor()
|
||||
monkeypatch.setattr(audit_store, "psycopg", object())
|
||||
store = PostgresAuditStore("postgresql://unused")
|
||||
store._initialized = True
|
||||
monkeypatch.setattr(store, "_connect", lambda: _RecordingConnection(cursor))
|
||||
|
||||
store.record_llm_call({
|
||||
"request_id": "request-1",
|
||||
"session_hash": "session",
|
||||
"ip_hash": "ip",
|
||||
"job_type": "ocr",
|
||||
"model": "vision-model",
|
||||
"estimated_input_tokens": 12,
|
||||
"max_output_tokens": 256,
|
||||
"estimated_cost": 0.01,
|
||||
"actual_output_chars": 42,
|
||||
"actual_cost": 0.02,
|
||||
"queue_ms": 10,
|
||||
"run_ms": 20,
|
||||
"total_ms": 30,
|
||||
"status": "completed",
|
||||
"error_code": "",
|
||||
"metadata": {"source": "test"},
|
||||
})
|
||||
|
||||
assert "INSERT INTO llm_call_audit" in cursor.query
|
||||
assert cursor.query.count("%s") == 16
|
||||
assert len(cursor.params) == 16
|
||||
@@ -90,6 +90,72 @@ def test_cancel_endpoint_cancels_running_task(monkeypatch):
|
||||
assert "event: cancelled" in response_box["body"]
|
||||
|
||||
|
||||
class FakeRedis:
|
||||
def __init__(self):
|
||||
self.acks = []
|
||||
|
||||
async def xack(self, *args):
|
||||
self.acks.append(args)
|
||||
|
||||
async def hincrby(self, key, field, amount):
|
||||
return 0
|
||||
|
||||
|
||||
class FakeManager:
|
||||
def __init__(self):
|
||||
self.redis = FakeRedis()
|
||||
self.statuses = {}
|
||||
|
||||
async def get_status(self, job_id):
|
||||
return self.statuses.get(job_id)
|
||||
|
||||
async def _set_state(self, job_id, state):
|
||||
self.statuses[job_id] = state
|
||||
|
||||
async def _metrics(self, job_type):
|
||||
return {"queued_count": 0, "running_count": 0}
|
||||
|
||||
async def _emit_event(self, job_id, event, data):
|
||||
self.statuses[job_id]["event"] = event
|
||||
|
||||
def _metrics_key(self, job_type):
|
||||
return f"metrics:{job_type}"
|
||||
|
||||
def _state_key(self, job_id):
|
||||
return f"state:{job_id}"
|
||||
|
||||
|
||||
async def _run_cancelled_after_handler(manager, job_type):
|
||||
worker = job_system.RedisWorker(manager)
|
||||
await worker._run_message(
|
||||
job_type,
|
||||
"queue",
|
||||
"group",
|
||||
"msg-1",
|
||||
{"job_id": "job-1"},
|
||||
asyncio.Semaphore(1),
|
||||
)
|
||||
|
||||
|
||||
def test_redis_worker_acks_when_handler_returns_cancelled_state():
|
||||
async def handler(payload, emit, is_cancelled):
|
||||
return {"ok": True}
|
||||
|
||||
async def coro():
|
||||
manager = FakeManager()
|
||||
manager.handlers = {"completion": handler}
|
||||
manager.statuses["job-1"] = {
|
||||
"request_id": "req-1",
|
||||
"type": "completion",
|
||||
"status": "running",
|
||||
"created_at": 1,
|
||||
}
|
||||
await _run_cancelled_after_handler(manager, "completion")
|
||||
assert manager.redis.acks == [("queue", "group", "msg-1")]
|
||||
|
||||
asyncio.run(coro())
|
||||
|
||||
|
||||
def test_cancel_not_found():
|
||||
with TestClient(main.app) as client:
|
||||
response = client.post(
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import base64
|
||||
import asyncio
|
||||
import base64
|
||||
import importlib
|
||||
import os
|
||||
import sys
|
||||
@@ -209,6 +210,16 @@ def test_post_convert_unsupported_extension_returns_500():
|
||||
assert "仅支持" in resp.json()["error"]
|
||||
|
||||
|
||||
def test_post_convert_rejects_mismatched_content_suffix():
|
||||
content = base64.b64encode(b"%PDF-1.4\n%%EOF").decode()
|
||||
with TestClient(main.app) as client:
|
||||
resp = client.post("/v1/convert", headers=HEADERS, json={
|
||||
"file": content, "filename": "sample.txt",
|
||||
})
|
||||
assert resp.status_code == 500
|
||||
assert "仅支持" in resp.json()["error"]
|
||||
|
||||
|
||||
def test_docs_nodes_crud_round_trip():
|
||||
with TestClient(main.app) as client:
|
||||
folder_resp = client.post("/v1/docs/folders", headers=HEADERS, json={
|
||||
|
||||
@@ -0,0 +1,334 @@
|
||||
"""Tests for the shared LLM speech adapter and speech job handlers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
BACKEND_DIR = Path(__file__).resolve().parents[1]
|
||||
if str(BACKEND_DIR) not in __import__("sys").path:
|
||||
__import__("sys").path.insert(0, str(BACKEND_DIR))
|
||||
|
||||
import job_handlers # noqa: E402
|
||||
import tts_asr # noqa: E402
|
||||
from audit_store import BaseAuditStore # noqa: E402
|
||||
|
||||
|
||||
def _wav_bytes(duration_ms: int = 100) -> bytes:
|
||||
sample_rate = 16000
|
||||
frames = max(1, int(sample_rate * duration_ms / 1000))
|
||||
data = b"".join((i % 32768).to_bytes(2, "little", signed=False) for i in range(frames))
|
||||
data_size = len(data)
|
||||
return (
|
||||
b"RIFF" + (36 + data_size).to_bytes(4, "little")
|
||||
+ b"WAVE"
|
||||
+ b"fmt " + (16).to_bytes(4, "little")
|
||||
+ (1).to_bytes(2, "little")
|
||||
+ (1).to_bytes(2, "little")
|
||||
+ sample_rate.to_bytes(4, "little")
|
||||
+ sample_rate.to_bytes(4, "little")
|
||||
+ (2).to_bytes(2, "little")
|
||||
+ (16).to_bytes(2, "little")
|
||||
+ b"data" + data_size.to_bytes(4, "little")
|
||||
+ data
|
||||
)
|
||||
|
||||
|
||||
def _run_async(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
class _CaptureAuditStore(BaseAuditStore):
|
||||
def __init__(self) -> None:
|
||||
self.llm_calls: list[dict] = []
|
||||
|
||||
def record_llm_call(self, payload: dict) -> None:
|
||||
self.llm_calls.append(payload)
|
||||
|
||||
|
||||
def test_tts_calls_shared_llm_speech_endpoint(monkeypatch):
|
||||
captured: dict[str, object] = {}
|
||||
monkeypatch.setattr(tts_asr, "LLM_API_KEY", "test-api-key")
|
||||
monkeypatch.setattr(tts_asr, "TTS_MODEL_ID", "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit")
|
||||
|
||||
def transport(request: httpx.Request):
|
||||
captured["url"] = str(request.url)
|
||||
captured["headers"] = dict(request.headers)
|
||||
captured["json"] = json.loads(request.read().decode("utf-8"))
|
||||
return httpx.Response(200, content=b"speech-ok", headers={"x-request-id": "tts-req-1"})
|
||||
|
||||
async def run():
|
||||
client = httpx.AsyncClient(
|
||||
base_url="https://speech.example/v1",
|
||||
transport=httpx.MockTransport(transport),
|
||||
)
|
||||
try:
|
||||
tts_asr._httpx_client = client
|
||||
return await tts_asr.generate_tts_response(
|
||||
"你好世界",
|
||||
instruct="A warm Mandarin voice.",
|
||||
speaker="Vivian",
|
||||
output_format="wav",
|
||||
)
|
||||
finally:
|
||||
await client.aclose()
|
||||
tts_asr._httpx_client = None
|
||||
|
||||
result = _run_async(run())
|
||||
|
||||
assert result["format"] == "wav"
|
||||
assert result["speaker"] == "Vivian"
|
||||
assert result["model"] == "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit"
|
||||
assert result["upstream_request_id"] == "tts-req-1"
|
||||
assert base64.b64decode(result["audio_base64"]) == b"speech-ok"
|
||||
assert captured["url"] == "https://speech.example/v1/audio/speech"
|
||||
assert captured["headers"]["authorization"] == "Bearer test-api-key"
|
||||
payload = captured["json"]
|
||||
assert payload["model"] == "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit"
|
||||
assert payload["voice"] == "Vivian"
|
||||
assert payload["input"] == "你好世界"
|
||||
assert payload["instructions"] == "A warm Mandarin voice."
|
||||
assert "instruction" not in payload
|
||||
|
||||
|
||||
def test_tts_uses_nonempty_default_instructions(monkeypatch):
|
||||
captured: dict[str, object] = {}
|
||||
|
||||
def transport(request: httpx.Request):
|
||||
captured["json"] = json.loads(request.read().decode("utf-8"))
|
||||
return httpx.Response(200, content=b"speech-ok")
|
||||
|
||||
async def run():
|
||||
client = httpx.AsyncClient(
|
||||
base_url="https://speech.example/v1",
|
||||
transport=httpx.MockTransport(transport),
|
||||
)
|
||||
try:
|
||||
tts_asr._httpx_client = client
|
||||
return await tts_asr.generate_tts_response("你好世界")
|
||||
finally:
|
||||
await client.aclose()
|
||||
tts_asr._httpx_client = None
|
||||
|
||||
_run_async(run())
|
||||
|
||||
payload = captured["json"]
|
||||
assert payload["instructions"] == tts_asr.DEFAULT_TTS_INSTRUCTIONS
|
||||
assert payload["instructions"].strip()
|
||||
|
||||
|
||||
def test_asr_calls_shared_llm_transcriptions_endpoint(monkeypatch):
|
||||
captured: dict[str, object] = {}
|
||||
monkeypatch.setattr(tts_asr, "LLM_API_KEY", "test-api-key")
|
||||
monkeypatch.setattr(tts_asr, "ASR_MODEL_ID", "Qwen3-ASR-0.6B-8bit")
|
||||
|
||||
def transport(request: httpx.Request):
|
||||
captured["url"] = str(request.url)
|
||||
captured["headers"] = dict(request.headers)
|
||||
captured["content"] = request.read()
|
||||
return httpx.Response(200, json={"text": "hello world", "language": "zh"}, headers={"x-request-id": "asr-req-1"})
|
||||
|
||||
async def run():
|
||||
client = httpx.AsyncClient(
|
||||
base_url="https://speech.example/v1",
|
||||
transport=httpx.MockTransport(transport),
|
||||
)
|
||||
try:
|
||||
tts_asr._httpx_client = client
|
||||
return await tts_asr.generate_asr_response(_wav_bytes(), language="zh-CN")
|
||||
finally:
|
||||
await client.aclose()
|
||||
tts_asr._httpx_client = None
|
||||
|
||||
result = _run_async(run())
|
||||
|
||||
assert result["text"] == "hello world"
|
||||
assert result["language"] == "zh"
|
||||
assert result["model"] == "Qwen3-ASR-0.6B-8bit"
|
||||
assert result["upstream_request_id"] == "asr-req-1"
|
||||
assert captured["url"] == "https://speech.example/v1/audio/transcriptions"
|
||||
assert captured["headers"]["authorization"] == "Bearer test-api-key"
|
||||
content = captured["content"]
|
||||
assert b'name="model"' in content
|
||||
assert b"Qwen3-ASR-0.6B-8bit" in content
|
||||
assert b'name="language"' in content
|
||||
assert b"zh" in content
|
||||
|
||||
|
||||
def test_invalid_tts_text_returns_http_exception():
|
||||
with pytest.raises(tts_asr.HTTPException) as exc:
|
||||
_run_async(tts_asr._call_tts_api("", speaker="Vivian"))
|
||||
assert exc.value.status_code == 400
|
||||
|
||||
|
||||
def test_invalid_asr_audio_returns_http_exception():
|
||||
with pytest.raises(tts_asr.HTTPException) as exc:
|
||||
_run_async(tts_asr._call_asr_api(b"", language="zh-CN"))
|
||||
assert exc.value.status_code == 400
|
||||
|
||||
|
||||
def test_status_config_routes(monkeypatch):
|
||||
app = FastAPI()
|
||||
app.include_router(tts_asr.meta_router)
|
||||
monkeypatch.setattr(tts_asr, "LLM_BASE_URL", "https://speech.example/v1")
|
||||
monkeypatch.setattr(tts_asr, "LLM_API_KEY", "")
|
||||
monkeypatch.setattr(tts_asr, "TTS_MODEL_ID", "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit")
|
||||
monkeypatch.setattr(tts_asr, "ASR_MODEL_ID", "Qwen3-ASR-0.6B-8bit")
|
||||
|
||||
with TestClient(app) as client:
|
||||
status = client.get("/status")
|
||||
config = client.get("/config")
|
||||
|
||||
assert status.status_code == 200
|
||||
assert config.status_code == 200
|
||||
assert status.json()["llm_url"] == "https://speech.example/v1"
|
||||
assert status.json()["tts_model"] == "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit"
|
||||
assert status.json()["asr_model"] == "Qwen3-ASR-0.6B-8bit"
|
||||
assert status.json()["status"]["api_key_configured"] is False
|
||||
assert status.json()["status"]["max_connections"] == tts_asr.SPEECH_MAX_CONNECTIONS
|
||||
|
||||
|
||||
def test_tts_concurrent_requests_respect_connection_limit(monkeypatch):
|
||||
monkeypatch.setattr(tts_asr, "SPEECH_MAX_CONNECTIONS", 4)
|
||||
monkeypatch.setattr(tts_asr, "SPEECH_MAX_KEEPALIVE_CONNECTIONS", 1)
|
||||
|
||||
class LimitedClient:
|
||||
def __init__(self):
|
||||
self.semaphore = asyncio.Semaphore(4)
|
||||
self.active = 0
|
||||
self.max_active = 0
|
||||
|
||||
async def post(self, url: str, **kwargs):
|
||||
async with self.semaphore:
|
||||
self.active += 1
|
||||
self.max_active = max(self.max_active, self.active)
|
||||
await asyncio.sleep(0.01)
|
||||
self.active -= 1
|
||||
return httpx.Response(200, content=b"speech-ok", request=httpx.Request("POST", f"https://speech.example{url}"))
|
||||
|
||||
async def run():
|
||||
client = LimitedClient()
|
||||
|
||||
async def get_client():
|
||||
return client
|
||||
|
||||
monkeypatch.setattr(tts_asr, "_get_speech_client", get_client)
|
||||
await asyncio.gather(*(tts_asr.generate_tts_response(f"文本 {index}") for index in range(20)))
|
||||
return client
|
||||
|
||||
client = _run_async(run())
|
||||
assert client.max_active <= 4
|
||||
|
||||
|
||||
def test_tts_asr_handlers_record_audit(monkeypatch):
|
||||
audit_store = _CaptureAuditStore()
|
||||
|
||||
async def fake_tts(*args, **kwargs):
|
||||
return {
|
||||
"audio_base64": base64.b64encode(b"ok").decode("utf-8"),
|
||||
"format": "wav",
|
||||
"duration_ms": 1200,
|
||||
"audio_bytes": 2,
|
||||
"text_chars": 2,
|
||||
"speaker": "Vivian",
|
||||
"model": "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit",
|
||||
"request_ms": 45,
|
||||
"upstream_request_id": "tts-upstream",
|
||||
}
|
||||
|
||||
async def fake_asr(*args, **kwargs):
|
||||
return {
|
||||
"text": "hello world",
|
||||
"language": "zh",
|
||||
"audio_bytes": len(_wav_bytes()),
|
||||
"model": "Qwen3-ASR-0.6B-8bit",
|
||||
"request_ms": 80,
|
||||
"upstream_request_id": "asr-upstream",
|
||||
}
|
||||
|
||||
monkeypatch.setattr(job_handlers, "generate_tts_response", fake_tts)
|
||||
monkeypatch.setattr(job_handlers, "generate_asr_response", fake_asr)
|
||||
monkeypatch.setattr(job_handlers, "get_audit_store", lambda *_args, **_kwargs: audit_store)
|
||||
|
||||
base_payload = {
|
||||
"request_id": "req-1",
|
||||
"risk": {
|
||||
"request_id": "req-1",
|
||||
"session_hash": "session",
|
||||
"ip_hash": "ip",
|
||||
"estimated_input_tokens": 12,
|
||||
"estimated_cost": 0.0,
|
||||
"policy": {
|
||||
"job_type": "tts",
|
||||
"model": "Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit",
|
||||
"profile": "speech_tts",
|
||||
"max_output_tokens": 0,
|
||||
},
|
||||
},
|
||||
"job_context": {
|
||||
"created_at": 1000,
|
||||
"started_at": 1200,
|
||||
"queue_ms": 200,
|
||||
},
|
||||
}
|
||||
|
||||
async def run():
|
||||
events = []
|
||||
|
||||
async def emit(event: str, data: dict):
|
||||
events.append((event, data))
|
||||
|
||||
tts_payload = {
|
||||
**base_payload,
|
||||
"text": "你好",
|
||||
"speaker": "Vivian",
|
||||
"format": "wav",
|
||||
}
|
||||
await job_handlers.tts_handler(tts_payload, emit, lambda: False)
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as handle:
|
||||
handle.write(_wav_bytes())
|
||||
audio_path = handle.name
|
||||
|
||||
try:
|
||||
asr_payload = {
|
||||
**base_payload,
|
||||
"risk": {
|
||||
**base_payload["risk"],
|
||||
"policy": {
|
||||
"job_type": "asr",
|
||||
"model": "Qwen3-ASR-0.6B-8bit",
|
||||
"profile": "speech_asr",
|
||||
"max_output_tokens": 0,
|
||||
},
|
||||
},
|
||||
"input_path": audio_path,
|
||||
"language": "zh-CN",
|
||||
}
|
||||
await job_handlers.asr_handler(asr_payload, emit, lambda: False)
|
||||
finally:
|
||||
job_handlers._safe_unlink(audio_path)
|
||||
|
||||
return events
|
||||
|
||||
events = _run_async(run())
|
||||
|
||||
assert any(event == "result" for event, _data in events)
|
||||
assert len(audit_store.llm_calls) == 2
|
||||
tts_audit = audit_store.llm_calls[0]
|
||||
asr_audit = audit_store.llm_calls[1]
|
||||
assert tts_audit["job_type"] == "tts"
|
||||
assert tts_audit["queue_ms"] == 200
|
||||
assert tts_audit["metadata"]["duration_ms"] == 1200
|
||||
assert tts_audit["metadata"]["upstream_request_id"] == "tts-upstream"
|
||||
assert asr_audit["job_type"] == "asr"
|
||||
assert asr_audit["metadata"]["language"] == "zh"
|
||||
assert asr_audit["metadata"]["upstream_request_id"] == "asr-upstream"
|
||||
@@ -1,5 +1,6 @@
|
||||
import asyncio
|
||||
import importlib
|
||||
import socket
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
@@ -44,12 +45,26 @@ def _payload():
|
||||
}
|
||||
|
||||
|
||||
def test_is_blocked_public_url():
|
||||
def test_is_blocked_public_url(monkeypatch):
|
||||
def fake_getaddrinfo(host, port, type=0, flags=0): # noqa: ARG001
|
||||
del host, flags
|
||||
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("93.184.216.34", port, 0, 0))]
|
||||
|
||||
monkeypatch.setattr(job_handlers.socket, "getaddrinfo", fake_getaddrinfo)
|
||||
assert job_handlers._is_blocked_public_url("http://127.0.0.1/test") is True
|
||||
assert job_handlers._is_blocked_public_url("file:///tmp/test") is True
|
||||
assert job_handlers._is_blocked_public_url("https://example.com/docs") is False
|
||||
|
||||
|
||||
def test_is_blocked_public_url_resolves_private_hostname(monkeypatch):
|
||||
def fake_getaddrinfo(host, port, type=0, flags=0): # noqa: ARG001
|
||||
del host, flags
|
||||
return [(socket.AF_INET, socket.SOCK_STREAM, 6, "", ("127.0.0.1", port, 0, 0))]
|
||||
|
||||
monkeypatch.setattr(job_handlers.socket, "getaddrinfo", fake_getaddrinfo)
|
||||
assert job_handlers._is_blocked_public_url("https://private.example.com/docs") is True
|
||||
|
||||
|
||||
def test_web_search_route_returns_done(monkeypatch):
|
||||
async def fake_call_ollama(prompt, system_prompt=None, tag="", **kwargs): # noqa: ARG001
|
||||
if tag.endswith("-webq"):
|
||||
|
||||
Reference in New Issue
Block a user