2026-04-07 12:46:45 +08:00
|
|
|
# API Benchmarking Report (2026-04-06 13:45:31)
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
**Base URL:** `https://api.imageteach.tech:8002`
|
|
|
|
|
|
|
|
|
|
## Executive Summary
|
|
|
|
|
| Task | Success Rate | Avg TTFB | Avg Latency | P95 Latency | TPS | RPS |
|
|
|
|
|
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
2026-04-07 12:46:45 +08:00
|
|
|
| Completion-Short | 90.0% | 9123.6ms | 9123.8ms | 20222.9ms | 7.9 | 0.08 |
|
|
|
|
|
| Completion-Normal | 10.0% | 10559.3ms | 10559.6ms | 10559.6ms | 66.4 | 0.66 |
|
|
|
|
|
| Completion-Long | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 8.97 |
|
|
|
|
|
| OCR-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 6.75 |
|
|
|
|
|
| TTS-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 10.17 |
|
|
|
|
|
| ASR-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 13.02 |
|
|
|
|
|
| Convert-Concurrent | 0.0% | 0.0ms | 0.0ms | 0.0ms | 0.0 | 5.98 |
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
## Stability & Context Analysis
|
|
|
|
|
Detailed analysis of how context length affects TTFB and overall performance.
|
|
|
|
|
|
|
|
|
|
### Completion-Short Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 123.93s
|
|
|
|
|
- **Top Errors:**
|
|
|
|
|
- `[0]`
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
### Completion-Normal Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 15.21s
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
- **Top Errors:**
|
2026-04-07 12:46:45 +08:00
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
<body>
|
2026-04-07 12:46:45 +08:00
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
2026-04-07 12:46:45 +08:00
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
<body>
|
2026-04-07 12:46:45 +08:00
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
2026-04-07 12:46:45 +08:00
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
<body>
|
2026-04-07 12:46:45 +08:00
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Completion-Long Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 1.11s
|
|
|
|
|
- **Top Errors:**
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
### OCR-Concurrent Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 1.48s
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
- **Top Errors:**
|
2026-04-07 12:46:45 +08:00
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
### TTS-Concurrent Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 0.98s
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
- **Top Errors:**
|
2026-04-07 12:46:45 +08:00
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
### ASR-Concurrent Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 0.77s
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
- **Top Errors:**
|
2026-04-07 12:46:45 +08:00
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
feat(tts): add api endpoints and optimization for apple silicon
Introduce a comprehensive TTS/ASR module that:
- Adds /v1/tts-asr/config, /status, /warmup, /tts, /asr endpoints with detailed JSON responses
- Implements Apple‑Silicon detection, device selection (MPS/CUDA/CPU), and memory limiting logic
- Supports selectable model size, quantization, and offline mode via environment variables
- Adds robust audio validation and multi‑path resampling fallback
- Provides new README sections for API usage, device detection, and performance benchmarking
- Includes a full testing suite: unit tests, integration tests, macOS simulation and performance reports
- Updates backend dependencies and CI scripts
- Adds new front‑end views and components for Univer editor integration
All changes are backward compatible; new features are exposed through environment variables and new API routes.
2026-04-06 11:14:09 +08:00
|
|
|
|
|
|
|
|
### Convert-Concurrent Details
|
|
|
|
|
- **Total Samples:** 10
|
2026-04-07 12:46:45 +08:00
|
|
|
- **Duration:** 1.67s
|
|
|
|
|
- **Top Errors:**
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
- `[502]` <html>
|
|
|
|
|
<head><title>502 Bad Gateway</title></head>
|
|
|
|
|
<body>
|
|
|
|
|
<center><h1>502 Bad Gateway</h1></center>
|
|
|
|
|
<hr><center>openresty</center>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
