Fu DaiandClaude Opus 4.8
1a585693be
FastAPI service wrapping the Surya-OCR-2 model (datalab-to) served through vLLM: legacy /v1/api/ai/* endpoints, an OpenAI-compatible /v1/chat/completions endpoint, a coalescing request batcher, a local OCR CLI, Docker packaging, multilingual example outputs, and quantization/concurrency benchmarks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2.4 KiB
2.4 KiB
Single Call Timing Result
Test image: /path/to/suya-ocr-api/temp_image_c3c56948-282e-453b-8fac-6c482243d1e5.jpg
Endpoint: /v1/api/ai/suya_ocr_vllm/
Run time: 2026-06-10 16:18 Asia/Dubai
Environment:
SURYA_INFERENCE_URL=http://127.0.0.1:43401/v1SURYA_INFERENCE_PARALLEL=8SUYA_OCR_MODE=blockSUYA_TIMING_ENABLED=true
Summary:
| Stage | Time ms | Notes |
|---|---|---|
| Client observed request | 7812.80 | Single HTTP call to API |
| API request total | 7800.86 | FastAPI middleware duration |
| Base64/PIL decode | 33.38 | 683,466 byte JPEG, 1410x1033 |
| API wait for vLLM batcher | 7755.73 | Includes queue, layout, recognition |
| OCR batch total | 7730.19 | Surya vLLM OCR path |
| Layout predictor total | 2114.89 | One page layout call |
| Layout vLLM generate | 2053.21 | Includes backend attach/start check |
| vLLM backend attach/start check | 97.84 | Health/model lookup on first request |
| Layout chat completion | 1902.34 | One vLLM /chat/completions call |
| Layout parse/output boxes | 61.48 | JSON parse, bbox conversion, blank filtering |
| Recognition block total | 5615.26 | Block OCR after layout |
| Recognition build crop batch | 1.09 | Crop 17 OCR blocks, skip 1 block |
| Recognition vLLM generate | 5613.12 | 17 block calls, parallelism 8 |
| Recognition output assembly | 0.28 | Clean HTML and build page result |
| Response assembly | 0.01 | No bbox image drawing |
Block OCR details:
| Metric | Value |
|---|---|
| Layout blocks returned in response | 18 |
| OCR block requests sent to vLLM | 17 |
| Skipped blocks | 1 |
| Sum of requested block max tokens | 6210 |
| Generated block tokens | 4454 |
| Sum of block chat completion times | 33259.06 ms |
| Mean block chat completion time | 1956.42 ms |
| Max block chat completion time | 4305.79 ms |
| Wall time for all block completions | 5613.02 ms |
Conclusion:
The bottleneck is vLLM generation, not Python parsing or image decode. The single request spends about 72% of wall time in block OCR generation (5613 ms / 7801 ms) and about 24% in layout generation (1902 ms / 7801 ms). CPU-side crop/build/parse/assembly is negligible for this image.
Most useful next optimization directions:
- Reduce number of OCR block requests per page.
- Reduce block max token budget.
- Merge or batch adjacent text blocks before recognition.
- Avoid running layout when caller can provide/skip detection.
- Tune vLLM scheduling/parallelism after reducing block count.