nkozobrod/surya-ocr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Suya OCR API — vLLM-backed, OpenAI-compatible OCR service

Browse Source 
FastAPI service wrapping the Surya-OCR-2 model (datalab-to) served through vLLM:
legacy /v1/api/ai/* endpoints, an OpenAI-compatible /v1/chat/completions endpoint,
a coalescing request batcher, a local OCR CLI, Docker packaging, multilingual
example outputs, and quantization/concurrency benchmarks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Fu Dai
2026-06-17 10:20:02 +04:00
co-authored by Claude Opus 4.8
commit 1a585693be
147 changed files with 13827 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SINGLE_CALL_TIMING_RESULT.md
+60
View File
@@ -0,0 +1,60 @@
# Single Call Timing Result
Test image: `/path/to/suya-ocr-api/temp_image_c3c56948-282e-453b-8fac-6c482243d1e5.jpg`
Endpoint: `/v1/api/ai/suya_ocr_vllm/`
Run time: 2026-06-10 16:18 Asia/Dubai
Environment:
- `SURYA_INFERENCE_URL=http://127.0.0.1:43401/v1`
- `SURYA_INFERENCE_PARALLEL=8`
- `SUYA_OCR_MODE=block`
- `SUYA_TIMING_ENABLED=true`
Summary:
| Stage | Time ms | Notes |
|---|---:|---|
| Client observed request | 7812.80 | Single HTTP call to API |
| API request total | 7800.86 | FastAPI middleware duration |
| Base64/PIL decode | 33.38 | 683,466 byte JPEG, 1410x1033 |
| API wait for vLLM batcher | 7755.73 | Includes queue, layout, recognition |
| OCR batch total | 7730.19 | Surya vLLM OCR path |
| Layout predictor total | 2114.89 | One page layout call |
| Layout vLLM generate | 2053.21 | Includes backend attach/start check |
| vLLM backend attach/start check | 97.84 | Health/model lookup on first request |
| Layout chat completion | 1902.34 | One vLLM `/chat/completions` call |
| Layout parse/output boxes | 61.48 | JSON parse, bbox conversion, blank filtering |
| Recognition block total | 5615.26 | Block OCR after layout |
| Recognition build crop batch | 1.09 | Crop 17 OCR blocks, skip 1 block |
| Recognition vLLM generate | 5613.12 | 17 block calls, parallelism 8 |
| Recognition output assembly | 0.28 | Clean HTML and build page result |
| Response assembly | 0.01 | No bbox image drawing |
Block OCR details:
| Metric | Value |
|---|---:|
| Layout blocks returned in response | 18 |
| OCR block requests sent to vLLM | 17 |
| Skipped blocks | 1 |
| Sum of requested block max tokens | 6210 |
| Generated block tokens | 4454 |
| Sum of block chat completion times | 33259.06 ms |
| Mean block chat completion time | 1956.42 ms |
| Max block chat completion time | 4305.79 ms |
| Wall time for all block completions | 5613.02 ms |
Conclusion:
The bottleneck is vLLM generation, not Python parsing or image decode. The single request spends about 72% of wall time in block OCR generation (`5613 ms / 7801 ms`) and about 24% in layout generation (`1902 ms / 7801 ms`). CPU-side crop/build/parse/assembly is negligible for this image.
Most useful next optimization directions:
1. Reduce number of OCR block requests per page.
2. Reduce block max token budget.
3. Merge or batch adjacent text blocks before recognition.
4. Avoid running layout when caller can provide/skip detection.
5. Tune vLLM scheduling/parallelism after reducing block count.