Suya OCR API — vLLM-backed, OpenAI-compatible OCR service

FastAPI service wrapping the Surya-OCR-2 model (datalab-to) served through vLLM: legacy /v1/api/ai/* endpoints, an OpenAI-compatible /v1/chat/completions endpoint, a coalescing request batcher, a local OCR CLI, Docker packaging, multilingual example outputs, and quantization/concurrency benchmarks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:20:02 +04:00
commit 1a585693be
147 changed files with 13827 additions and 0 deletions
@@ -0,0 +1,125 @@
+# Repository Instructions
+
+## Project Context
+
+This repo serves a FastAPI OCR API from `api.py`. The API keeps the legacy
+request/response shape:
+
+```json
+{"data": ..., "message": "success", "code": 200}
+```
+
+Input validation is handled with Pydantic in `Info`. Keep external payload
+fields stable unless the user explicitly asks for a breaking API change.
+
+Primary endpoints:
+
+- `POST /v1/api/ai/suya_ocr/`: legacy OCR route.
+- `POST /v1/api/ai/suya_ocr_vllm/`: vLLM-backed OCR route.
+- `GET /v1/api/ai/suya_ocr_vllm/health`: reports vLLM backend attachment.
+- `POST /v1/api/ai/suya_text_det/`, `/suya_layout_det/`, `/suya_table_rec/`.
+
+## Local Run
+
+Use the conda Python on this host; Homebrew `python3` may not have the API
+dependencies installed. For local direct `api.py` runs, start a vLLM OpenAI
+server separately and point the API at it.
+
+```bash
+PYTHONDONTWRITEBYTECODE=1 \
+PYTHONPATH=/path/to/suya-ocr-api \
+SURYA_INFERENCE_BACKEND=vllm \
+SURYA_INFERENCE_URL=http://127.0.0.1:8000/v1 \
+SURYA_INFERENCE_AUTOSTART=false \
+SURYA_INFERENCE_PARALLEL=8 \
+SURYA_MAX_TOKENS_FULL_PAGE=6144 \
+/opt/conda/bin/python api.py
+```
+
+The API listens on `http://127.0.0.1:5002`.
+
+For Kubernetes/container deployment, use `Dockerfile`. It is based on
+`vllm/vllm-openai:v0.20.1` and runs vLLM in the background, then `api.py` in
+the foreground through `scripts/start_single_container.sh`.
+
+## vLLM Backend Notes
+
+The repo vendors the Surya source under `surya/`; runtime must not depend on
+`/path/to/surya`. `vllm_tools.py` must set Surya/vLLM environment
+defaults before importing `surya.settings` or Surya predictor classes.
+Important defaults:
+
+- `SURYA_INFERENCE_BACKEND=vllm`
+- `SURYA_INFERENCE_URL=http://127.0.0.1:8000/v1`
+- `SURYA_INFERENCE_AUTOSTART=false`
+- `SURYA_INFERENCE_LOGPROBS=false`
+- `SURYA_INFERENCE_MAX_RETRIES=1`
+- `SURYA_MAX_TOKENS_FULL_PAGE=6144`
+- `SURYA_MAX_BLOCKS_PER_PAGE=80`
+- `SUYA_OCR_MODE=block`
+
+The Dockerfile is tuned conservatively for a T4 machine:
+
+- `VLLM_GPU_TYPE=t4`
+- `VLLM_DTYPE=float16`
+- `SURYA_INFERENCE_PARALLEL=8`
+- `VLLM_MAX_NUM_SEQS=16`
+- `VLLM_MAX_BATCHED_TOKENS=4096`
+- `VLLM_GPU_MEMORY_UTILIZATION=0.85`
+
+If running on A100 hardware, do not silently change the Dockerfile T4 defaults;
+ask or document the deployment target first.
+
+## Docker/NVIDIA
+
+The production container must not run nested Docker. Kubernetes should provide
+GPU access to the single container. The old Surya Docker auto-spawn path is
+disabled unless `SUYA_ALLOW_NESTED_DOCKER=true` is explicitly set.
+
+For host-level debugging only, Docker GPU runtime can be checked with:
+
+```bash
+docker info --format '{{json .Runtimes}}'
+docker run --rm --runtime nvidia --gpus all --entrypoint nvidia-smi vllm/vllm-openai:v0.20.1 -L
+```
+
+If the named runtime is missing, use:
+
+```bash
+sudo nvidia-ctk runtime configure --runtime=docker --config=/etc/docker/daemon.json
+sudo systemctl restart docker
+```
+
+`scripts/docker_nvidia_runtime_compat.sh` is legacy support for local debugging,
+not the Kubernetes deployment path.
+
+## Concurrency Benchmark
+
+Run the comparison benchmark with the same image and settings when updating
+results:
+
+```bash
+/opt/conda/bin/python concurrency_test.py \
+  --image /path/to/surya/static/images/excerpt.png \
+  --requests 16 \
+  --concurrency 16 \
+  --timeout 900 \
+  --output concurrency_test_results.json
+```
+
+After a successful run, update:
+
+- `concurrency_test_results.json`
+- `IMPLEMENTATION_PLAN_AND_TEST_RESULTS.md`
+
+Report latency columns as seconds. Mean latency is the average elapsed time per
+request, not wall-clock time for the whole benchmark.
+
+## Editing Guidance
+
+- Use `rg`/`rg --files` for search.
+- Use `apply_patch` for manual edits.
+- Do not revert user changes or generated benchmark artifacts unless requested.
+- Keep logging structured and include request IDs for API paths.
+- Avoid import-time OCR work; model or backend attachment should happen only as
+  needed by predictor setup and requests.