Design a PDF-to-Markdown inference API (sync and async)

Build a service that turns PDF files into Markdown out of three existing functions: one that splits a PDF into page images (CPU-heavy), an OCR model that reads a page image or a batch of them (GPU-heavy), and a step that turns OCR output into Markdown and stitches the pages together (memory-heavy).

1. One user sends a 1,000-page PDF and wants all of it back as fast as possible over a synchronous API. How do you get the latency down, keep the GPU busy, deliver pages in order, and stay within a connection's time limit?
2. Many users send documents and are happy to collect the result later. Design the asynchronous API and the pipeline behind it for throughput and cost: what sits between the stages, how GPU batches are filled across jobs, how one huge document is kept from starving small ones, and how each stage scales.

Cover where intermediate pages live, retries for a single bad page, back-pressure, and what you would watch in production.