vLLM

1. Quickstart

   Offline inference and starting an OpenAI-compatible server.

2. PagedAttention

   KV cache paging that underlies vLLM throughput.

3. Supported models

   Model architectures vLLM can serve.

4. OpenAI-compatible server

   Chat, completions, and embeddings endpoints and server flags.

5. Automatic prefix caching

   Reusing KV cache for shared prompt prefixes.

6. Quantization

   AWQ, GPTQ, FP8, and other quantization schemes.

7. Speculative decoding

   Draft model and n-gram speculation to cut latency.

8. Structured outputs

   Guided decoding with JSON schema and grammar.

9. Parallelism and distributed serving

   Tensor, pipeline, and data parallelism across GPUs and nodes.

10. Performance tuning

    Preemption, chunked prefill, batching, and memory tuning.

11. Deployment

    Docker, Kubernetes, and production deployment.

12. Metrics and observability

    Prometheus metrics for monitoring throughput and latency.

Each section links to the official page. Titles and summaries by Extra Credit; the docs belong to their authors.