Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constra…
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
davila7
cli
free
Others in the same category, ranked by how often they are opened.
