Guidance for optimizing LLM inference request batching and scheduling problems. This skill applies when designing batch…
Guidance for optimizing LLM inference request batching and scheduling problems. This skill applies when designing batch schedulers that minimize cost while meeting latency and padding constraints, involving trade-offs between batch count, shape selection, and padding ratios. Use when the task involves grouping requests by sequence lengths, managing shape compilation costs, or optimizing multi-objective scheduling with hard constraints.
benchflow-ai
cli
free
Others in the same category, ranked by how often they are opened.
