Intermediate → Advanced

End-to-end: routing, serving compilers, KV-cache, streaming, evals, cost telemetry. Hands-on with vLLM, TensorRT-LLM, and Triton on real models.

• Stand up a production-grade inference endpoint
• Tune throughput and latency with compilers and batching
• Wire cost-per-call telemetry that holds up in finance review

Intermediate

Bin-packing, spot/on-demand mixing, preemption-safe training, and the dashboards that actually move utilization. Less waste, same hardware.

• Diagnose under-utilization in a real cluster
• Mix spot, on-demand, and reserved capacity safely
• Build $/utilization dashboards by team and workload

All levels

Chunking, embedding choice, hybrid retrieval, re-ranking, eval design — and the pipelines to keep all of it fresh. Tuned to product metrics, not benchmark vanity.

• Design a retrieval system tied to a product KPI
• Operate hybrid (lexical + semantic) retrieval
• Build an eval harness that gates deploys

Intermediate → Advanced

Tool execution, sandboxing, durable workflow state, per-step evals, and budget enforcement. The boring engineering that lets agents ship.

• Design and operate a durable agent runtime
• Sandbox tool execution safely
• Add per-step evals and budget caps that hold