https://blog.0xnullpath.cc/en/tags/wip/Recent content in WIP on Code NowHugoenMon, 20 Apr 2026 04:27:43 +0000https://blog.0xnullpath.cc/en/posts/note-snippet-15-model-inference-a-brief-look-at-cuda-graph/Mon, 20 Apr 2026 04:27:43 +0000https://blog.0xnullpath.cc/en/posts/note-snippet-15-model-inference-a-brief-look-at-cuda-graph/<p>CUDA Graph is probably one of the most frequently mentioned features in inference optimization. The principle isn&rsquo;t complicated: package the kernels that were originally launched one at a time into a static graph, hand the whole thing to the GPU in one shot, and skip the scheduling overhead in between.</p> <p>But how exactly does this &ldquo;packaging&rdquo; work? How do you use it in PyTorch? And most importantly — how much faster is it actually on an H200?</p>