AI, As I Go

Research

Inference & serving

What happens when a model is actually used: how your question turns into an answer, and what it takes to serve lots of people at once. A new page lands here each time I finish a lesson.

1 entry · newest first

How your question becomes an answer

The model is the recipe, the GPU is the cook. What happens between hitting Enter and the answer streaming in, and why one GPU serves many people at once.