AI, As I Go

Hardware basics

CPU, GPU, TPU, DPU: the chips behind AI

A real AI server doesn't run on just one kind of chip. A couple of CPUs orchestrate, many GPUs — or TPUs, on Google's own stack — do the heavy math, and a DPU quietly keeps data flowing so nothing stalls. These aren't competing chips; they're teammates with very different jobs.

Four chips, four jobs: CPU, GPU, TPU, DPU Four hand-drawn chips side by side, each labeled with its job. CPU, the manager: a handful of big cores (4 to 256), runs the operating system and tells everyone what to do. GPU, the math crowd: thousands of small, simple cores working in parallel. TPU: hard-wired for matrix math, efficient per watt, Google's own chip (an ASIC). DPU: keeps data moving and does no AI math, handling networking, storage and security. Below, how they team up inside one server: 2 CPUs orchestrate many GPUs (or TPUs) that do the heavy math, while a DPU feeds them data from the network and storage, so the expensive GPUs never sit idle waiting for data. Read about the CPU ↓CPUthe manager4–256 big coresruns the OS,tells everyone what to do Read about the GPU ↓GPUthe math crowdthousands of coressmall & simple,all in parallelbusy! Read about the TPU ↓TPUhard-wired formatrix mathefficient per wattGoogle's own chip(an ASIC) Read about the DPU ↓DPUkeeps data movingno AI math!networking, storage& security Read how they team up (the DPU's job) ↓how they team up inside one server2× CPUorchestratemany GPUs (or TPUs)do the heavy mathDPUfeedsdatanetworkstorage,securityso the pricey GPUs never sit idlewaiting for data Four chips, four jobs: CPU, GPU, TPU, DPU Four hand-drawn chips side by side, each labeled with its job. CPU, the manager: a handful of big cores (4 to 256), runs the operating system and tells everyone what to do. GPU, the math crowd: thousands of small, simple cores working in parallel. TPU: hard-wired for matrix math, efficient per watt, Google's own chip (an ASIC). DPU: keeps data moving and does no AI math, handling networking, storage and security. Below, how they team up inside one server: 2 CPUs orchestrate many GPUs (or TPUs) that do the heavy math, while a DPU feeds them data from the network and storage, so the expensive GPUs never sit idle waiting for data. Read about the CPU ↓CPUthe manager4–256 big coresruns the OS,tells everyone what to do Read about the GPU ↓GPUthe math crowdthousands of coressmall & simple,all in parallelbusy! Read about the TPU ↓TPUhard-wired formatrix mathefficient per wattGoogle's own chip(an ASIC) Read about the DPU ↓DPUkeeps data movingno AI math!networking, storage& security Read how they team up (the DPU's job) ↓how they team up2× CPUorchestrateGPUs(or TPUs)heavymathDPUfeedsdatanetworkstorage, securitypricey GPUs never sit idlewaiting for data
Four chips, four jobs — and how they share the work inside one AI server. Each chip gets its own section below. Click or tap a title or picture in the sketch to jump to its section.

The general-purpose brain of almost every computer. It runs the operating system, handles input and output, and executes instructions one after another extremely fast — built to do any task well, including ones where each step depends on the result of the last.

Originally built to render graphics, which means doing the same simple calculation across millions of pixels at once. That same trick turned out to be exactly what AI needs — repeating the same math, like matrix multiplication, across huge amounts of data in parallel — which is why GPUs ended up running the field instead of graphics cards.

The real difference: a CPU is built around a handful of large, complex cores meant to handle any task well. A GPU is built around thousands of small, simple cores meant to repeat the same math across huge amounts of data at once. That's also why GPUs get ultra-fast HBM memory stacked directly on the chip: feeding thousands of cores at once takes far more bandwidth than a CPU's smaller core count ever needs.

CPU die 4–256 large, complex cores GPU die thousands of small, simple cores
Shown schematically — a real GPU die packs thousands of these tiny cores, not eighteen. Same wafer, same manufacturing line, opposite design goal. In a real package, that difference shows up in die count too: AMD's EPYC “Venice” CPU is 10 dies total (8 core chiplets + 2 I/O dies, no stacked memory), while Nvidia's B200 GPU package has 70+ dies (2 compute dies + 8 HBM3e memory stacks, 8–12 layers each).
AMD (AMD) — design Nvidia (NVDA) — design

A TPU (Tensor Processing Unit) takes specialization even further than a GPU. Where a GPU is a general-purpose parallel processor that happens to be great at AI math, a TPU is an ASIC — a chip hard-wired for one job, the matrix multiplication at the core of neural networks. That trade cuts flexibility but improves efficiency per watt. TPUs are Google's own chip, used internally and rented out through Google Cloud; Ironwood (TPU v7) was built primarily for inference at scale, and Google has since split its 8th-generation TPUs into separate training (TPU 8T) and inference (TPU 8I) chips. Amazon (Trainium) and Microsoft (Maia) have since built their own equivalents, for the same reason: less dependence on Nvidia and lower cost per inference.

Alphabet (GOOGL) — design Amazon (AMZN) — design Microsoft (MSFT) — design

A DPU (Data Processing Unit) does something different from all three chips above — it doesn't do AI math at all. It's a specialized chip that offloads networking, storage, and security tasks away from the CPU. That matters in an AI cluster because training and inference move enormous amounts of data between GPUs constantly; if the CPU has to manage that network traffic itself, it becomes a bottleneck, leaving the far more expensive GPUs sitting idle waiting for data. Nvidia's BlueField is the best-known example, with AMD's Pensando as another option.

Nvidia (NVDA) — design AMD (AMD) — design