Lesson 4: Where Does the Data 'Live'? — CPU, GPU, and the Trip Between

Still almost no code. Last lesson we understood what inference is; now we'll understand where the data sits while it runs. A GPU has its own memory, separate from the regular computer. For it to work on your data, you must first send the data over — and that's a trip that costs time. We'll grasp the

A tensor is a suitcase of numbers. The GPU is a giant workshop across town. For it to work on the suitcase, you must ship it there — and every shipment costs time.

Tensor: A box of numbers — the basic type AI works with. An image, text, or vector — all become a tensor.
Device: Where a tensor 'lives' and the compute happens: 'cpu' (regular memory) or 'cuda' (GPU memory).
CPU↔GPU transfer: Copying data between regular memory and GPU memory, over a bus called PCIe. Relatively slow — you minimize it.