Lesson 11: Warp Divergence
A warp is a group of 32 threads that execute the same instruction together (lockstep) under the SIMT model. As long as every thread in a warp follows the same code path, the hardware runs them together in one step. But what happens when an if/else sends some threads down the if branch and others dow
Imagine a choir of 32 singers who must sing the same note at every moment. If half need to sing 'la' and half 'do', the conductor first lets all the 'la' singers sing (the rest stay silent) and then all the 'do' singers. That takes double the time. But if one whole choir sings 'la' and a second whole choir sings 'do', both sing at once with no waste.
- warp divergence
- When threads in the same warp take different if/else branches, so the hardware runs the paths serially — the time is the sum of the paths.
- lockstep
- All 32 threads in a warp execute the same instruction in the same step. This is the basis of SIMT.
- predication
- When only some threads in a branch are active, the rest are masked: they run the same instruction but write no result.
- uniform branch
- A condition that all threads in a warp evaluate the same way (for example by warpId), so there is no divergence and no penalty.
