Graphical Processing Unit, Graphics Card, GPU

A specialized computer component normally used for the processing of graphics, but widely applicable to a large number of general-purpose computing use cases. GPUs have a large number of small cores (potentially numbering in the thousands) that are good for both doing certain types of calculations related to graphics as well as for doing the calculations that underlie AI training and inference, as well as a wide array of other computing tasks. The GPU has been one of the most ubiquitously used accelerator cards in HPC for some time, especially as AI has become a more and more prominent use case.

GPUs are good for AI training/inference over CPUs because those processes rely on a very large number of simple calculations being able to be done in parallel. There are two points here: first, CPUs typically have 8, 16, 32, 64, or up to 128 or more cores in high-end modern server processors versus potentially thousands in a GPU, and second, these cores are highly sophisticated and include a great deal of attention in their design to doing pipelining and other sequential-type operations—much of this complexity is wasted in the case of AI training/inference, where as said, the need is for a very large number of simple calculations to be done at once. The GPU, with thousands of simple cores that can all run in parallel on these calculations, can perform them in the end much faster and more efficiently than the CPU can with its much lower core count and cores much better suited to more advanced operations.