High-Throughput Computing (HTC)

High-throughput computing (HTC) uses large, widely distributed networks of resources to run computational jobs that do not require the massively parallel, tightly coupled resources typical of high-performance computing (HPC). Instead of coordinating one program across many cores at once, HTC focuses on completing an enormous number of independent jobs over long stretches of time. It is closely associated with grid computing, which federates computing capacity from many sites into a shared pool.

What is high-throughput computing?

HPC is usually measured in operations per second and optimized for how fast a single large problem can be solved. HTC is measured differently: by how much work can be finished over days, weeks, or months. Its workloads rarely need many cores or nodes working in lockstep on one calculation. Instead they consist of many separate jobs that can run wherever spare capacity exists, which makes HTC a natural fit for embarrassingly parallel work, where each task is independent of the others.

Why HTC matters

Many scientific and analytical problems are made of vast numbers of small, independent computations rather than one tightly coupled simulation. For these, gathering throughput from many ordinary machines is more effective and far cheaper than reserving a tightly integrated supercomputer. HTC lets institutions pool otherwise idle resources across data centers, campuses, and even volunteers' personal computers, turning scattered capacity into a large aggregate throughput.

HTC and grid computing in practice

Grid computing is the model most often used to deliver HTC at scale. One of the best-known examples is the Open Science Grid, which draws on resources worldwide to provide a general-purpose grid used by many institutions; much of its capacity powers analysis of data from the Large Hadron Collider. Another well-known project, Folding@home, harnesses spare CPU cycles from computers around the world to simulate protein folding and has reached more than an exaflop of distributed computing capacity. Both illustrate the HTC principle: coordinate many independent jobs across a broad, loosely connected set of resources rather than one high-speed cluster.

HTC vs. HPC

	High-throughput computing	High-performance computing
Optimizes for	Total jobs completed over time	Speed of a single large problem
Job style	Many independent tasks	Tightly coupled, communicating tasks
Coupling	Loosely connected resources	High-speed interconnect required
Typical resources	Distributed grids, spare cycles	Dedicated cluster or supercomputer

The two approaches are complementary rather than competing. Tightly coupled simulations belong on HPC systems, while campaigns of independent analyses are best served by the distributed, throughput-oriented model of HTC.