Team 15 (HPC Hacks)

Ember

A neural surrogate for wildfire simulation that learns while the physics keeps running.

Trains a U-Net to imitate the ForeFire wildfire simulator, with CPU cores generating new simulations while both GPUs train one shared model.

Cadence Centers, Danish Nadar, Vijay Pratap Singh, Sashank Reddy Kadiveti, Raj Dwivedi

2,880

completed ForeFire simulations

2,400 for training, 300 for validation and 180 for test.

0.48 IoU

on 180 held-out fires

Dice 0.65, precision 0.78 and recall 0.55.

7.1 min

arrival-time error

Mean over a 60-minute forecast, on cells that burn.

1m 40s

for the two-node loop

Simulate, merge and fine-tune on two GPUs, end to end.

Figures are the team's own measurements from the hackathon weekend, as reported in their submission.

The problem

Fire simulators answer one question per run. The questions people actually ask need many: what if the wind shifts, what if it starts 500 m east, what happens over the next 6 hours? A trained surrogate answers each in one GPU pass.

What they built

Ember trains a U-Net to imitate the open source ForeFire physics simulator, so the footprint and arrival time of a fire come back in one pass. Around it the team built the loop that makes that possible on two nodes.

  • Physics and AI at once. On each node ForeFire runs on pinned CPU cores while the GPU trains, so neither waits for the other.
  • Labels are checked. Every new simulation is tested for shape, bad values, duplicates and split leakage before it enters training. Validation and test sets never change.
  • One model, two GPUs. PyTorch distributed training over NCCL, with final weights identical on both ranks.
  • A fix to the simulator. The stock US model in ForeFire read wind in the wrong units and doubled slope. The team’s patch matches the BehavePlus reference within 1 to 4% for wind.

The data

Seven regions, from Corsica to Southern California, Nevada, Texas and Montana, with Arizona held out entirely. Fuel maps come from LANDFIRE and cover 28 fuel types on real terrain from sea level to 2,924 m. Each sample is a 12.8 km square with 19 input layers and a 60-minute forecast. About 26,000 scenarios are prepared and 2,880 are fully simulated so far.

How they used the cluster

  • Fuzzball runs the loop: one phase-one job per node, a merge barrier, then a distributed job on two GPUs. Outputs move between stages as hashed artifacts.
  • NVIDIA GB10: the Grace cores run ForeFire and the Blackwell GPU trains and serves the surrogate.
  • The two Lenovo ThinkStation PGX machines are linked by 100 Gb Ethernet.

Limits the team states

Accuracy means agreement with held-out ForeFire simulations, not with real fires. ForeFire agrees with itself at IoU 0.90 to 0.92 under tiny input changes, and Ember reaches 0.48. Recall of 0.55 means it misses about 45% of burned cells. The team claims no speedup: at a 1-hour horizon ForeFire is already fast, and the case for a surrogate is longer horizons and ensembles, which they plan to measure first.

Research prototype.