Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training

I slightly tweaked the personality of the hosts.