Cutting inference cold starts by 40x with LP, FUSE, C/R, and CUDA-checkpoint

Cutting inference cold starts by 40x with LP, FUSE, C/R, and cuda-checkpoint We are in the age of inference. Billion- to trillion-parameter neural networks are run on specialized accelerators at quadrillions of operations per second to generate media, author software, and fold proteins at massive scale. Inference workloads are more variable and less predictable than the training workloads that pre...