Simulating fault tolerance with stage skipping in pipeline-parallel training
Our most recent work at Templar explores fault tolerance in Crucible, our distributed pre-training platform. The goal is to keep healthy workers training when another pipeline stage goes offline.