Tauon: A new optimizer outperforming Muon on GPT-Mini (lower loss, ~8.5% faster step time)
Hey r/MachineLearning ! I’ve been working on a new optimizer called Tauon (turns out there is already "teon" but well if you have better idea, - i will gladly accept it!