Have URMs and UTs been integrated into frontier models? Or did they disappear into the dustbin of forgotten papers?
UT-based small models, despite being trained from scratch on these tasks without internet-scale pre-training, consistently outperform most standard Transformer-based Large Language models (LLMs)...