#knowledge distillation

AI Models May Have Found a Way Past Tokens

A tiny vocabulary sounds like a shortcut, but it comes with a costly speed penalty—and a surprising training advantage. The real breakthrough may be less about replacing today’s chatbots and more about making small models easier to teach.