The Association for Computational Linguistics has awarded its 2026 Test-of-Time award to a 2016 paper co-authored by Aveni co-founders Dr Barry Haddow and Dr Alexandra (Lexi) Birch, together with Professor Rico Sennrich. The award was announced this week at ACL 2026 in San Diego.
The paper, Neural Machine Translation of Rare Words with Subword Units, introduced the method that determines how almost every large language model reads and writes text today. The ACL is the leading worldwide academic society for research in natural language processing and gives the Test-of-Time award each year to papers that have demonstrated lasting impact on computational linguistics. Around 800 papers appeared across ACL’s three premier conferences in 2016. This one was selected in the ten-year category.
The problem the paper solved
In 2016, translation systems built on neural networks were beginning to outperform the statistical systems that had powered tools such as Google Translate for a decade. The new systems carried a design constraint. Each one held a fixed vocabulary, typically capped at 50,000 words by the computing power of the time, and language produces far more words than that. Compounds, names, coinages and inflected forms fell outside the list, and the systems garbled them. English-to-German output showed the problem clearly: translations read well until the model hit one of German’s long compound words, which often came out truncated.
The authors solved it by changing what the model treats as a word. Their method breaks rare words into smaller, reusable pieces called subwords, keeps common words whole, and lets the model assemble words it has never seen from parts it knows. German makes the idea concrete. Einkaufen means to shop, zentrum means centre, so an einkaufzentrum explains itself. The technique, adapted from a decades-old compression algorithm called byte pair encoding, took neural translation from a closed vocabulary to an open one.
Why it shaped the decade that followed
A 2017 paper introduced the transformer, the neural network design that made large-scale language models possible, and it used subwords as its basic building block. Every large language model in production today, including GPT, Llama and Mistral, reads and writes subwords chosen by the method this paper introduced. LLM usage is priced in tokens because a token is a subword: the unit these models actually consume and generate.
The Aveni connection
Barry and Lexi co-founded Aveni and lead Aveni Labs, the research team behind FinLLM, the large language model built for UK financial services. The same subword approach they introduced in 2016 sits inside FinLLM today. How a model breaks down specialist vocabulary affects how well it handles the language of regulated finance, and the team that wrote the method now applies it to terms such as Consumer Duty and SM&CR.
Alexandra Birch: “In 2016 we were trying to fix truncated German compounds in translation output. We had no idea the method would end up inside every large language model in the world. Ten years on, watching the same idea carry the field through an era of extraordinary scaling has been remarkable.”
Barry Haddow: “The algorithm has lasted because it balances simplicity, computational efficiency and performance in a way that more theoretically principled alternatives have struggled to beat at scale. We are proud the field still builds on it, and we apply the same thinking every day in our work on FinLLM.”
The paper
Sennrich, R., Haddow, B. and Birch, A. (2016). Neural Machine Translation of Rare Words with Subword Units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715 to 1725. https://aclanthology.org/P16-1162/
