Dr Lexi Birch, is officially ranked as one of the top 100 Natural Language Processing (NLP) Engineers in the world. She is a Senior Research Fellow at The University of Edinburgh School of Informatics and Head of Aveni Labs. Her research on machine translation has made a significant impact on the field with methods she has developed in Neural Machine Translation (NMT) now standard practice. Lexi coordinates an EU project on low-resource machine translation called GoURMET, which provides translations for languages such as Tamil and Igbo for the BBC. She is also area chair for computational linguistics and machine learning conferences, and she coordinates a high profile workshop on neural generation and translation, WNGT.
We catch up with her to find out more about her path to NLP distinction, the role she has played in Aveni’s commercialisation and her ambitions for the business.
NLP is a pretty specific field of AI. What interested you about this domain and what has your career path been to land you as one of the foremost experts in this field?
After studying a BSc in computer science in my hometown South Africa, I went on a rock climbing holiday to Spain. I loved it and kept visiting, eventually moving to Spain to live and work. However, after working as a programmer for a number of years, I felt the need to specialise. Learning to speak Spanish had been a wonderful experience which opened my eyes to the delights and vagaries of human language. I discovered that there was a field called Computational Linguistics which combined two things I found fascinating: computer science and the study of language, and I thought that this must be my specialism.
I moved to Edinburgh to do a master’s degree in artificial intelligence with a focus on language. After that, I had the opportunity to pursue a PhD, and since, I have never left. I am very lucky that Edinburgh University has one of the world’s largest and most respected groups of computational linguists in the world. I have had the chance to work together with people like Philipp Koehn, whose paper on phrase-based machine translation eventually resulted in translation systems which could be trained in days and replaced rule-based systems which took decades to craft.
When deep learning and neural networks started making waves in related fields such as image processing and speech processing, I quickly transitioned to working on understanding and improving neural network models and applying them to processing text. I led a project where we tackled some basic limitations of these new models and we made quick progress. These papers have been very influential and have led to further opportunities such as my fellowship, and to leading two EU projects.
Who or what has influenced you the most when it comes to how you approach your work?
Philipp Koehn has had a deep impact on my work. He started our group, the StatMT group in 2005, but he left to move to John Hopkins in 2014. Apart from being a great researcher and teacher, it was his generosity that really stuck with me. He worked tirelessly to provide an environment where others could flourish. He put effort into toolkits, corpora, text-books and competitions which pushed forward, not only the field but also the people who worked in it.
How did you meet Joseph Twigg (Aveni’s CEO) and what were the key elements leading to Aveni being founded?
I was managing an EU project called SUMMA, and we collected a compelling pipeline of NLP tools together in a media monitoring platform for the BBC. Unfortunately, this platform was not maintainable as it involved 50+ researchers across nine institutions. But it made me realise that, even though most of the components on their own were not commercially viable, groups of these components when put together could be transformative for many different industries and applications. The challenge was really finding a problem to apply them to. Joseph Twigg was the perfect collaborator. He had a deep understanding of the finance industry and an impressive network of colleagues which meant that we could speak to the right people.
What is your role at Aveni? What does a typical day look like for you?
I’m head of Aveni Labs. My role is to understand the challenges facing our partners and to seek opportunities for deploying NLP techniques. I also work part-time at The University of Edinburgh, supervising students working on research problems that are relevant to Aveni, such as segmenting conversations into topics and extracting the relations between entities in dialogues.
This might ultimately allow financial advice firms to detect what topics customers are most concerned about, and make sure advisors are covering important information. It will also help to extract customer information from conversations to automatically populate CRMs.
Had you ever worked in this capacity at a start-up before?
No, so Aveni’s journey and growth have been a very new and exciting experience for me!
What have the biggest revelations/learnings for you been throughout this start-up journey?
It takes an enormous amount of effort and time to establish a partnership where you can access sensitive data. However, with the right relationships in place, there is really an amazing number of opportunities across the finance industry that NLP can address. For example, unstructured data can be transformed into business insights, and customer interactions can be mined to improve services in real-time.
What ambitions do you have for the business? Where do you see it 5 years from now?
Aveni is perfectly positioned to deliver transformative technologies for the finance industry. My hope is that we will be able to create a suite of products that help professionals to be more efficient and have greater reach. Ultimately, I would like our technology to help everyone to be able to afford just the right amount of high-quality human financial advice.
What exciting advances are happening in the field of NLP that you can see having a real impact on the professional advice industry?
One of the major problems facing our field is how to learn from small amounts of labelled data. Machine learning often uses millions of pieces of information to learn to predict an outcome e.g. predicting stock market prices. But for many problems related to NLP we often only have hundreds of examples, for example where customers say that they have a vulnerability like a health problem.
In order to learn from very few examples, we need to emulate human learning. We need to transfer our world knowledge, or our learning on other related tasks, to this new task. So we could learn about the English language by learning to predict the next word and then training on millions of sentences of English text freely available from the internet. We can then use a small number of labelled examples to train a classifier built on top of this sophisticated model of English.
We can also use data augmentation by creating more training data by paraphrasing the examples we do have. There is a large amount of recent progress occurring in our field addressing these issues. We have already seen huge increases in performance of NLP technologies on tasks such as question and answering, sentiment detection, and sentence classification using transfer learning. I think that these technologies will make it possible for us to develop solutions faster, more cheaply and with greater accuracy than ever before.
