If you’ve read some of our previous posts, then you already know we are all about evoking apocalyptical scenarios when it comes to the role machines play in postmodern societies and the moral dilemmas posed by our ever growing dependence on technology. That, and making Kubrick film references whenever possible, of course.
Well, we might just cut right to the chase this time, ‘cause HAL 9000 could have just become operational, this time put to the service of our beloved translation industry.
It seems Google has unleashed a deep learning tech on language with Neural Machine Translation that has the ability to “learn” on its own, being able to translate into unknown language pairs by creating a “meta-language” or “interlingua” as part of the process, producing very respectable results.
Neural machine translation (NMT) is an approach to machine translation in which a large neural network is trained by deep learning techniques. It is a radical departure from what we may now call the “classical approach” used by most machine translation engines so far, who usually break down the corpus of the source material into individual pieces as if it were a jigsaw puzzle that can later be reassembled.
NMT models apply deep representation learning. They require only a fraction of the memory needed by traditional statistical machine translation models. Furthermore, unlike conventional translation systems, all parts of the neural translation model are trained jointly, in an end-to-end pattern, to maximize translation performance.
Translating from one language to another is no monkey business, and creating a system that does it automatically is no walk in the park either, partly because there are just so many words, phrases and rules to deal with in the process. Fortunately, neural networks eat big, complicated data sets for breakfast, and they always seem hungry for more.
Google Neural Machine Translation (from here on GNMT) is the latest and by far the most effective tool to successfully leverage machine learning in translation. It looks at sentences as whole syntagms, while keeping “in mind”, so to speak, smaller pieces like words and common phrases.
It’s much like the way we look at an image as a whole while being aware of the individual pieces that put it together (think of pixels),which is by no means coincidental. Neural networks have been trained to identify images and objects in ways that try to replicate human perception, and there’s more than a passing resemblance between identifying the gestalt of an image and that of a sentence.
Interestingly enough, there’s little in there that’s actually specific to language: The system doesn’t know the difference between the future perfect and future continuous, and it doesn’t break up words based on their etymologies. It’s all stats and crazy math extravaganza, but no humanity or soul to it.
Reducing translation to a mechanical task is admirable, but in a way it’s also chilling, to say the least.
The resulting system though, is highly accurate, beating phrase-based translation engines and gradually approaching human levels of quality. You know it has to be good when Google just deploys it to its public website in the shape of an app made for a difficult process like translating Chinese into English.
So what are the downsides? When is HAL gonna try to kill us all and eject us into space?
Well, the spooky part is we don’t really know…
One of the downsides of using this vanguard technology is that, as with so many predictive models produced by machine learning, we don’t really know how it works.
“GNMT is like other neural net models — a large set of parameters that go through training, difficult to probe” Google’s Charina Choi said in a recent interview about the subject.
It’s not that they have no idea of what’s going on, it’s not all a bunch of arcane “mumbo jumbo” after all, but the many moving parts of phrase-based translation engines are designed by people, and when a piece goes rogue or becomes outdated, it can be swapped out. Because neural networks essentially design themselves through millions of iterations, if something goes wrong, we can’t reach in and replace a part. Training a new system isn’t trivial, though it can be done quickly (and likely will be done regularly as improvements are conceived).
Google is betting big on machine learning, and this translation tool, now live for web and mobile queries, is perhaps the company’s most public demonstration yet. Neural networks may be complex, mysterious and a little creepy, but it’s hard to argue with their effectiveness.
Still, the puzzle machine translation posses is yet to be solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page. There is still a lot of work to be done. However, GNMT represents, without a doubt, a significant milestone on its way to greatness.
We’ll just have to wait and see how that path unravels…
