11.11 The Biggest Deals of the Year. 40% OFF on selected cloud servers with a free 100 GB data transfer! Click here to learn more.
Many of you may already be familiar with common machine translation tools such as Google Translate and Bing Microsoft Translator, but have you heard of Alibaba's machine translation tool? For Alibaba, machine translation is vital to the survival and growth of Alibaba's cross-border e-commerce business.
In this article, Chen Bo'xing, Alibaba Group Senior Algorithm Expert in Machine Intelligence Laboratory, introduces Alibaba's machine translation capabilities in the e-commerce industry. We will be looking at the challenges and countermeasures by Alibaba Group's Machine Translation Technology team in the cross-border e-commerce field. Specifically, we will be discussing about data processing and selection, machine translation models, and other innovative work of Alibaba Machine Translation Technology team.
Chen Bo'xing is a Senior Algorithm Expert in Alibaba Group's Machine Intelligence Laboratory. His research concentrates on machine translation, natural language processing, and machine learning. Before joining Alibaba Group, he was a researcher (2009-2017) at the National Research Council Canada (NRC). Tracing further back, he served as a researcher in Singapore Information and Communication Research Institution after obtaining his Postdoctoral degrees from Université Grenoble Alpes (France) and FBK-IRST (Italy).
He obtained a Ph.D. from the Institute of Acoustics of the Chinese Academy of Sciences in 2003 and a bachelor's degree from Peking University in 1998. He and his colleagues successively posted over 50 meeting and journal papers. He once served as a reviewer of NLP important meetings and journals and was a member of the Procedure Committee. He and his team have successively obtained outstanding achievements in various machine translation evaluations. For example, at the 2017 Second Conference on Machine Translation (WMT17), they won the first place for Russian to English MT, at NIST 2012 won first place for Chinese to English MT, and at the International Workshop on Spoken Language Translation (IWSLT) took first place in 2005 and 2007.
The following content is extracted from his presentation.
You may already be familiar with many machine translation tools, such as Google Translate, Bing Microsoft Translator, and even Youdao Translate. However, many of you may not have heard of Alibaba's machine translation. Before we dive into the technical details of machine translation, let me share an example of using this technology for cross-border e-commerce. Every year during the Double 11 Shopping Festival, Russia's logistics system crashes due to large spikes in e-commerce trading. That's because a lot of Russians buy and sell things from China via AliExpress. How can this happen? Few Russians speak Chinese, and few Chinese sellers know Russian. Well, the answer is by using Alibaba's translation system.
I believe most of you are very familiar with Alibaba's business scope. Generally speaking, Alibaba is an international network-and-artificial intelligence company. Alibaba's mission is to do a business around the world much more comfortable, and its core business is about establishing connections between buyers and sellers. Alibaba has a series of e-commerce platforms, including AliTrip, Tmall, Taobao, AliExpress, Alibaba.com, ju.taobao.com, and Lazada in Southeast Asia. Alibaba also provides expansion platforms for such e-commerce services, such as finance, logistics, and advertisements. If buyers and sellers speak the same language, then there is no problem. However, if buyers and sellers talk different languages, machine translation will be needed.
Alibaba set up Alibaba DAMO Academy in November 2016. First, Alibaba DAMO Academy has an Academic Advisory Committee, which has set up DAMO Academies in Asia, America, and Europe. Second, Alibaba will build four internal laboratories, one of which is the Machine Intelligence Technologies Laboratory (MIT Lab); three more will follow. Also, Alibaba DAMO Academy will cooperate with many universities to build joint laboratories.
We are from the Machine Intelligence Technology (MIT) Lab (formerly known as iDST). Founded in 2014, MIT Lab now has more than 400 scientists and engineers. Our researchers got scattered across many cities and regions around the world, including Hangzhou, Beijing, Seattle, Silicon Valley, Singapore, and Moscow. The MIT Lab is dedicated to fundamental research in the AI industry. The MIT Lab mainly consists of four big teams, which are respectively Speech Technologies, Natural Language Processing (comprising smaller groups, such as machine translation, question answering, and sentiment analysis), Image/Video Content Analysis, and Deep Learning & Optimization.
We work in the machine translation team. As I have mentioned before, we have a total of approximately 40 engineers and researchers scattered across the world in Hangzhou, Beijing, and Seattle. Our technical team and business team collaborate closely with each other. The business team is mainly responsible for breaking down and forwarding Alibaba's internal machine translation demands. Taking AliExpress as an example, our colleagues in AliExpress do not directly communicate with the technical team. Instead, they turn to the business team and raise their requests. Then the business team will break down the requests into different technical procedures, and assign them to different technical teams—such as the image recognition team or the machine translation team.
In this case, the technical teams will be able to stay focused on technology and technological development. The entire machine translation team then gets divided into many different small groups, such as MT technology innovation, system building, data, architecture, and human translation platform.
If Alibaba's mission is to make a business around the world more comfortable, then Alibaba Translation Platform's mission would be to clear language obstacles for people in business around the world.
What are the e-commerce scenarios to which we can provide value and use translation? First of all, traffic direction, in-site searches, reservations, and purchase, as well as communication between buyers and sellers. People who have worked on machine translation may know that we may use BLEU Score—a machine translation evaluation standard—to evaluate the machine translation quality, just like we may determine the faithfulness and smoothness of human translation. However, with regards to business, there are some more indicators. For example, has machine translation improved a commodity's conversion rate? In this way, we can know whether machine translation has brought value to commodities or not.
The following figure shows Alibaba's Ecosystem. At present, our machine translation platform supports many Alibaba internal partners, including AliPay, Tmall, Taobao, Lazada, AliExpress, and Alibaba.com. Therefore, our machine translation platform has been providing support for many businesses.
The following figure shows Alibaba MT's capability. Assume that we need to translate an offline Chinese website into other languages, for example, Taobao, Alibaba MT needs to translate product information with billions of words every day, and the number of daily online translation requests may be several hundred million. Notably, during the Double 11 Shopping Carnival, there will be tens of thousands of translation requests every minute, making a daily total word count of over 200 billion words. What does it mean by 200 billion words? Let me make it easier for you to understand. Google Translate provides MT service in a few hundred language pairs to all people around the world. Their data disclosed in 2016 indicated that they translated approximately 140 billion words on a daily basis. Alibaba is a single company, and most ordinary users have never used Alibaba MT service. The translation capacity of 200 billion words means something. Other platforms can hardly achieve it. My point is Alibaba has its machine translation service. Alibaba MT is widely used and has a huge capacity.
Now, let's have a look at Alibaba Translation's business ecosystem. The first is the data. We have obtained many parallel corpora from the Internet. Alibaba has its machine translation platform and human translation platform. The human translation platform works in a manner that Alibaba raises translation requests on Alibaba's crowdsourcing translation platform, then external translators translate on the platform and get paid in return. In this way, Alibaba can process data in general fields. Alibaba also provides data from some specific fields or the e-commerce field on its crowdsourcing platform. Industry-specific data allows Alibaba to improve machine translation performance, and the enhanced machine translation can help improve human translation efficiency. By translating different types of data, such as product titles, descriptions, category information, review, and messages, Alibaba MT can provide support for various business platforms.
In early 2012, Alibaba started cooperation with the academic circle on machine translation. Alibaba MT team was officially set up in 2014. After accumulating experience in all these years, Alibaba has built its closed-loop process for machine translation performance improvement. First, Alibaba deploys a machine translation system and posts it online. A large number of users use machine translation. Alibaba processes user-preferred data to prepare corpora (including corpora from both the Internet and the crowdsourcing platform), optimizes the MT engine, and then conducts some automatic and human evaluation work to identify shortcomings. However, the improvements mentioned above are limited to the translation dimension. To achieve improvements in users' perspective, we need to deploy two system versions (A/B test) for comparison and post the better version online after the improvements get verified.
There are many challenges for machine translation in the e-commerce field, which may be different from the general fields. First, it has a very high requirement on target language text readability or smoothness. If the translation is very poor and can be hardly understood, users may lose the patience to continue reading. They'll go to other products instead. Nowadays, the neural networks MT has dramatically improved the translation smoothness, and can almost meet the high requirement on target language readability.
Second, translation for key messages must be accurate. Critical key messages include product names, the quantity for purchase, and numbers. If a brand name is incorrectly translated, we will get complaints from sellers. If the amount for purchase is incorrect, disputes between sellers and buyers will arise. You may have read this piece of news not too long before; the Norwegian Olympics team ordered 15,000 eggs by mistake when they only needed 1,500. This mistake successfully amused a lot of people around the world. Alibaba Translation must conduct exact translation on numbers and brand names. Also, Alibaba Translation also must have a prompt intervention mechanism to process violent and pornographic information. The above are challenges regarding translation quality.
Alibaba Translation also finds itself facing challenges regarding translation speed. First, Alibaba Translation must speed up to cope with the enormous quantity of commodities. For example, data traffic during the Double 11 Shopping Carnival is tremendous. Alibaba Translation must finish translating a sentence with 20-30 words in less than 100 ms. Imagine the translation speed cannot meet the requirement, say it provides a translation result for a phrase within 2 seconds, the buyer may lose the patience to wait for the result, and may turn to a similar product that does not require translation. The result is that, although the translation may be accurate, it cannot bring value to the seller.
In addition to translation quality and speed, there are also high requirements on service quality. Alibaba Translation must support highly available and flexible interfaces (for mobile phones, iPad, PC client, and other devices used for online shopping), translation from and into multiple languages, as well as efficient deployment and updating.
Alibaba Translation has taken several strategies for all these challenges. These strategies cover data, model, feature, and other aspects of machine translation.
At present, Alibaba Translation has maintained corpora of over 20 languages. For the most widely spoken language pairs, such as Chinese and English, the corpora have reached several hundred million sentences. Many other-language corpora such as French, have reached a level of tens of millions. The corpora for less widely spoken languages have also reached a level of millions. Alibaba has acquired Lazada, a significant e-commerce platform serving users in Southeast Asian countries, such as Indonesia and Thailand. Therefore, Alibaba Translation is working hard on providing support for Southeast Asian language pairs.
Alibaba Translation uses methods that get commonly used in academic circles (such as the IBM model and recurrent neural network-based (RNN-based) force decoding method) in rating the available data. Data with different quality levels get used in different application scenarios.
Also, the field-specific data relevance must get ensured. For data from general fields, corpora in the e-commerce field may not deliver the desired translation result. Furthermore, corpora for different products in the e-commerce field must also be separated. For example, it will be difficult to tell the meaning of the word "Apple" without context. You don't know whether it refers to a brand or a fruit. Therefore, the translation must get done in a field-specific manner. All machine translations are a result of data learning. Alibaba has also been using data selection methods that get used in academic circles, including information from the data source, subject model-based, voice model-based, and convolutional neural network-based data selection methods.
Because Alibaba Translation must accurately translate brand names, quantities, and other information of commodities, we need to build a multi-lingual knowledge graph. Based on Alibaba's knowledge graph (with approximately 10 billion data entries), Alibaba Translation has been enhancing its support for multiple languages. The primary work is to translate e-commerce information into Chinese, English, Russian, and other most widely spoken languages. This work is in progress and has not been wholly applied to the system yet.
MT has gone through several development stages: the rule-based MT stage (lasted for 2–3 decades), the statistical MT (SMT) stage (the 1990s–2014), and then the neural networks MT (NMT) stage started from 2014. If you attend academic meetings, you will scarcely see articles about RBMT or SMT. Does it mean that these two models have already been made obsolete? Alibaba Translation believes that these models have their advantages in specific scenarios. For example, although RBMT has not been used for many years, it can deliver excellent results and high accuracy when it has been configured with simple rules and used in concert with dictionaries to translate numbers, dates, addresses, and commodity-related information. Therefore, a part of Alibaba's translation system is rule-based.
Alibaba's MT systems also include SMT. Although NMT features terrific smoothness, SMT has its advantages in specific scenarios. For example, product titles are strings made up of independent phrases, and there is no order or logic between them. SMT can deliver outstanding results in this case. Also, when users search for a product, they search by entering one or two phrases. The translation result will be perfect if phrase-based SMT is used (most commonly used academic SMT).
Of course, Alibaba Translation has NMT systems and has developed an RNN-based seq-seq model. In 2017, Alibaba Translation launched a new model - Transformer. The NMT model has the advantage of smooth translation, with good order and logic. For example, when you use NMT to translate from English into Chinese, there will be no English sentence structures remaining in the Chinese sentence. It is suitable for use in strings of 20 to 30 words, such as product description, messages (communication between buyers and sellers), and buyers' review. Alibaba Translation would usually use two systems for comparison. Although Transformer's performance is usually better, it does not rule out the case that the RNN-based seq-seq model will have equally good performance. Human evaluation ratings and the testing results are the basis of the final selection.
We are the MT Lab under the DAMO Academy. Apart from work on the existing MT systems, we have a lot of researchers and engineers (with Master's Degree or Ph.D.) dedicated to innovative work concerning machine learning. A brief introduction is as follows.
The first is about neural inflection prediction. Chinese characters do not have singular or plural forms or tense changes. However, all these must get considered for English. English morphology is relatively simple in comparison with many other languages. For example, Russian. For the same Russian noun, when the stem stays unchanged, there can be dozens of different suffixes. Translating from Chinese or English into Russian will not be able to generate the required suffixes, because they do not exist in the source.
Many Russian people buy from Alibaba platforms, posing enormous demands for Chinese to Russian and English to Russian translation. To solve this problem, Alibaba Translation has made some achievements in inflection prediction, which divides Russian words into stems and suffixes. The source language has a Seq; the target has two. After predicting the stem (displayed on the left of the following figure), predict the suffix. Three types of information will get used at this time, which is respectively the source language information, the stem of the current target word, and the suffix of the previous target word. These three types of information can significantly improve the accuracy of inflection predictions. This achievement got published in a 2018 AAAI paper.
Another work is about translation intervention, provided that the MT system has translated key information accurately. However, it is challenging to conduct translation intervention in an NMT system as it does not translate word-by-word. Instead, it reads a sentence, comprehends it, and paraphrases it in the target language. In this case, some information will get missed.
There are still some bugs in this technology. Alibaba Translation extracts key information from a sentence and copies them into the target language. The COPY operation is simple and useful, but it only solves 80% of problems. In the e-commerce field, we must solve 99% or more problems. Via collaboration with external universities, we have approximately solved 95% of issues. We hope the successful intervention rate will eventually reach 99% or higher after further research is performed.
As we have mentioned before, the training corpora have reached a level of more a billion sentences. Using a single GPU for MT training is far from efficient. We need multiple machines and GPUs to divide data into multiple blocks. After each GPU finishes the training, it generates a model. Then we average the models and continue the training. In this way, multiple machines and GPUs can effectively increase the training speed.
Alibaba Translation's goal is to finish translating a sentence with 20–30 words within 100 ms, while many open-source platforms may currently need 1–2s. Google Translate uses TensorFlow code, while Alibaba Translation uses both Python code and TensorFlow code. Google Translate conducts computing in CPU, while Alibaba Translate performs computing in GPU. Alibaba Translation's strategy is simple but efficient. It is true that computing all the code in GPU will reduce GPU's service efficiency, but it increases the decoding speed.
We have done some memory optimization work; these are mainly engineering strategies.
We have previously mentioned that we can improve machine translation by using a knowledge base. Therefore, we have left an API in our MT system for the knowledge base. Because the knowledge base is still under construction, we are still primarily using glossaries and bilingual dictionaries. This is a project under cooperation with CAS's automation research team. Further achievements of this project will be integrated into our MT system as shown in the following figure.
Alibaba Translation has also done some multi-modal translation. Of course, the current requests are mainly text translation. At CES 2018, Alibaba Translation showcased a demo for voice translation and will launch an Alibaba voice translation system shortly. We are currently working on translation based on pictures, which requires cooperation with other internal teams.
It is true that machine translation is far from perfect, and its quality is not even close to professional human translation. If a technology is not perfect, can it be used? It depends on the scenario. You may not want to use machine translation to translate formal legal documents. First, it can get used in a cross-border e-commerce environment. When people view commodity information, they can tolerate some trivial issues. Machine translation can provide efficient services for cross-border e-commerce sellers, and bring quite some value to Alibaba.
Second, in the cross-border e-commerce scenario, there are still some particular challenges for machine translation, and we must continue our research. Third, RBMT, SMT, and NMT have their advantages in different scenarios. At last, we must consider quality, flexibility, stability, and other key factors.
Read similar articles and learn more about Alibaba Cloud's products and solutions at www.alibabacloud.com/blog.
Enabling Blue-Green Release with Alibaba Cloud Container Service for Kubernetes
2,599 posts | 762 followers
FollowAlibaba Cloud Community - March 22, 2022
Alibaba Clouder - October 31, 2019
Alibaba Clouder - November 27, 2019
Alibaba Clouder - September 19, 2018
Alibaba Clouder - July 22, 2020
PM - C2C_Yuan - April 16, 2024
2,599 posts | 762 followers
Follow
Platform For AI
A platform that provides enterprise-level data modeling services based on machine learning algorithms to quickly meet your needs for data-driven operations.
Learn More
Image Search
An intelligent image search service with product search and generic search features to help users resolve image search requests.
Learn More
Intelligent Service Robot
A dialog platform that enables smart dialog (based on natural language processing) through a range of dialog-enabling clients
Learn MoreMore Posts by Alibaba Clouder
