The study involved a specialized human evaluation of translated website content in seven language pairs. Professional linguists were recruited to conduct the evaluation.
Five major MT engines were used to generate the machine translations. These were Amazon Translate, DeepL, Google Cloud, Microsoft Translator, and ModernMT. Only the stock models were used, which means they weren’t trained on any additional data.
Rather than make a comparison with human-translated content, the evaluation focused on the overall usability of the machine translation, as well as accuracy and reliability.
The results are promising. The quality of machine translated samples were on average deemed to be good enough for practical use by the evaluators, with 85% being marked as acceptable or very good, and none marked as very bad. Very good means that the text segments rated didn’t need any editing, while acceptable means that they only required minimal editing, if at all.
This means that overall, the quality of machine translation is south africa mobile database now higher than it used to be, and is now at a point where people browsing a translated website have everything they need in order to navigate the site well, and to understand the content of the site.
The study does seem to have its limitations—in focusing on the criteria of usability, it’s hard to tell the likelihood of the website’s audience to take the action they want, like making a purchase. That is, whether usability translates to leads. Although going off the example of ecommerce giant eBay, which saw a 10% increase in sales through the availability of machine translation, the possibilities are also promising.
In any case, the study shows that the state of machine translation is getting better and better, and we’ve reached the threshold where it’s practical enough for certain consumer-facing use cases in the commerce sector.