Batch translations via Google Translate

Slightly related to my previous post, one of my tasks today was to translate a list of keywords into different languages. The inputs I received generally looks something like below. There were many more, but I truncated the bulk of it to provide a simpler example:

- Institutions: disputes, grievance, court, penal,  judicial, prosecutorial, punishment, lawyer, attorney
- Infrastructure: railways, traffic, trucks, cars, vehicles, road, taxi, public transportation, port, logistics, collision, passenger

Reshape the text into a processible format

Firstly, I need to transform these texts into a better format. A good text editor certainly helps. I was able to quickly convert these texts into csv format, using Emacs’ macro functionality and Vi’s keybindings.

The functionalities provided by good text editors really shine when small, spontaneous text manipulations like this is involved. Sure, when text inputs are big and repetitive, you should write a script to handle it, but for me, these small-to-average operations on texts are the bulk of my work.

Using googletrans to automate translations

For automation of translations, a quick search pointed me to the module googletrans (PyPI repo). Here is the disclamer from the package’s author:

DISCLAIMER: this is an unofficial library using the web API of and also is not associated with Google.

Thus, the package doesn’t work perfectly. On my first few attempts, the API refused to translate many words and returned them untranslated. I ended up write a custom wrapper function around googletrans.Translator.translate() that re-request the translation if it fails.

The final script looks something like this:

#!/usr/bin/env python3

import time
import pandas as pd
from googletrans import Translator

kwd = pd.read_csv("keywords-en.csv")

langs = ["ar", "id", "ml", "bg", "ca", "zh-CN", "zh-TW", "cs", "da",
         "nl", "fi", "fr", "de", "hu", "it", "ja", "ko", "no", "pl",
         "pt", "ru", "sk", "es", "sv", "th", "tr", "vi"]

def ggtrans(text, src, dest):
    """Translate `text` from `src` to `dest` language"""
    while True:
            trans = Translator()
            result = trans.translate(text, src = src, dest = dest).text
            print("Translated: {:>20} -> {:>20}".format(text, result))
        except Exception as e:
            print("Error: {} from {} to {}".format(text, src, dest))

for lang in langs:
    kwd[lang] = kwd['Keyword'].apply(ggtrans, src = "en", dest = lang)
    kwd[lang+"-retrans"] = kwd[lang].apply(ggtrans, src = lang, dest = "en")

print("Writing to CSV")
kwd.to_csv("translations.csv", index = False)

Admittedly, the script may take a long time to finish, because it sends one (or several) new request to the web API of for each and every translation it performs. But only then I can make sure that all of my words are properly translated.