Category Archives: Course

Some may say that among programming, localization-related and linguistic skills, you can’t have everything. Picking all up seems almost impossible for them. Yet, Zach, Will and I decided to put that theory to a test. We would like to integrate the knowledge we have around Python to automate some of the processes we learned about localizing websites and apps.

To automate the process, we first need to remind ourselves what 24 ways i18n is about. We’ll give you a brief intro and if you’re interested in learning more about 24 ways, click this link to check out the full tutorial on 24 ways.

About 24 ways i18n method

So about 24 ways i18n. It’s a way for you to insert translations into your website in a more dynamic manner. You might wonder: Why do we need this so-called 24ways to insert our translation? Isn’t it as simple as find the string and replace them with our translation? Unfortunately, that’s not the case.

Remember, different languages have different formats. For example, let’s say we’re managing a recipe website and there’s a recipe for a simple sandwich:

1. 2 slices of toast
2. 1 slice of cheese
3. 10 slices of honey ham
4. Some butter
5. Some ketchup

In traditional Chinese, the recipe would be translated into:

1. 吐司 2 片
2. 起司 1 片
3. 蜂蜜火腿 10 片
4. 奶油少許
5. 番茄醬少許

Notice something? Yes the format seems a bit different. Structures of the two languages aren’t exactly the same. That’s why instead of blindly dumping translations into our website, we need to make sure we’re dynamically updating the translation according to the actual language structure. And that’s where 24 ways i18n shine. By using 24 ways, you’re not only making LSPs’ lives easier but also preparing your business to advance to a global level.

What do we actually achieve with Python?

Other than the main function, you’ll need:

• String.js (Storing the translation)
  (prototype ready)
• Re-formatting/replacing the string with the _() gettext method
  e.g. “RandomText” –> _(“RandomText”)
  (still in progress)

For the semester, Zach, Will and I decided to start things off by attempting automating the string extraction process, specifically pulling from Html/XML files. Here’s the prototype file we come up with, feel free to try it out along the way. In the file, you should find the “scratch.py” Python file we wrote and the “Sample.html” file we toyed around with.

Extract strings and store them in “String.js” file with BeautifulSoup package

BeautifulSoup is a powerful package for web developers these days to process website-related information. With BeautifulSoup, you can have Python reads HTML files, filters out tags as needed or finding strings within a specific type of tag.

Here’s a quick sneak peek of what our program looks like:

And here’s what the sample HTML looks like:

With the power of BeautifulSoup (Specifically, the “get_text()” and “prettify()” method), we successfully extract and store the string into our target – the “String.js” file. On top of that, we also provide our user with a “demofile1.txt” file to help them visualize what the file looks like without tags:

Along the way, we also realize that with BeautifulSoup comes a significant amount of possibilities. We’re merely scratching the surface here. Combining with packages like “urllib“, you’ll be able to pull info straight from the Internet!

Take a step further?

In our weekly discussions, we also have our focus on topics like execution of the program with bash scripts, and localization best practices such as duplicating source files in case of messing up. With the time and knowledge we have right now, we succeed in achieving some but not all. You can find out more about what we achieve in our Python program. They are stored as comments in the program since they don’t exactly contribute to our main goals – 24 ways or _() gettext method.

One thing worth mentioning is that we successfully incorporated some basic level of GUI(Graphical User Interface) features in our program. With the tkinter package, our users can run the program -> select the file via GUI -> get the same result! That sounds amazing, isn’t it?

Here’s the updated Python program.

What’s next?

It’s clear that we’re far from done, but as our graduation approaches, we have no choice but to wrap up what we have for now and perhaps pass on what we’ve achieved to our colleagues who might be interested in taking over our project next year.

Here are what we have achieved so far, open for improvement though:

• Basic GUI (tkinter)
• Strings extraction (BeautifulSoup)
• “String.js” file prep
• Duplicate files capability

And moving forward, some suggestions on what the goals could be for our future colleagues:

• Re-format the strings with the _()gettext method using Regex (perhaps like this? ” _\(“(.*)”\)” )
• Optimize the overall program, QA for potential bugs
• Come up with an i18n guide to help streamline the string extraction process
  (Some people build their website like building Frankenstein, thus makes it almost impossible to pull the strings properly)
• See if we can integrate everything we have and fully automate the 24 ways i18n process

Overall speaking, I have a blast working on this project. Plus, my colleagues, Zach and Will, are highly competent in working with Python. We’re able to exchange ideas and feedback all the time. With ideas flying around, we’re able to come up with possible workarounds that we might have never thought of ourselves. Eventually, it leads to the fruitful results we have here.

Here we have a video presentation to help you visualize what we’ve achieved today:

Link to our video presentation

Expanding on what we’ve learned in the Localization Project Management class, we get more hands-on experience about localization project management.

Hands-On Experience Recreating PDF File in Microsoft Word

As you may know, PDF files are not really compatible with CAT tools. Most PMs and linguists prefer working with Microsoft Word instead of PDF. CAT tools can’t really handle PDF files and usually the format of the exported files would be off.

In most cases, the client that requested the service should have the original file in Microsoft Word format because the PDF files are more often than not originated from Microsoft Word. But what if the client for some reason refuse to provide the original file or doesn’t have access to the Word document? Well, as a PM, you got to do what has to be done for the sake of the project – recreate the PDF in Microsoft Word.

In general, most PDF requested that needed recreations are forms or questionnaires. Format in these type of documents are must-have and can’t be neglected. There might not be many contexts within these type of documents, but the efforts invested are undoubtedly high.

Here’s one of my three hands-on experience with recreating PDF in Word:

Non-Disclosure Agreement (NDA) and Legal Documents

We also looked into the elements of the common legal documents within the industry and learn how to draft one ourselves.

NDA

NDA is a mutual agreement between at least two parties on which information is considered to be confidential, how are we going to protect the information and what is the possible consequences for breaking the NDA.

For our program, we have yet to establish an NDA for the incoming students. That’s the reason why we, together with our professor, decide to draft an NDA for the program to make the program even better.

In short, there are two major step to draft a NDA:

The first step would be identifying who the stakeholders are for this NDA. Obviously, we must have the student and school authorities. On top of that, we may also consider people like guest speakers to be part of the stakeholder as well because they share their knowledge with us.

The second step, which is the most important part, is to define what confidential information is and a brief of the consequence of breaking the deal. For this part, you need to have close discussions with the one another because this is the most crucial aspect of NDA.

Wanna check out our class’s final NDA? Click Here

Handoff in Real Life

We’re not perfect, sometimes you or your colleagues might not be able to complete the whole job. Family, health or even your work can be a part of the reason. That’s why sometimes we need to take over others’ work or pass our work to them. Which is exactly what “handoff” is all about.

“Handoff” refers to a state where you transfer the work to someone else who is willing or supposed to take over the task from where you left off.    

In the LPM class, we assume that we’re in a perfect world and everyone can dedicate themselves 100% to the project. In this class, we spice things up a bit. We assume that the team of the company is constantly switching their roles with one another, and they hardly work on the same project every week. To be honest, it was kind of intimidating at first, In certain cases, taking over others’ works can be disastrous. However, if you have systematic hands-off mechanisms inherited by all your colleagues, passing or taking over projects can be as smooth as butter.

What Defines A Good “Handoff”?

To perform a good handoff, we can start with pinpoint what exactly contributes to a so-called good handoff.

First off, an organized folder packed with clear directory to the work documents is a must-have. Who would want to deal with a mess in the first place?A messy structure intimidates peoples and would result in unnecessary waste of time and money due to the extra work of figuring things out.

Secondly, a streamlined process management system can be a saving grace to you and your colleagues. If everyone can keep track of their work in the system. People taking over can refer to the system for guidance pretty easily.

Lastly, which circle back to ourselves, be open for any query. Communication is the key. No matter how meticulous you are preping your handoff, there can always be, either obvious or unexpected, query from your colleagues. Try to be there when your teammates need you, this positive vibe can be a great addition to you and your team on the long run.

For more details of our work, please visit our “company’s website”

Final Big TakeAway – Drafting our own Code of Ethics for LPM

Code of Ethics is a series of guidances or principles for a certain business professions. In my opinion, drafting a code of ehics requires solid understanding of the industry to come up with these standards in it. With the lead of our professor and the review of our colleagues. Each one of us come up with our own code of ethics that reflects our visions and hopes for the industry.

I really enjoy getting to know more about this evolving industry. To me, the code of ethics is like a milestone of my career of LPM and I would like to share with everyone.

After acquire the basic knowledge of CAT tools such as SDL Trados, MemoQ, Memsource, etc. Let’s take a step further into some of the advanced features hidden within the system and some real-world practice related to CAT tools such as training machine translation engine.

Alright, let’s start with my experience training an Statistical Machine Translation (SMT) engine!

First, let’s clarify what Statistical Machine translation (SMT) is.

SMT is a type of machine translation that is based on a large amount of bilingual text corpora.

The system would first build up a translation model by analyzing the corpora, and then perform the translation task using the same model.

Me

The above is my general interpretation of what SMT is, you may click here if you want to learn more about SMT.

Now that we get that out of the way, time to discuss about the three fundamental elements of technology translation:

Fully automated, high quality and unrestricted and the three goals that we aim to achieve regarding technology translation including machine translation. For now, we’re not yet able to achieve the perfect goal but we’re able to achieve either two out of the three.

SMT focuses more on fully automated and high-quality aspects. Which means if we want to succeed in building a robust SMT engine, we need to work with a more controlled language such as manual descriptions or recipes. The context within those documents is usually repetitive which is ideal for SMT training.

Think about it this way, if you want to train a new cook to make Japanese cuisine, specifically sushi, for you. What would you do?

Are you going to tell him/her to read the Constitution to learn how to make sushi? No, because legal documents are less relevant to sushi. You probably will give him/her documents that are at least related to cooking such as recipes and guides. In that way, he/she can actually learn something about cooking and improve upon it.

After we settle down what the actual domain is, let’s figure out what type of cooking data should we be handing to the cook? American cuisine recipes, Chinese cuisine recipes or French cuisine recipes? Well, they might help the cook to get a better grip on general cooking techniques but still can’t efficiently educate our cook on how to make sushi.

You’re probably shouting at this point, “I get it. Give the cook Japanese cuisine recipes, specifically the sushi recipes!!!” That’s the spirit, but what if we take a step further and provide recipes written by the same master chef? Would our rookie cook be able to learn faster because of the similar formats and thought processes generated by the same individual? Pretty likely. Our rookie cook can definitely benefit from not needing to re-learn everything from the ground up and keep on populating the knowledge model without interruption.

There you have it. That’s exactly why we prefer working with more restricted text when training SMT engine so that we can get a higher BLEU score for our SMT engine.
*BLEU (bilingual evaluation understudy) is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another. There’s no set standard to tell if it’s good or bad, only if it’s relatively better or worst.

My group members Josue, Cody, Meng, and Chen-ye and I decided to train an en-us <-> zh-cn engine based on Sony PlayStation manuals, and the Microsoft Azura is the platform we used. First, we need to harness and separate our data into three different categories – Testing, tuning and training data.

• Training Data: Your engine is going to create a translation model based on these data. Minimum segments required is about 2000
• Tuning Data: Extra information that your engine can refer to in order to provide a more accurate translation. Roughly about 500 segments
• Testing Data: Data used to evaluate your engine. Think of it as a test with answers and training data as a student’s answer. Roughly 500 segments as well.

Due to the limits of available data and file formats, we decided to have PS4 manuals as testing data since it’s standardized, PS Vitas as tuning data cause it’s still relevant in some parts of the world and lastly PS TV as training data.

We’ve encountered issues such as google sheets not accepted by Azura and unclear error messages. We had to download our aligned data from google sheets and convert them into actual excel formats, otherwise, Azura is going to reject them. Also, we were constantly reaching out to Microsoft for possible solutions.

Here are our Final presentation and our finalized proposal for this pilot project.

CAT Tools Tips and others
(Regular expressions, Handy utility tools)

Project Fluent

In this class, we take a look at a localization system called project Fluent. Unlike traditional localization, translators can refer to the complete context directly, not just piecemeal context. Plus, there’s a dynamic live preview of the localized content in project Fluent.

In the past, people extract and separate the content for translation because translators might mess up the code. Or maybe the code is still under construction but the prompts are set. Either way, people now can have a different option to localize their content. Here’s a quick demonstration of me adding a basic language selector for the login prompt and a different level of your feeling toward the post.

Regular Expression

In your CAT tools QA system, you can input coding languages to catch errors regardless of the actual context. Regex101 is a handy website where you can test out your guide with quick references on your side.

Here are some samples I come up with for English(en-us) and my mother tongue traditional Chinese(zh-tw):

• \, -> \，:
  Change the half-width punctuation to full-width punctuation for traditional Chinese.
• (\d{2})\:(\d{2}) -> (\d{2})\點(\d{2})\分 :
  Change the time format into traditional Chinese format
• (\d{3})\-(\d{3})\-(\d{4}) -> \+1\((\d{3})\)(\d{3})\-(\d{4}) :
  Change the phone number format into traditional Chinese format.
• \$(\d{1,}) -> (\d{1,})\美元 :
  Change the currency format into traditional Chinese format.
• \S[^,]+\,? -> …$4 $3 $2 $1 :
  Change the address format into traditional Chinese format.

Utility tools

The tools we’re discussing here are not limited to localization and translation. In fact, the tools can be really useful for most people from different background.

Screen magnifier is a great example and one big takeaway for me in this course. Sometimes, when we’re giving a presentation or meeting, we might want to magnify part of the screen to explain our point more clearly.

For me, I decide to take a look at a tool name Renamer Portable. It’s an app that can rename multiple files at once instead of you clicking through each file one by one. Here’s a utility demo video I made for Renamer Portable.