Month: May 2017

Workflow of the DTP final project: Localizing a movie trailer

May 16, 2017 / Wanxin Li / 0 Comments

Introduction

For the DTP final project during the 2017 spring term, I localized a one-minute trailer of a Disney movie called Moana into Chinese. I did dubbing and recreated the audio with Chinese lines, similar sound effects to the original audio file and the Chinese version of the background song using Adobe Audition CC 2017. Finally, I added subtitles for the on-screen texts in the video and integrated the video and the audio file I created using Adobe Premiere CC 2017.

The original video can be watched here: https://www.youtube.com/watch?v=mxhI4sh85wc

The video that I have localized into Chinese can be watched here: https://www.youtube.com/watch?v=EzSLQLRaqEw

Process of the project

The process of this project consists of five steps, including preparation, dubbing, audio editing, integration and subtitling for on-screen texts. Details of each step are as follow:

Preparation

First of all, I wrote a spreadsheet as shown in the screenshot below, listing the background music, all the sound effects, English lines and their Chinese translations in the order of time sequence according to the video. Because recreation of the audio means I have to make sure all the sounds could perfectly match the video, I recorded their time duration in the third column, and “V” means their starting and ending time in the video. This made the audio editing process much easier and faster.
Then, I searched all the needed sound effects and the Chinese version of the background song on YouTube. I at first tried to find the resources on free online sound effects libraries such as org, but later I realized that there were so many special sound effects in the video like the sound of a lightsaber, the sound when the spotlight is on, and the sound of monster roaring which I wasn’t able to find, so I ended up searching them on YouTube.
After that, I converted all the sound effect YouTube videos into .mp3 with the help of a website named OnlineVideoConverter. I simply just pasted the YouTube video links to this website, converted them into mp3 audio files with the quality of 320 kbps and downloaded them. I listened to the files again and again and recorded which parts in the sound effect audios were needed, which is shown in the “S” in the “Time Duration” in the screenshot above.

Dubbing

Because there’s a male character in the video, I asked a classmate to dub for the character. We then recorded our voices at the school DLC recording booth using the tool Camtasia. We recorded one Chinese line after another while watching the original video to make sure we speak at the same speed as the original voices. Everything including the lines and our discussions was recorded into one audio file, but there was a few seconds’ pause before we said our lines, which made the audio editing less painful.

Audio Editing

After all the audio resources had been found or recorded, I created a new project in Adobe Audition CC 2017 and imported all the audio files. I used the razor selected clips tool to cut the audio clips into the length I wanted and placed them on multiple tracks if some of the sounds had to be overlaid. With the reference of the spreadsheet written in the preparation step, it was much easier to place the audio clips accurately. This step was not tough but it really was time consuming and required me to listen to the tracks while watching the video over and over again. Finally, I adjusted the volume of some audio clips because some of the sound effects were too loud and I hoped to lower the volume of the background music when there was a human voice. I mainly used two approaches as illustrated in the screenshot below:

When I wanted to adjust the volume of the whole track, I simply just changed the dB number under the track. When I only wanted to adjust the volume of a single clip when there was more than one clip on the same track, I set up key frames in that audio clip, which is very similar to the way we’ve learned in adjusting the opacity in Adobe After Effects. Then I raised or lowered the yellow bar to adjust the volume. This is the final look of all the audio clips placed correctly in Audition:

Integration

After the audio editing work had been done and everything sounded good enough, I exported the entire session of the multitrack into one single audio mp3 file. Then I imported both the video file and the audio file I created into Adobe Premiere CC 2017 and placed them on different tracks. As you can see here, the second track is the original audio file which comes with the video. I muted it because it couldn’t be deleted, but this didn’t affect what the final product sounded like.

Subtitling for on-screen texts

After the integration, I added still subtitles for the on-screen texts also in Adobe Premiere. First I moved the pointer to where there was an on-screen text, and under the “Title” tab, I chose “New Title-Default Still”. Then a window popped up. I created a new text box, typed in the Chinese translations, chose a perfect gradient font, adjusted the size of the font, placed the text box under the original texts and used the eyedropper to change the color of the texts.

After all the subtitles were created, they would appear on the right of the interface.

I dragged each of them to another track. Like what I did in the audio editing step, I adjusted the length of each subtitle clip to make sure they appear and disappear simultaneously with the original on-screen texts. Last but not least, I exported the media.

Challenges

There are two challenges during the process of this project:

It was very hard to find the resources of sound effects because I didn’t know how to describe the sounds, such as the sound when the male character transformed from an eagle to his human form, and the sound when the male character’s weapon glowed little by little. I ended up searching on YouTube with every word I could use to describe the sounds and luckily found all of the similar ones in the end.
Editing the audio clips was so time consuming. Since most of the sound effects appeared for only a very short time in the video like the sound of sword unsheathing, the head hitting the wood, and whoosh, it was hard to edit the clips and placed them correctly to make sure the sound effects exactly match the video.

Lessons Learned

I’ve been able to keep passion to make my product perfect throughout the whole project because first of all, I love doing jobs related to sounds, to music, and secondly, I love the movie I chose. For a self-selected topic, I strongly suggest choosing something you really enjoy.

There are two things I’ve learned from this project:

Writing a spreadsheet and pinning the time of every sound appears in the video really helps. It can make the step of audio editing faster and much easier.
If possible, it’s better to record the lines and save them respectively into separate audio files. This helps enhancing the efficiency of audio editing because the sounds that are not needed in the video don’t have to be reviewed and edited.

Conclusion

In a word, I’ve really learned a lot during this project, especially from the classmates who also chose to do dubbing. I’ve gained new skills of using Adobe Audition and Adobe Premiere, and most importantly, I’ve now had a clearer picture of the process DTP localization.

Comparisons Between KantanMT And Microsoft Translator Hub

May 10, 2017 / Wanxin Li / 0 Comments

The topic of my group’s final MT project is about IMF world economic outlooks. To figure out which engine, KantanMT or Microsoft Translator Hub, is better for training machine translation on a topic like this, I’ve made comparisons between the two from different aspects and the results are as follows:

Comparison of the advantages and disadvantages of KantanMT and Microsoft Translator Hub

Comparison of BLEU scores

In order to make an objective comparison between KantanMT and Microsoft Translator Hub, I fed the same training, tuning and testing data respectively to the two engines. There is a huge gap between the BLEU scores of the two: the score Microsoft Translator Hub gains for the machine translation is 44.03, while KantanMT only 21. This means that the machine translation generated in Translator Hub is closer to the testing reference than KantanMT, while there’s no guarantee that the quality of the MT in the former is definitely higher than the latter. According to the instructions on the KantanMT website, a BLEU score lower than 39 stands for no fluency. Absolutely ungrammatical and for the most part doesn’t make any sense. Translation has to be re-written from scratch.

Comparison of human evaluated machine translations

To compare the quality of machine translations, I conducted a human evaluation based on the first 30 segments from the testing data according to the six error types our group has agreed on (including omission, mistranslation, untranslated, inconsistent with termbase, inconsistent with terminology and grammar).

There is actually also a huge gap between the results. For the MT generated in Microsoft Translator Hub, there are altogether 6 minor and 5 major grammar errors, 4 critical and 1 minor mistranslation errors and 1 critical omission error. For the MT generated in KantanMT, however, there are 6 critical, 2 major and 1 minor grammar errors, 19 critical untranslated errors, 1 critical omission error, 1 minor, 1 major and 3 critical mistranslation errors. The comparison is pretty obvious here, and the machine translation from KantanMT is indeed unreadable and influent as indicated by its instruction.

Comparison of features and benefits

However, even though the result of the machine translation generated from KantanMT is not ideal, the engine actually has many advantageous features that Microsoft Translator Hub doesn’t include:

There is more visibility of the backstage in KantanMT, letting the users know at what stage the machine training process is at certain point. This makes statistics-based machine training not mysterious.

It is possible to delete certain files that have already been uploaded. In some cases, for example, when adding certain files makes the BLEU score of the system go down, we may want to duplicate a new system and delete the files that cause the lower score. In Microsoft Translator Hub, however, there’s no way to delete the files and we could only uncheck the files instead, which is much more inconvenient.

There are many other indexes to evaluate the quality of machine translation than just BLEU scores, such as F-Measure scores, TER scores, and Gap analysis. This can provide multiple dimensions for the users to evaluate machine translation.

Simply for the BLEU scores, KantanMT also has really deep analyses. The BLEU scores are not only applied to the whole translation as Microsoft Translator Hub, they are also used for every one of the segments. This provides a much clearer and more detailed view of the quality of each segment.

In KantanMT, the users can also set up their own KPI indexes designed for specific projects, which is very customized and personalized. The reviewers can then evaluate the machine translation online based on those KPI indexes. Everything is organized and convenient here.

But KantanMT also has some disadvantages that Microsoft Translator Hub can complementary to:

There’s no “Tuning” and “Testing” tabs on the system training page. All the data files, including tuning, testing and training files all have to go under the “Training” tab, which may cause confusion.
TMX, PDF and other document types are not accepted as tuning or testing data in KantanMT, except Excel files and aligned UTF8 encoded text files, which is not convenient for aligning segments and storing translation memories.

In our group’s case, because all the files have already been aligned into TMX files, we use Olifant to convert the TMX files into Excel files. The converted file looks awful, and many of the sentences seem to disappear or be replaced by unreadable codes. For example, the original English sentence should look like this:

But it looks like this instead in Olifant:

As the previous point shows, the naming of the testing and tuning data should be exactly the same as the instruction, i.e., “test.reference.set.xlsx”. Our group tried several times and then we figure out that if the file names don’t follow the naming rules, the engine would not recognize the files as testing or tuning data and would use its automatic tuning and testing data instead. This is extremely inconvenient for group projects since in this way, extra integration work has to be done after all the team members finish their own.

Recommendation for the engine suitable for training

By balancing all of the pros and cons of KantanMT and Microsoft Translator Hub, I think Microsoft Translator Hub is better for training machine translation on the topic of IMF economic reports.

There are two reasons:

The quality of the MT in Microsoft Translator Hub is apparently much higher than KantanMT according to the human evaluation and the BLEU scores.
Since there are a large amount of tuning and testing files we have to upload, it’s extremely inconvenient to integrate all of them into one Excel file.

But there are still steps needed to make sure Microsoft Translator Hub is better for our machine training than KantanMT. As mentioned above, the tuning and testing data we fed to KantanMT are the Excel files converted from TMX files which are quite messy. This maybe the main reason why the BLEU scores and the quality of the machine translation in KantanMT are so low. So we still have to clean up the Excel files, make sure the materials are identical to the ones we imported to the Microsoft Translator Hub system. Then the system has to be trained again with the clean Excel files in KantanMT and the results have to be reevaluated. If the BLEU scores and the quality are still lower than those in Microsoft Translator Hub, then we can confidently say that Translator Hub is a better engine to use for continuing our project.

Advanced CAT Final Mini-Portfolio Introduction

May 3, 2017 / Wanxin Li / 0 Comments

This portfolio is a compilation of the Machine Translation final project files my team Machina completed in the course Advanced: Computer-Assisted Translation(CAT) during the 2017 spring term, and a blog post titled “Comparison Between KantanMT and Microsoft Translator Hub” based on our final project, all of which support my comprehension of machine translation training and my evaluation on the suitabilities of two different MT engines for training projects related to the IMF world economic outlooks.

With the final project files (which are basically files our team created when doing a machine translation project with Microsoft Translator Hub) including a pilot proposal, the presentation slides on lessons learned and an updated proposal), my portfolio chiefly explores the comparisons between two machine translation engine–Microsoft Translator Hub and Kantan MT and recommendations for using which machine translation when training engines related to the topics that our final project is on.

Kimi Li's Localization Wonderland

Never stop creating.