Machine Generated Subtitle Quality Research

Over the past two decades, the development of digital technology has made audiovisual products an indispensable form of entertainment. Fansubbing, an online subtitling activity made by fans for fans, has shown that audiences are no longer passive products but are playing an increasingly active role in both the production and consumption of audiovisual materials (Pérez-González 2020: 105).

Fansub groups are an established phenomenon in China. They are groups of people that produce and publish subtitles for movies and TV programmes in foreign languages. Viewers are often not charged to watch. The translators themselves are frequently paid very little or nothing for their efforts.

The contribution of fansubbing to society has been given much credit by the general public and members of the National Committee of the Chinese People’s Political Consultative Conference. It was also highly praised for its free-sharing public spirit (Zhang 2013: 37).

Yet, some of the bigger subtitle groups do make money by placing advertisements within subtitled content or by relying on platform views. Because of this, a lot of subtitle groups use machine-translated subtitles after downloading a movie so that they can be the first channel to release the film and consequently receive more viewers and profit from it. Nevertheless, quality is another issue. When publishing the video, some subtitle organisations will indicate that it was machine translated, but others won’t. The viewing experience can nevertheless be impacted by evident inaccuracies, despite the fact that machine translation has significantly improved in recent years.

This blog will be a case study analysing the Chinese subtitle quality of the video: “How the British Took Over India” – TREVOR NOAH. The video is published on Bilibili, a Chinese video platform and marked as machine translated subtitle. The quality assessment model I will use to discuss the subtitles is the FAR model put forward by Pederson (2017). The FAR model assesses subtitle quality in three areas: Functional equivalence, acceptability and readability.

For the chosen video, the three aspects will be analysed below.

Functional Equivalence

The first area is functional equivalence, i.e., how well the message or meaning is rendered in the subtitled translation. Ideally, a subtitle would convey both what is said and what is indirectly meant. Especially for stand-up comedy, the humour can occur on different levels: it can arise from the interaction between word and image, or a play on words (Díaz Cintas & Remael,2007: 214–215). Translation of humour is challenging for humans, not to mention machines. The debate over whether it was presumptuous to add the word “great” to the country’s name, Great Britain, does not convey humour at all because it is primarily concerned with word-for-word translation.

Acceptability

The second area is the acceptability of the subtitles, i.e., how well the subtitles adhere to target language norms. The errors in this area are those that make the subtitles sound foreign or otherwise unnatural. Error types include grammar errors, spelling errors and idiomaticity errors. When referring to the god who appointed the queen in the video, the Indian representative asked “Which god,” which is translated as “哪位大神(which great god).” In China, the term “great god” is typically used to refer to someone who is an authority in a particular field. There is obviously a difference and sounds unnatural.

Readability

The third aspect is readability, or how simple it is for the viewers to understand the subtitles. Overall, the subtitle makes sense. The reader is able to follow along because lines are typically five characters long. It does, however, occasionally have odd punctuation and segmentation. For instance, between 0:37 and 0:42, there is a line ending with “突然” (sudden-) and the word “间” (-ly) begins the following line. Furthermore, from 1:47 to 1:49, there is merely one period mark. Despite not affecting comprehension, they do affect the viewing experience.

Concluding Remarks

This analysis shows that machine-translated subtitles generally make sense when the video is viewed for entertainment purposes. Functional equivalence, acceptability, and readability are all three areas where it has errors. These subtitles are unreliable and should only be used for casual watching. The subtitles, however, must be of a better calibre if the movie is to be educational. Therefore, the moral responsibility should fall on the subtitling groups. They must carefully consider the purpose of subtitles as well as which types of media content are suitable for machine translation.

References:

Díaz Cintas, J., & Remael, A. (2007). Audiovisual translation: Subtitling. St. Jerome. doi: https://link.springer.com/book/10.1057/9781137290342

Pérez-González, L. (2020). “From the ‘Cinema of Attractions’ to Danmu: A Multimodal Theory Analysis of Changing Subtitling Aesthetics across Media Cultures.” In Monica Boria et al (eds). Translation and Multimodality: Beyond Words, London and New York: Routledge, pp.94-116.

Pederson, J. (2017), “The FAR model: assessing quality in interlingual subtitling”, The Journal of Specialised Translation, 28, pp. 210-229

Zhang, X. (2013), “Fansubbing in China”, Multilingual, pp.30-37

Guest post by Yuxiang Yuan.