From Disabilities to Capabilities: testing subtitles in immersive environments with end users

Contenido principal del artículo

Belén Agulló
Anna Matamala
Pilar Orero


User testing in media accessibility has often profiled users based on their disabilities: subtitles for the deaf and hard of hearing, for instance, have been generally tested with their expected target audience, which is deaf and hard-of-hearing users. This article argues that selecting users based on sensorial disabilities may not be the best strategy to obtain relevant results, as other capabilities—for instance, technological capabilities—may have a greater impact on the results. Moreover, the article argues that access services should not be exclusively aimed at persons with disabilities but can actually benefit all audiences. If accessibility is mainstreamed, and ideally integrated in the creation and production processes, testing should expand from an exclusive approach based on accessibility to a more general approach based on usability in which users with diverse capabilities are considered.

To illustrate this point and propose a new approach to user testing in media accessibility, in which we would move from a disability to a capabilities model, specific examples from the EU funded project ImAc (Immersive Accessibility) project are shown in a chronological order. Then, the article presents the initial testing, targeting persons with disabilities, and describes the poor data results leading to a new approach. We propose a new testing focus and justify the methodological shift. Then, we describe the second test in which the new approach is implemented, using the same stimuli but different able technology users. The article finishes with conclusions and final remarks in which we open the door to a move from an accessibility approach to testing to a usability approach.

Detalles del artículo

Biografía del autor/a

Belén Agulló, Universitat Autònoma de Barcelona

Belén Agulló is a predoctoral researcher in the Department of Translation, Interpreting and Eastern Asian Studies at the Universitat Autònoma de Barcelona, where she is working on the Horizon 2020-funded project Immersive Accessibility (ImAc). Her PhD focus is subtitling for the deaf and hard-of-hearing in immersive media. Previously, she worked for more than 5 years in the game localisation industry. Her research interests include audiovisual translation and media accessibility.

Anna Matamala, Universitat Autònoma de Barcelona

Anna Matamala, BA in Translation (UAB) and PhD in Applied Linguistics (UPF), is an associate professor at the Universitat Autònoma de Barcelona. Currently leading the TransMedia Catalonia group, she has participated and led projects in audiovisual translation and media accessibility. She has taken an active role in the organisation of scientific events (M4ALL, ARSAD), and has published in journals such as Meta, Translator, Perspectives, Babel, Translation Studies. She is currently involved in standardisation work.

Pilar Orero, Universitat Autònoma de Barcelona

Pilar Orero (PhD, UMIST) is a full professor at the Universitat Autònoma de Barcelona (Spain). She is a world-leading scholar in media accessibility with vast experience in standardisation and policy-making. She is a scientific/organizing committee member of many conferences, including Media4AllARSAD, and Video Games for All. She has delivered, upon invitation, more than 15 plenary lectures and 30 guest lectures all over the world, including at the 9th United Nations Conference of the States Parties to the Convention on the Rights of Persons with Disabilities (New York, 2016).

She either coordinated or participated in more than 40 national and international research projects, of which more than 20 were on media accessibility. She has published over 70 papers and edited 10 books on Media Accessibility. Her works are some of the most widely-cited publications in the field of which she is one of the founding scholars. Among her publications: A. Matamala and P. Orero (eds), Researching Audio Description (Palgrave Macmillan 2016); A. Maszerowska, A. Matamala and P. Orero (eds), Audio Description. New perspectives illustrated (John Benjamins 2014); C. Mangiron, P.  Orero and M. O'Hagan (eds), Videogame Localisation and Accessibility: Fun for All (Peter Lang 2014); A. Remael, P. Orero and M. Carroll (eds), Media for All: Audiovisual Translation and Media Accessibility at the Crossroads (Rodopi 2012); E. Franco, A. Matamala and P. Orero, Voice-over Translation: An overview (Peter Lang 2010); A. Matamala and P. Orero (eds), Listening to Subtitles. Subtitles for the Deaf and Hard of Hearing (Peter Lang 2009); J. Díaz Cintas, P. Orero and A. Remael (eds), Media for all. Subtitling for the Deaf, Audio description and Sign Language (Rodopi 2007), P. Orero (ed.), Topics in Audiovisual Translation (John Benjamins 2004).


Asociación Española de Normalización y Certificación (AENOR) (2003). Subtitulado para personas sordas y personas con discapacidad auditiva. Subtitulado a través del teletexto (Norma UNE 153010). Retrieved from:

Arnáiz-Uzquiza, V. (2008). La Objetividad en el Subtitulado. Justificación de los Parámetros Formales Mediante EyeTracking. In A. Pérez-Ugena, and R. Vizcaíno (Eds.), ULISES: Hacia el desarrollo de tecnologías comunicativas para la igualdad de oportunidades. Retos y perspectivas para sordos signantes (pp. 73-85) Madrid, España: Observatorio de las Realidades Sociales y de la Comunicación.

Arnáiz-Uzquiza, V. (2012). Subtitling for the deaf and the hard-of-hearing: Some parameters and their evaluation (Doctoral thesis). Universitat Autònoma de Barcelona, Spain. Retrieved from:

Bartoll, E. (2004). Parameters for the classification of subtitles. In P. Orero (Ed.), Topics in Audiovisual Translation (pp. 53-60). Amsterdam, Netherlands and Philadelphia, USA: John Benjamins.

Bartoll, E. (2008). Paràmetres per a una taxonomia de la subtitulació (Doctoral thesis). Universitat Pompeu Fabra, Spain. Retreived from

Bartoll, E. (2012). La subtitulació: Aspectes teòrics i pràctics. Barcelona, Spain: Eumo.

Bartoll, E., and Martínez-Tejerina, A. (2010). The positioning of subtitles for the deaf and hard of hearing. In A. Matamala, and P. Orero (Eds.), Listening to Subtitles: Subtitles for the Deaf and Hard of Hearing (pp. 69-86). Bern, Switzerland: Peter Lang.

Bernard, M., Chia, H. L., and Mills, M. (2001). The effect of font type and size on the legibility and reading time of online text by older adults. In CHI EA’01 Extended Abstracts on Human Factors in Computer Systems (pp. 175-176). New York, USA: ACM.

Brown, A. and Patterson, J. (2017). Designing Subtitles for 360° Content. In BBC Research & Development blog. Retrieved from

Brown, A. (2017). User Testing Subtitles for 360° Content. In BBC Research & Development blog. Retrieved from

Brown, A., Turner, J., Patterson, J., Schmitz, A., Armstrong, M., and Glancy, M. (2017). Subtitles in 360-degree Video. In TVX '17 ACM International Conference on Interactive Experiences for TV and Online Video (pp. 3-8). New York, USA: ACM.

Brown, A., Turner, J., Patterson, J., Schmitz, A., Armstrong, M., and Glancy, M. (2018). Exploring Subtitle Behaviour for 360° Video. [White Paper]. Retrieved from

Brown, S., and Lent, R. (eds.) (2008). Handbook of Counseling Psychology. New Jersey, USA: Wiley.

Díaz-Cintas, J., Orero, P., and Remael, A. (Eds.) (2007). Media for all: subtitling for the deaf, audio description, and sign language. Amsterdam, Netherlands: Rodopi.

Dubois, J. L., and Trani, J. F. (2009). Extending the capability paradigm to address the complexity of disability. ALTER, European Journal of Disability Research, 3, 192–218.

D’Ydewalle, G., Pollet, J., and van Rensbergen, J. (1987). Reading a message when the same message is available auditorily in another language: the case of subtitling. In J.K. O’Regan, and A. Lévy-Schoen (Eds.), Eye Movements: From Physiology to Cognition (pp. 313-321). North Holland, Netherlands: Elsevier Science Publishers.

D’Ydewalle, G., Praet, C., Verfaillie, K., and van Rensbergen, J. (1991). Watching Subtitled Television: Automatic Reading Behaviour. Communication Research, 18(5), 650-666.
D’Ydewalle, G., and Gielen, I. (1992). Attention Allocation with Overlapping Sound, Image, and Text. In K. Rayner (Ed.), Eye Movements and Visual Cognition (pp. 415-527). New York, USA: Springer Verlag.

D’Ydewalle, G., and Pavakanun, U. (1997). Could enjoying a movie lead to language acquisition? In P. Winterhoff-Spurk and T.H. van der Voort (Eds.), New horizons in media psychology (pp. 145–55). Opladen, Germnay: Westdeutscher Verlag.

D’Ydewalle, G., and van de Poel, M. (1999). Incidental foreign language acquisition by children watching subtitled television programs. Journal of Psycholinguistic Research, 28, 227–244.

De Bruycker, W.Y., and d’Ydewalle, G. (2003). Reading Native and Foreign Language Television Subtitles in Children and Adults. In J. Hyöna, R. Radach, and H. Deubel (Eds.), The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research (pp. 671-684). Amsterdam, Netherlands: Elsevier Science Publishers.

Di Giovanni, E., and Gambier, Y. (Eds.) (2018). Reception Studies and Audiovisual Translation. Amsterdam, Netherlands: John Benjamins.

Ellis, G. (2016). Impairment and Disability: Challenging Concepts of “Normality”. In A. Matamala and P. Orero (Eds.), Researching Audio Description (pp. 35-45). London, UK: Palgrave Macmillan.

Greco, G. M. (2016). On Accessibility as a Human Right, with an application to Media Accessibility. In A. Matamala, and P. Orero (Eds.), Researching Audio Description. London, UK: Palgrave Macmillan.

Hu, Y., Kautz, J., Yu, Y., and Wang, W. (2015). Speaker-Following Video Subtitles. ACM Transactions on Multimedia Computing, Communications, and Applications, 11(2).

Hughes, C. J., Armstrong, M., Jones, R., and Crabb, M. (2015). Responsive Design for Personalised Subtitles. In W4A '15 Proceedings of the 12th Web for All Conference. New York, USA: ACM.

International Organization for Standardization (forthcoming). Information technology -- User interface component accessibility -- Part 23: Visual presentation of audio information (including captions and subtitles) (Standard no. ISO/IEC FDIS 20071-23). Retrieved from:
International Telecommunication Union (ITU) (2017). Question 7/1: Access to telecommunication/ICT services by persons with disabilities and with specific needs [Report]. Retrieved from

Lambooij, M., Murdoch, M. J., Ijsselsteijn, W. A., and Heynderickx, I. (2013). The impact of video characteristics and subtitles on visual comfort of 3D TV. Displays 34(1), 8-16.

Linder, G. (2016). ACCESS SERVICES PAN EUROPEAN SURVEY 2016 [Report]. Retrieved from:

Marks, D. (1997). Models of Disability. Disability and Rehabilitation, 19(3), 85-91.

Matamala, A., Orero, P., Rovira-Esteva, S., Casas-Tost, H., Morales, L. F., Soler-Vilageliu, O., Agulló, B., Fidyka, A., Segura, D. and Tor-Carroggio, I. (2018). User-centric approaches in Access Services Evaluation: Profiling the End User. In I. Schuurman, L. Sevens, V. Yaneva, J. O’Flaherty (Eds.), Proceedings of the LREC 2018 Workshop. “Improving Social Inclusion using NLP: Tools, Methods and Resources” (pp. 1-8). Miyazaki, Japan: ISI-NLP 2.

Miquel-Iriarte, M. (2017). The reception of subtitling for the deaf and hard of hearing: viewers’ hearing and communication profile & Subtitling speed of exposure (Doctoral thesis). Universitat Autònoma de Barcelona, Spain. Retrieved from:

Mitra, S. (2006). The Capability Approach and Disability. Journal of Disability Policy Studies, 16(4), 236 – 247.

Neves, J. (2007). There is Research and Research: Subtitling for the Deaf and Hard of Hearing (SDH). In C. Jiménez (Ed.), Traducción y Accesibilidad. Subtitulación para sordos y audiodescripción para ciegos: Nuevas modalidades de Traducción Audiovisual. Frankfurt, Germany: Peter Lang.

Neves, J. (2008). 10 fallacies about Subtitling for the d/Deaf and the hard of hearing. The Journal of Specialised Translation, 10, 128-143.

Oncins Noguer, E. (2014). Accessibility for the scenic arts (Doctoral thesis). Universitat Autònoma de Barcelona, Spain. Retrieved from:

Orero, P. (2008). Three different receptions of the same film. The Pear Stories applied to Audio Description. European Journal of English Studies 12 (2), 179–193

Orero, P., and Matamala, A. (2016). User-centric audio description: a topsy-turvy research approach. In A. Manco, and A. Mancini (Eds.), Scrittura brevi: segni, testi e contesti. Dalle iscrizione antiche ai tweet (pp. 376-387). Naples, Italy: Università degli Studi di Napoli "L'Orientale”.

Orero, P., Doherty, S., Kruger, J.L., Matamala, A., Pedersen, J., Perego, E., Romero-Fresco, P., Rovira-Esteva, S., Soler-Vilageliu, O., and Szarkowska, A. (forthcoming). Conducting experimental research in audiovisual translation (AVT): A position paper. The Journal of Specialised Translation, 30.

Orero, P., and Tor-Carroggio, I. (forthcoming). User requirements when designing learning e-content: interaction for all. In E. Kapros and M. Koutsombogera (Eds.), Designing for the User Experience in Learning Systems. Switzerland: Springer International Publishing.

Patel, S. (2016). 85 percent of Facebook video is watched without sound. In DigidayUK. Retrieved from

Perego, E., Del Missier, F., Porta, M., and Mosconi. M. (2010). The Cognitive Effectiveness of Subtitle Processing. Media
Psychology 13(3), 243–272.

Pereira, A. (2010). Criteria for elaborating subtitles for the deaf and hard of hearing adults in Spain: Description of a case study. In A. Matamala, and P. Orero (Eds.), Listening to subtitles (pp. 87-102). Vienna, Austria: Peter Lang.

Prensky, M. ( 2001). Digital Natives, Digital Immigrants. On the Horizon. 9 (5), 1–6.

Romero-Fresco, P. (2009). More haste less speed: edited vs. Verbatim respeaking. Vigo International Journal of Applied Linguistics, 6, 109–133.

Romero-Fresco, P. (2010). D’Artagnan and the Seven Musketeers: SUBSORDIG travels to Europe. In A. Matamala, and P. Orero (Eds.), Listening to subtitles (pp. 175-189). Vienna, Austria: Peter Lang.

Romero-Fresco, P. (2013). Accessible filmmaking: Joining the dots between audiovisual translation, accessibility and filmmaking. The Journal of Specialised Translation, 20, 201-223.

Romero-Fresco, P. (ed.) (2015). The Reception of Subtitles for the Deaf and Hard of Hearing in Europe. Bern, Switzerland: Peter Lang.

Szarkowska, A., Krejtz, I., Kłyszejko, Z., and Wieczorek, A. (2011). Verbatim, standard, or edited? Reading patterns of different captioning styles among deaf, hard of hearing, and hearing viewers. American Annals of the Deaf, 156(4), 363-378.

Szarkowska, A., Krejtz, I., Pilipczuk, Dutka, L., and Kruger, J.L. (2016). The effects of text editing and subtitle presentation rate on the comprehension and reading patterns of interlingual and intralingual subtitles among deaf, hard of hearing and hearing viewers. Across Languages and Cultures, 17(2), 183-204

Tsaousi, A. (2017). Sound effect labelling in Subtitling for the D/deaf and Hard-of-Hearing: An experimental study on audience reception (Doctoral Thesis). Universitat Autònoma de Barcelona, Spain. Retrieved from:

Tullis, T., and Albert, B. (2013). Measuring user experience. Collecting, analyzing, and presenting usability metrics. USA: Morgan Kaufmann.

Udo, J. P., and Fels, D. I. (2010). The rogue poster-children of universal design: closed captioning and audio description. Journal of Engineering Design, 21(2-3), 207-221.

Utray, F., Ruiz B., and Moreiro, J.A. (2010). Maximum font size for subtitles in Standard Definition Digital Television: Tests for a font magnifying application. In A. Matamala, and P. Orero (Eds.), Listening to Subtitles: Subtitles for the Deaf and Hard of Hearing (pp 59-68). Bern, Switzerland: Peter Lang.