Brown Bag Seminar: Algorithm-based facial expression analysis

In this talk, we present a research method that enables the automatic analysis of facial expressions from video recordings. In 2023 HSSH organized a workshop on the method that inspired its use in the context of language education and language assessment. We give two examples on how the method has been used in the Aasis project (Research Council of Finland 2023 – 2027) developing ways to automatically measure and assess verbal and nonverbal features of spoken interaction in L2 Finnish.

We briefly present a manual (Juselius et al. 2026) introducing the use of algorithm-based facial expression analysis in research that was published for researchers at the University of Helsinki City Centre Campus. The development of the manual was supported by HSSH’s Catalyst Grant funding. We will also explain what we have learnt about the limitations of the method, and what new possibilities the software released this summer, Py-Feat Live, offers (https://py-feat.org/pages/pyfeat_live/). Finally, we provide guidance to researchers on how to book Interlab, which is located at Metsätalo.

The method offers new opportunities for analysing interaction and interview data, for example. It can also be used in research on online work, such as examining users’ facial expressions, depending on the type of video material the researcher has collected and the research questions the data are intended to address. With the help of the manual, University of Helsinki researchers can independently use the Py-Feat software to analyse videos recorded, for example, in Interlab (bookings via UH’s Outlook calendar).

Py-Feat enables algorithm-based analysis of facial expressions and can therefore complement researchers’ methodological toolkit, especially when the research focuses on interaction, reactions, or nonverbal communication using video data. Please let us know if you would like to take part in the workshop we are planning for late autumn.

Anna von Zansen, is a University Researcher working for two Academy Projects, Aasis (Research Council of Finland 2023 – 2027) and DigiTala in action (2025 – 2026) at the Faculty of Educational Sciences, University of Helsinki. Her research interests include educational technology, language assessment, multimodality, L2 listening and speaking, spoken interaction.

Joona Juselius, is a Research Assistant at the Faculty of Educational Sciences, University of Helsinki. He worked with funding from HSSH (Catalyst Grant 2026) and produced a manual on Py-Feat to support researchers at the city centre campus. He works in Anna von Zansen’s team and uses the method in his master’s thesis, majoring in English.

Mariel Wuolio, is a Project Planner at the Helsinki Institute for Social Sciences and Humanities. She works as the coordinator of the interaction research laboratory Interlab, administered by HSSH, being responsible for the laboratory’s services and user guidance.

The Methodological unit at the Helsinki Institute for Humanities and Social Sciences (Uni Helsinki) organizes a recurrent Brown Bag Seminar to highlight novel methodological approaches in the humanities and social sciences. The idea of the meetings is to introduce methodological innovations and cutting-edge research in various disciplines in an easily accessible manner and have an interdisciplinary discussion in an easy-going atmosphere over lunch.

There will be a 20-minute introduction to the methodological theme, followed by an open discussion of 40 minutes. The seminars are open to everybody and welcome a multidisciplinary and methodologically curious audience. The language of the meetings can be Finnish or English.

This is a hybrid event taking place at the University of Helsinki Main Building, Fabianinkatu 33, 3rd floor, room F3010. For remote participation, use this link.

More information is available on the event site.

References:

Juselius, J., Henttonen, P., Wuolio, M., & von Zansen, A. (2026). An addition to researchers’ toolkit: Algorithm-based facial expression analysis in L2 education (v1.0).  Zenodo. https://doi.org/10.5281/zenodo.19812192