Effect of gamification intervention
on children's speech rehabilitation
motivation
2022.9-2022.12 Children's hospital of Shanghai： Field study & expert interview
2023.2-2023.6 TU/e：Gamification Strategy research
2023.8-2023.12 SJTU：Software development & experimental research
Human computer interaction
Empirical study in HCI
Human-centered computing
Game Strategies
Imitation pronunciation
speech rehabilitation
Journal Paper (under review) contact me for more information
Assessment
Treatment
The three components of this study
Basic: Gamification Strategies Research
The increasing demand for medical services in hospitals has sparked interest in exploring alternative methods to support children with functional articulation disorders. Online speech games have emerged as a promising avenue to motivate children to engage in speech therapy.
This study investigates the impact of gamification strategies on children’s motivation within the context of speech rehabilitation games. Four distinct game prototypes were developed, with a specific emphasis on stimulating children’s motivation to speak. Two sets of experiments involving 48 participants were conducted to assess the influence of (1) time-limitation and (2) interactive imitation objects on children’s motivation.
The results revealed that time limitation significantly increased motivation, while the effect of imitation objects on motivation was not statistically significant.
These findings offer valuable insights into designing effective speech games for children. By leveraging gamification strategies in online speech games, we can address the motivation challenges faced by children with functional articulation disorders and potentially enhance the efficiency of speech therapy interventions.
(a) Static bubble
(b) moving bubble
Sub question 1：How the time limitation influences the motivation?
(c) Static bubble
(d) Static bubble + shell
Sub question 2：How an interactive imitation object impacts the motivation ?
Extension 1:
Speech collection software
Dysarthria is a common and treatable speech problem in children, and computer-assisted speech therapy is a promising way for children’s speech therapy. However, data collection poses a significant challenge for computer-assisted therapy, especially when it comes to gathering speech data from young children, particularly those with dysarthria. Finding a better way to collect young children’s speech data is, therefore, an urgent need.
This paper prompted a gamified speech collection method and carried out an experiment to compare the participation time, error rate, and collection efficiency with the gamified method and with a traditional method where adults are imitated. Moreover, we also explore whether the gamified collection methods increase the children’s positive engagement.
A feasibility study including 10 children with dysarthria and 10 children without speech problems was conducted. Their participation duration, number of spoken utterances, number of mispronunciation utterances, and a questionnaire about children's engagement attitude were recorded.
The findings indicate that the gamified collecting method reduces pronunciation mistake rates in children with dysarthria while also increasing their engagement to participate.
Design based on the speech collection workflow
real time segment and label through the game strategy
Extension 2:
Children's positive engagement on imitation pronunciation（Game guide VS Human guide）
For young children, speech assessment is very important for early detection of articulation problems and obtaining timely rehabilitation treatment. Both assessment and treatment rely on pronunciation imitation. Meanwhile, children’s engagement during pronunciation imitation significantly influences the effectiveness of these interventions.
This study aims to examine how gamified approaches affect children's engagement during pronunciation imitation when compared to traditional method with human therapists. Additionally, the impact of adding a visualization supplement to both (game and human-guided) methods on children's engagement was explored. Furthermore, a novel method was developed to estimate the children’s engagement stages based on their observed behaviors.
game description
visualization supplement
human guided method
Gamified method
For young children, speech assessment is very important for early detection of articulation problems and obtaining timely rehabilitation treatment. Both assessment and treatment rely on pronunciation imitation. Meanwhile, children’s engagement during pronunciation imitation significantly influences the effectiveness of these interventions.
This study aims to examine how gamified approaches affect children's engagement during pronunciation imitation when compared to traditional method with human therapists. Additionally, the impact of adding a visualization supplement to both (game and human-guided) methods on children's engagement was explored. Furthermore, a novel method was developed to estimate the children’s engagement stages based on their observed behaviors.
We recruited forty Mandarin-speaking children aged 4 to 7 for our study. These children were divided into two groups: Group 1 participated in a human-guided method without a visualization supplement and a gamified method without a visualization supplement. In contrast, Group 2 participated in a human-guided method with a visualization supplement and a gamified method with a visualization supplement. We compared the duration of time each child actively participated before stopping, the number of words they pronounced, and their behavior throughout the entire experiment to evaluate children’s engagement. Additionally, two experts analyzed the children’s behavior data, resulting in consistent findings regarding the stages of children’s engagement during the imitation pronunciation process.
experimental set up of four conditions:(1)Children participating in the experiment; (2)experimenter; (3) example speech; (4)imitation speech from children; (5)desk;(6)camera; (7)paper visualization supplement; (8) laptop; (9)keyboard for remotely control the game
The results indicate that the gamified method leads to longer participation time compared to the human-guided method. Additionally, the addition of visualization supplements enhances children’s engagement in both the human-involved and gamified methods. The observed behavior of the children also supports these conclusions. This study shows that a combination of gamification and visualization supplement enhance children’s engagement during imitation pronunciation exercises. We propose a four-stage framework based on children’s observed behavior to gain deeper understanding of children’s engagement during the task. Additionally, the study provides recommendations for designers, assessors, and therapists to optimize speech practice therapies.