Serious Gaming for Children with Autistic Spectrum Disorder
Project details
Client: ShoSho, Kentalis
My contribution: Interviews, research analysis, brainstorming, concept generation, physical modeling, hardware prototyping, user testing.
Team members: Lennart Overkamp, Marta Kaczmarczyk and Bart Geelen.
Challenge
Children with the Autistic Spectrum Disorder (ASD) have difficulties with recognizing emotions, understanding the meaning of emotions in a social context and with responding appropriately to emotions of others.
At Kentalis school, different methods are used to teach social interaction and communication skills to these children. However, teaching autistic children to overcome their difficulties is challenging, and places a large burden on teachers.
Shosho, an Amsterdam based digital studio, had been working with Kentalis on digital game-like solutions to try to improve the children's emotional development.
However, after interacting with the games for some time, children would start memorizing the steps of the game. As such, the question on whether they were learning something, was raised.
The quest was set: how to design an interaction that would make the game experience more immersive?
Process
To start, we held a workshop with the different stakeholders to understand the Who, What, Where and Why of the design challenge. This helped us refine the scope of the project.
- Who: children with ASD, their teachers and their parents
- What: teach children about emotions, to be able to recognise emotions from facial expressions in a social context
- Where: school and outdoors
- Why: learning the meaning, expression, recognition and effect of emotions in other people might help the children with ASD communicate better with other people
With a clearer goal in mind, we set out to interview the teachers of the school, those who dealt daily with the children daily, to better understand about the teaching process and challenges on the classroom and playground. During the interview we would took notes and notes and afterwards, we used Mind Mapping to cluster in and out the important findings.
In parallel, we researched and benchmarked solutions for similar challenges. Those which focused on facial expressed were mostly digital and approached learning on a sequential manner, by combining sentences and pictures with questions.
With many solutions being digital and having the design challenge in mind, we felt we could to try approach it from a physical-digital point of view.
At the same time, ShoSho, with full experience on digital modelling and animation, was trying create a simplified model of expression, to turn animation more efficient and possible to automate.
An hypothesis, based on scientific research findings relating emotions and hormones, was that emotions were directly related to 3 main hormones: Serotonin, Dopamine and Noradrenaline.
If emotions were also related to facial expressions, could we (in)directly link the 3 hormone input with facial expressions, simplifying the process of expressing emotions with only 3 inputs?
More, could this be a model that children could naturally use and absorb to communicate?
With this idea in mind, we prototyped and tested different physical approaches with several children. From sliders, to joysticks, or even distance sensors.
During testing, we explored how children would use these physical interactions to create different expressions.
However, it was unclear if the 3 hormone approach was understood by the children.
Therefore, we felt we needed to take a step back and simplify both the interaction and the expression communication.
Perhaps we could use directly emotions, instead of indirectly with hormones.
Inspired by a cube-like interaction design used by one of our colleagues, Katy Downey, and also by the approached used with emotions in the film "Inside-Out", we tried out a cube with colours and emotion labels as an input to facial expressions.
During sessions with children, this approach seemed to work better. Children would respond easier and faster.
However, with this approach, children were limited to a 6 way communication.
How could we give them a broader tool to communicate?
We decided to turn-twist the input:
What if, instead of emotion-labels in a cube, children could directly choose facial expressions?
With this in mind, we had the idea of using 2 cubes, one for the upper facial expression and the other for the bottom facial expression.
With this approach, we could communicate 6x6, i.e. 36 different expressions.
With arduino and micro-controllers, we turned the expression cubes to live by connecting them to a game-like software, that would change the expression of a character based on the cube sides facing up.
Having this "phygital" prototyping working, we showed it to the teachers, who were eager to try it out in the classroom.
The sessions in the classroom were well received by both the teachers and the students.
Serious Gaming for Children with Autistic Spectrum Disorder
Categorias
