Subscribe Subscribe to Atlas

A digital storytelling app for the deaf

Atlas-Winning Team is Prototyping Apps to Promote Literacy

Photo from Leandro Flórez-Aristizábal

Each month the Elsevier Atlas Award recognizes research that could significantly impact people's lives around the world.

Digital transformation to support literacy teaching to deaf Children: From storytelling to digital interactive storytelling

Leandro Flórez-Aristizábal, Sandra Cano, César A. Collazos, Fernando Benavides, Fernando Moreira, Habib M. Fardoun
Telematics and Informatics
, Volume 38, May 2019, Pages 87-99

Advances in digital technology including tablets, smartphones, and mobile apps have revolutionized the way people interact, communicate, and learn at home and in classrooms around the world. Now, an Atlas award-winning team is working to ensure that this shifting educational landscape will help to support literacy in deaf children.

“We’re proposing a framework that uses storytelling [and digital technology] as a way to motivate children to learn,” said Leandro Flórez-Aristizábal of the Institución Universitaria Antonio José Camacho and University of Cauca.

As reported in Telematics and Informatics, Flórez-Aristizábal and colleagues are working to design and test an interactive storytelling app for sign language-speaking deaf children. To help in this effort, they’ve enlisted the experts: deaf children themselves.

Photo from Leandro Flórez-Aristizábal

Flórez-Aristizábal and his colleagues began by designing a series of black and white cards depicting images. The images followed a story about a kid and a strange creature on Halloween. But the sequence of events in the story was up to the kids to decide.

To test out this interactive storytelling approach, the researchers conducted a case study with 5th graders at the Institute of Special Therapy of Senses in Cali, Colombia. The task was for a single student to arrange the cards in any order she chose. Then, she was to tell the story to the rest of the class in sign language.

One thing that struck the researchers as they watched the interaction was the response of other students in the classroom. As the student selected to arrange the cards worked, other students got up and began offering suggestions. They were turning the storytelling experience into one of interactive collaboration.

In a second phase, the researchers added additional details and colors to the cards and tested those out with kids. Finally, they converted the colorful cards to a digital format. And, the kids appeared to love it. The researchers report that children’s motivation increased when technology was part of the activity. The technology allowed them to incorporate extra bells and whistles, such as the time children took to arrange the cards. They also offered the kids rewards in the form of virtual trophies.

At each step along the way, the researchers took the responses and suggestions of the children into account in making the next version. Flórez-Aristizábal reports that they recently conducted a 5th case study with deaf children in Scotland on an app that incorporated storytelling and vocabulary.

The plan, he says, is to continue working and ultimately to develop an app with multiple levels and four languages: English, Spanish, British sign language and Colombian sign language. While their primary target audience at the moment is deaf children, he says the app might also find use as a means for hearing kids to learn sign language. Ultimately, they hope to expand even farther.

“We started with deafness, but we don’t want to stop there,” Flórez-Aristizábal said. “We want to invite other researchers to join us in this journey and to help us develop more prototypes and adaptations for other disabilities and other educational goals. In this way we’re trying to fill the gap. Ultimately, we want to forget about disabilities and think about different abilities.”

Atlas-Winning Team is Prototyping Apps to Promote Literacy
Alfredo Rodrigues Marques, rector Universidad Portucalense, Claudio Colaiacomo, Vice President, Academic Relations, Elsevier, Leandro Florez-Aristizábal, Fernando Moreira, authors of the awarded article, Hugo González, Rector Institucion Universitaria Antonio José Camacho (left to right)

A Conversation with Leandro Flórez-Aristizábal

podcast Listen now

Atlas: How did you begin thinking about digital transformation and what it means for the deaf community?

Flórez-Aristizábal: This is part of my PhD thesis. When I started, I was continuing work of my supervisor with deaf children who used cochlear implant. I wanted to continue this work by including children who use sign language. In this paper, we have done some case studies with deaf children—trying to understand how they learn, what they like, the stories they enjoy. We came up with the idea of involving children in the design of a prototype system but also in designing the story behind the idea. We want to support teachers in doing their job, but also to provide children with something they can enjoy to promote literacy.

Atlas: What did you learn from the kids?

Flórez-Aristizábal: We wanted to know what they thought about the story and the drawings, and it was interesting. We as adults didn’t care much about some details that they paid attention to. For instance, one of the characters was supposed to be surprised, but the kids thought he was scared because of the way the eyes were drawn. It’s something we knew we had to take into account and that’s why it’s important to have a paper prototype first. In the next case study, we already implemented the kids’ suggestions about the drawings, the facial expressions, and so on.

Atlas: How did things change when you moved to the digital version?

Flórez-Aristizábal: For the 3rd case study, we had a digital version of the story and this is something they actually enjoy much more. They could arrange the cards [on the screen] how they wanted and they didn’t want to stop. They wanted to continue working with the computer. We added something we didn’t have in the previous case studies including some gamification aspects. After they finished arranging, we showed them their time and a trophy like a reward. They immediately understood and tried to arrange cards over and over. They wanted to beat their previous times. We knew it was a way to engage children through rewards and gamification. But it’s something you should do carefully to ensure children don’t lose focus on the purpose of the activity.

Atlas: This looks promising, so what are you doing now?

Flórez-Aristizábal:  We recently did our 5th case study with Scottish deaf children. We tested a prototype mobile app with storytelling and a vocabulary stage. In the end, they had to play a game about matching words with signs and images and they really enjoyed it. We’re now ready to have the framework evaluated by some experts. The pedagogical aspect has to be evaluated by teachers for it’s potential for not just storytelling but also to promote collaborative learning among deaf children.

Atlas: Was there anything that particularly struck you in watching these kids interact?

Flórez-Aristizábal: In this last case study we did with deaf children in Scotland, I was surprised how well they interact with mobile devices. It’s natural for them. They immediately knew how to interact with the app. I thought they’d need an introduction or help, but no, they just did it from the beginning to the end of the activity. They didn’t need any help and they really enjoyed it. We are going to test the same prototypes in Colombia and see how differences in digital literacy also may influence this. I think it could give us different insights about how children use technology in different countries. Many deaf children in Colombia are from low income families and don’t have access to this technology at home or in school. So, we’ll see how they will interact with the app.

Atlas: Are there plans to make such an app available?

Flórez-Aristizábal: Well, yes, so far we’ve done prototypes. This one we just tested is supposed to have nine levels, but so far we have one. All we wanted to know is how children interact with it. The idea is to have a full version of the app with different levels and languages by next year and to make it available in the app store. But we don’t want to stop there. We want to focus on other skills like math or perhaps children with other disabilities like blindness or cognitive disabilities.

We’ve been working on this for the last four years with researchers from Saudi Arabia, Portugal, Columbia, and Spain. It’s very rewarding to do this kind of research. It’s not something that you just type or write in a paper and it will be there in a database nobody uses. We want to do something that can impact on people’s life and this is why we are going to focus our research on disabilities.

Related Resources

About Telematics and Informatics

Telematics and Informatics is an interdisciplinary journal publishing innovative theoretical and methodological research on the social, economic, geographic, political, and cultural impacts of digital technologies. Application areas include smart cities, sensors and information fusion, the digital society and digital platforms, internet of things (IoT), cyber-physical technologies, privacy, knowledge management, distributed work, emergency response and hazards, mobile and wireless communications, health informatics, psychosocial effects of social media, ICT for sustainable development, blockchain, e-commerce, and e-government.


Written by

Kendall Morgan, PhD

Written by

Kendall Morgan, PhD

Kendall Morgan, PhD, is a scientist turned science writer via the Science Communication Program at the University of California, Santa Cruz. Her work has been featured in publications including Big Science Media's Genome, Cancer Today, The Scientist, Science News, Science Now, and by organizations including Addgene and the Life Sciences Foundation. She lives in Durham, NC.


comments powered by Disqus