Communication Aid System – Conclusions

Language is a complicated matter that is extremely important to our social life. Communicating with others is not just a question of transferring messages back and forth, but also about sharing experiences and entertaining each other. It is also a fact that a large portion of the population has temporary or permanent difficulties in understanding and using spoken or written language [1].

The language is our most important means of communication, to use and adequate language between two persons improves communication. The Communication Aid System proposed in this project is meant to serve as a series of tools that works together for helping the user to learn and communicate.

The importance of learning signs and imaging languages resides on the fact that for some CDP it’s the ONLY means of communication. If we care about this, we can say: improving use, learning and understanding of words, signs and imaging languages means also improving communication for children with special needs.

This System works in a different way for the Child, the Caregiver and Relatives, and into a Network.

[1] Buur, J. and Bødker, S. 2000. From usability lab to “design collaboratorium”: reframing usability practice. In Proceedings of the 3rd Conference on Designing interactive Systems: Processes, Practices, Methods, and Techniques (New York City, New York, United States, August 17 – 19, 2000). D. Boyarski and W. A. Kellogg, Eds. DIS ‘00. ACM, New York, NY, 297- 307.

Symbol Creator Mobile App

This video is a simulation of how the Mobile App would work while creating a new symbol/icon. After finishing doing a symbol/icon it would be possible to add it to the whole library of symbols as shown at the end of the video playback.

How to Create Personalized Symbols for the Child?

I proposed a “Symbol Creator” application for generating personalized symbols for every child with special needs. It’s meant for being used by the caregiver and relatives. It could be use together with a series of applications that would use those symbols as interface elements. This application is designed as part of a Communication Aid System.

This application by itself is incomplete for communication purposes because it doesn’t offer an Output for the created symbols, in other words, it is necessary to have another application that would make use of the created symbols. This “Symbol Creator” application could work as “Content Generator” where the content could be shown in many different ways by using other applications for Scheduling, Games, Messaging, etc. all useful applications for the child.

Image: The application’s diagram of use is shown.

What to do with the findings?

During the testing sessions I had a nice surprise when the child’s mother told me “It is the first time I see him (the child) trying to say some of the words and hand signals included in the mobile phone application”.

This meant the mobile phone icons and videos motivated him to copy and remember gestures and words. That was especially important because the more he practices to say words and signals the more he would be able to communicate.

Some of the successful characteristics of the application were to include videos with sound, also that it was easy to use for child and that it has clear sequence of steps to follow. The videos were taken from previous visits and always included known people for the child. In this way we assured the child could find familiar faces and locations which kept his attention and focus.

Having said that, a collection of personalized symbols it’s a need for the child. Creating a personalized database for the child could improve the learning process of the user. Adding video and sound could make more enjoyable the whole experience of using the application. When talking about children applications it is especially important to make them fun for them.

Image: 30 icons/buttons that the child can choose from in the mobile phone

Second User Testing – EyeTracker

Using an Eye Tracking System of Tobii Technologies model T-60 we tested the Symbols Application. At first, we were interested in knowing what part of the screen the child was looking at. We knew he liked to use the application and watch the videos, but we were not sure what was catching more his attention. Since the application has symbols with text, animations and videos, it was important for us to know what was working better for him.

The first thing to do for preparing the testing session was to adapt the mobile phone application to be a PC application in order to be able to use it with a laptop computer and using the mouse instead of the mobile phone touch screen.

The first big challenge was to carry out the calibration process where the child should watch 5 points on the screen without losing attention, but this step was very difficult since he lost his attention very easily and very often. Out of around 15 tries, we managed to calibrate the system a couple of times, enough to run a few tests.

At the end of the 30 minute session we watched the recorded videos and we got some interesting findings.

The most important observation was to realize when he looked at one of the symbols he was actually looking at the text area, contrary to what we had thought before that he was looking mostly at the images of the symbol. We could’t say that he was reading the text, but at least his attention at that moment was placed on the text in most of the symbols.

Second User Testing – Findings

The caregiver-teacher started to attach the symbols to the child’s body and leave them there during breakfast, the symbols she chose were, of course, the ones that contained images related with breakfast such as sandwich, butter, oats, etc.

The mobile phone application was tested also. The child quickly understood what to do and spent time playing with it in periods of about 30 minutes during that day.

Contrary to the mobile application that seemed to work very well that day, the wearable symbols where not as exciting for him, even though the teacher used it with him in the way they were meant to.

She used the mobile device the whole day at school and she also used the wearable symbols attached to the child’s t-shirt. During that day I wasn’t present at the school because I didn’t want to interfere with the normal activities of the child.

Video of the Test with Ludvig, first together with the teacher and at the end with the mother of him.

During the testing sessions I had a nice surprise when the child’s mother told me “It is the first time I see him (the child) trying to say some of the words and hand signals included in the mobile phone application”.

This meant the mobile phone icons and videos motivated him to copy and remember gestures and words. That was especially important because the more he practices to say words and signals the more he would be able to communicate.

Some of the successful characteristics of the application were to include videos with sound, also that it was easy to use for child and that it has clear sequence of steps to follow. The videos were taken from previous visits and always included known people for the child. In this way we assured the child could find familiar faces and locations which kept his attention and focus.

Second User Testing – Wearable Symbols & Mobile App

Due to the fact that I didn’t want to interfere in his normal behavior, the wearable system had to adapt to his normal activities.

After doing some testing on how to attach the symbols to his body it turned out that having a module that would be attached to most of the child’s t-shirts and sweaters would be the best option. For doing that I chose to have individual two-part objects containing the printed symbol; the base would be behind the t-shirt and the front part would be over the t-shirt holding each other while pressing the textile as the image # shows. In this way we can attach as many symbols to the t-shirt as we decide, even on the sleeves, for example. For the testing session I built 15 modules and printed 30 symbols that can be used together with the modules.

The mobile application and the wearable symbols were meant to work together as a tool for the kid to understand what the symbols are about and then use them for communicating. At that moment the Mobile App was still just a complementary tool for the wearable symbols that doesn´t need any special device to work, more “natural” tool it seemed to be for me.

Wearable symbols attached to Ludvig´s Sweater

Ludvig wasn´t very comfortable with the wearable symbols

Ludvig was practicing some hand signs after watching some videos in the Moible App

Common Icons – Mobile App

The right device for having the interaction, image and sound quality I was looking for, resulted in a touch-screen Nokia mobile phone model 5800 compatible with Flash Lite 3.0, with good video playback 640px X 360px and speakers.
The application was made using Adobe Flash Lite 3.0 and designed totally from scratch for this project.
The characteristics that we wanted for the application were easy to use, stable and intuitive.

“Mobil”Application Design

For the Third visit and Second testing of the Communication System with my user, I brought a mobile device with a Flash Lite Application in it. The aim of this application is to help the kid to practice images recognition and understand the links between image, text and meaning.

Here is a diagram showing the steps to follow for watching a videos or images.

User vocabulary – ordförråd

Before staring the design of the imaging system and screen based application, it was very important to select carefully the images and concepts we are going to use. For doing this, I asked the mother and the caregiver of my user the most common words/concepts (vocabulary) the kid has.

Since him (user) has a small vocabulary, I was able to create with images almost all his commonly used concepts.

For this test I decided to stick to the original symbols he use normally and just adjust the frame, shape and color; as well as the font, but I haven’t redesign the symbols yet. Here are some of the symbols I used for preparing the next testing session.

New Symbols – Text, Shape and Color

Our user normally uses images for communicate.

For this project it would be necessary to have to versions of the same symbol; one printable and one clickable for screen based applications.

For creating a screen based application is important to have consistent symbols that would work also as clickable buttons. Whether for printed symbols is important to make more emphasis in, for example, the color of the circular frame and less visual distractions.

Here, I’m showing the process for creating a picture based symbol.

Textures for the Images

In order to bring Interactivity to the system, I was thinking in how is going work the interaction. When talking about communication is very important to make sure that our message to the other person is being received, in other words, if I talk the other person must listen and understand what I’m saying. Because of this, when a kid with special needs wants to communicate through images, is important that the receiver look at the images. One way to achieve this is having a glowing image when the kid select it. Here are some alternatives on how the texture would look like in order to be “touchable”.

Wearable Images – Main Alternatives

Aiming to create a portable system of images for communicate with. These are some of the main ideas of how is going to work. All of them have basic principles of portability and adaptability. Another aspect to take in count is the durability.

Imaging Languages – Example of a Sofa’s Image

Since the Communication Aid I’m working in needs to be used and observed by other people than the child, is important to make some changes to the images in order to make them more understandable for observers. Here are some alternatives of what would look a modified image of a sofa.

Findings from 2nd Visit – Gestures for Communication

Gestures, mumbles, and physical activities. All three of these, are ways for Ludvig to communicate, for this second visit to Stockholm I focused totally in trying to identify the ways he express him self. My aim is to use this “gestures”  for designing something that would feet his actual needs of communication.

One of the most common gestures is the eye-contact, meaning to pay attention or to ask for attention.

Ludvig’s grabbed his sweater repetitively during the visit, I think is a way of communication.

When he really wants to show or do something together with someone else, he pull the hand of them.

Part of his normal activities involves physical games, where he also communicates with the other kids.

Another particular gesture of Ludvig is to rise the hand (mostly left hand), to ask for attention from the teacher or another adult.

All photographs by Jose Ledon