Design and evaluation of a tactile memory game for visually impaired children

Citation preview

Interacting with Computers 19 (2007) 196–205 www.elsevier.com/locate/intcom

Design and evaluation of a tactile memory game for visually impaired children Roope Raisamo *, Saija Patoma¨ki, Matias Hasu, Virpi Pasto Tampere Unit for Computer-Human Interaction (TAUCHI), Department of Computer Sciences, FIN-33014 University of Tampere, Finland Available online 27 September 2006

Abstract Visually impaired people have a lack of proper user interfaces to allow them to easily make use of modern technology. This problem may be solved with multimodal user interfaces that should be designed taking into account the type and degree of disability. The purpose of the study presented in this article was to create usable games for visually impaired children making use of low-cost vibro-tactile devices in multimodal applications. A tactile memory game using multimodal navigation support with high-contrast visual feedback and audio cues was implemented. The game was designed to be played with a tactile gamepad. Different vibrations were to be remembered instead of sounds or embossed pictures that are common in memory games for blind children. The usability and playability of the game was tested with a group of seven 12–13-year-old visually impaired children. The results showed that the game design was successful and a tactile gamepad was usable. The game got a positive response from the focus group.  2006 Elsevier B.V. All rights reserved. Keywords: Visually impaired children; Multimodal user interfaces; Low-cost haptic devices; Tactile games; Tactile feedback; Usability testing

1. Introduction When making information technology accessible for disabled people an answer is to use customized devices and design the interfaces according to established guidelines that are maintained by the organizations representing disabled people (Tiresias.org, 2006). Another solution is to implement multimodal applications making use of the latest interface technology. When lacking of a certain ability the rest of the abilities are used to substitute the impairment in the best possible ways (Goldstein, 1999). In multimodal user interfaces the loss of sight, hearing or mobility can be replaced with other modalities. Well designed multimodal user interfaces can be invaluable aids for people with disabilities in many different areas, such as education, training, rehabilitation, and communication. They can offer access to information, entertainment and games. *

Corresponding author. Tel.: +358 3 3551 7056; fax: +358 3 3551 6070. E-mail addresses: [email protected].fi (R. Raisamo), [email protected].fi (S. Patoma¨ki), [email protected].fi (M. Hasu), [email protected].fi (V. Pasto). 0953-5438/$ - see front matter  2006 Elsevier B.V. All rights reserved. doi:10.1016/j.intcom.2006.08.011

The International Game Developers Association has addressed the question of game accessibility and has formed a Game Accessibility Special Interest Group (GA-SIG). The ambitious goal of the GA-SIG is to make all game genres equally accessible despite the disability of the user (IGDA Accessibility Group, 2004). They state that the game industry should be pushed to design games that take into account persons with various disabilities. There is special assistive technology (VisionConnection, 2006) designed and developed particularly for the needs of blind computer users. For example, with the aid of a screen reader it is possible for a blind person to discover the textual contents of a program. Information on the screen is read aloud through speech synthesizer. Screen magnifiers are handy when a partially sighted user has to enlarge a portion of the display. A Braille display passes information through the user’s fingertips as its mechanical parts physically change position and form Braille sentences. However, these aids are expensive, and mainly designed for blind adult users. As such, they are not enough to meet the needs of blind children.

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

We chose visually impaired children as the main focus group. By visually impaired we are referring to both blind and partially sighted children. They are a very challenging group of users. These children are even more different from each other than average children, since there are so many different kinds of visual impairment. It is also hard to diagnose the impairment of children in a young age, as they cannot explain accurately what they see, especially if they have always seen that way. Visually impaired children start to learn Braille typically in the elementary school. Before that they have to master some fine-motor, directional and tactile skills that are taught at a pre-school age of five to six. We chose to create a tactile memory game, because it would teach these vital skills to the children in a simple way and in the form of a possibly already familiar classical memory game. In this paper, we present a case study of designing and evaluating the tactile memory game to be played with low-cost tactile feedback devices and multimodal navigation support. There is a wide selection of haptic feedback devices on the market. There is an earlier experiment (Raisamo et al., 2005) investigating the limits of low-cost haptic feedback devices, such as force feedback steering wheels, joysticks, and gamepads. In the present study these earlier results have been used to design the tactile effects. When designing game interfaces for blind children the essential sensory channels should be hearing and touch. Furthermore, based on earlier research partially sighted children need to have clear, high-contrast visual stimuli (Patoma¨ki et al., 2004). Benefits of multimodal games for blind children are evident. Game play may motivate to use computers also for other purposes and thus supports the development of computing skills. Therefore, the special interfaces presented in this paper help visually impaired children to gain entrance in the modern information society dominated by the sighted. 2. Previous work In this section, we give an overview of previous work in interface technology and applications used by blind computer users. The emphasis is on interfaces especially designed for blind children. Applications for blind adult users are also presented when applicable. We have divided the previous work in three categories: auditory interfaces, tangible tablets and keyboards, and dynamic haptic interfaces. 2.1. Auditory interfaces In auditory interfaces the audio can be the main or the only feedback channel and it can be used in various ways. There may or may not be visual feedback that can be designed in two different ways: it can just be present without giving any benefit in gaming that is based on audio hints only, or visual feedback significantly eases up game

197

play for partially sighted and sighted users. Even in games for blind children use of visual feedback can be justified because it also benefits blind children who can share the excitement of playing with their sighted friends. Sonification, visualization by sound, has been studied widely for decades and it has also been applied in software for blind people. The aim is to display, for example, graphics, line graphs or even pictures using non-speech sound (Meijer, 2006). By altering the various attributes of sound, for example pitch, volume and waveform, the sound is changed according to its visual counterpart. In the SoundView application (van den Doel, 2003) a colored surface could be explored with a pointing device. The idea was that the characteristics of colors, hue, saturation and brightness were mapped into sounds. Roth et al. (2002) used audio and kinesthetic rendering jointly and separately. The audio-kinesthetic encoding of the graphs was found to be the most usable approach in visualizing shapes to blind people. The Swedish Library of Talking Books and Braille (Barnens, 2006) has audio games suitable for young blind children. The games are played with an ordinary keyboard and they are based on sound feedback, but visual feedback is also present. A memory game is played by recalling animal sounds. Based on the tests (Eriksson and Ga¨rdenfors, 2006) that four visually impaired children participated in, the auditory memory game was difficult for them. Non-visual computer games can not provide such a good overview of the play area as graspable cards can. Their advice was to provide proper instructions that help a blind child to understand the mental model of the game. AudioBattleship (Sa´nchez et al., 2003) was based on the classical battleship game using only audio cues. User tests with four visually impaired children showed that collaboration between blind children was enhanced in the game. AudioDoom (Lumbreras and Sa´nchez, 1999) is an interactive game where spatial sound is used to present the environment that the player experiences from a first-person perspective. It was found that blind children preferred playing the game with a joystick over a keyboard; obviously the clear directions of the joystick gave additional spatial information. As the game was played for several times the children could construct the environment accurately with LEGO bricks. All of these earlier results helped us to design the use of audio in our game. 2.2. Tangible tablets and keyboards One way of creating tangible human-computer interfaces for visually impaired young children is to use alternative keyboards or tablets. The goal is to combine the use of real tangible materials with advantages of information technology. The benefit these devices have compared to most haptic devices is that they are used in a simple and natural manner. The application can be manipulated directly with fingers by touching and pressing the surface of the device. Hand-made or industrially manufactured overlays can be

198

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

set on top of these devices, such as Flexiboard (Flexi Forum, 2006) and IntelliKeys (Inclusive Technology, 2006). Overlays are customizable and can be constructed individually for a particular child. For example, fabrics, pearls and waxed strings can be glued on overlays, or printed embossed graphics and figures can be used. Computer games originally intended for sighted children can be played with minor adjustments with the aid of tangible tablets. In the TIM project (TIM, 2001) several games (X-tune, Reader Rabbit, Can You Hear It?) were made usable for blind children as appropriate overlays were prepared. Hammarlund (1999) suggests that special games for blind children should be designed from the scratch to make full use of the combination of real-world materials, sound illustrations and recorded speech. Geometric shapes and maps can be shown to blind people using tablets. In a study by Roth et al. (2000), geometric shapes were taught to visually impaired pupils as raised figures using a printed swell paper overlay on a tablet. When shapes were explored audio feedback was simultaneously provided. A study by Landau et al. (2003) showed that with the Talking Tactile Tablet (Touch Graphics, 2006) their students learned geometrical shapes better than when using common mathematics materials intended for blind pupils. Holmes and Jansson (1997) presented a street network in an audio-enhanced virtual map. It was found that a tactile map overlay needed less exploration time than a matrix overlay. Based on these previous results we decided to include a tangible model in the testing procedure presented in this article. 2.3. Dynamic haptic interfaces The benefit of creating dynamic haptic applications compared to using of physical materials is that the content can be designed freely and changed dynamically making it more interactive. Nemec et al. (2004) used a force feedback joystick and a haptic mouse together with spatial sound that was found to be applicable when creating a mental model of indoor environments. In their study a haptic mouse was proven to be usable in haptic sensing of the virtual model, but a force feedback joystick was confusing for all of the users. In a pilot study by Feintuch et al. (2004) a blind girl, after training with navigation using a force feedback joystick, could walk quickly and confidently in real surroundings, not known to her before. These examples emphasize a good design of the navigation methods. Baptiste-Jessel et al. (2004) used a Logitech Wingman force feedback mouse to present graphical data to blind users in web-based documents. For example, a geographical map was possible to be explored with the mouse by giving a magnetized effect, when the center of the region was approached, and a textured effect, when the mouse was in the periphery of the area. The test subjects told them that haptic feedback helped to get a mental model of the geographical area.

SensAble PHANTOM devices (SensAble, 2006) have been used in applications for blind computer users. Sjo¨stro¨m (2002) implemented several games for blind children. For example, a haptic battleship game, a haptic mathematics program, and a painting application in which textures correspond to different colors have been developed. Patoma¨ki et al. (2004) constructed several simple games for visually impaired 4–6-year-old children making use of dynamic haptic effects with the PHANTOM device. A graph visualization tool for blind students using a PHANTOM device has been studied by Ramloll et al. (2000). They found that haptic graphs should be visualized as groove lines rather than embossed lines. The previous research presented in this section gave us insight and some empirical results to design the auditory and haptic feedback methods for the tactile memory game. However, many of the design decisions needed to be made during the iterative development based on our own pilot tests and preliminary investigations, as directly applicable results did not exist. Next, we describe this process. 3. Design of a tactile memory game Visually impaired children are very dependent on their parents and other people who help them in their daily lives. Blind children may therefore have problems with getting in contact with other children. Our goal was to create a game that would be played by both sighted and visually impaired children, and would encourage visually impaired children to use computers independently. Our primary focus group is visually impaired young children who do not have other disabilities. We also wanted to know how low-cost tactile devices such as a gamepad, a joystick or a wheel can support both navigation and game experience for visually impaired child-users. Existing memory games for visually impaired children are either tangible games, made of touchable or sound-making materials, or auditory computer applications. Instead of graspable objects or sounds, we used tactile effects produced by a tactile gamepad as the items to be memorized: images were replaced by tactile effects. The Swedish Library of Talking Books and Braille’s Djur Memory sound memory game (Barnens, 2006) was an inspiration for this design decision, as they used audio in a similar way. To our knowledge a tactile device has not been used this way before. Compared to tangible games computer-based haptic interfaces give more freedom to easily adjust haptic and sound feedback adapting to each user. Furthermore, the content of the virtual game can be richer as it is not limited to real artifacts. Dynamically changing contents and advancing in the game makes the game more expressive and interactive. The aim of the classical memory game is to find image pairs from cards that are laid out on a plane. At the beginning all the cards are faced down. The user starts the game by choosing one card and turning it over. Then the user

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

199

tries to find a pair for it by turning another card over. If the image of the second card does not match with the first one, both cards are returned to the face down position. If they do match, the cards are removed. The game ends when all the pairs have been found. The less there are turns of pairs the better the score is. In a previous study (Raisamo et al., 2005) 18 adult users took part in a series of tests aimed to find out distinguishing thresholds for tactile effects with a gamepad, a joystick and a wheel. The study was aimed to roughly estimate the tactile parameters that would be usable for visually impaired children. The results supporting the selection of tactile gamepad were the following: if the difference in the balance of the motors was high enough (more than 50%), all the users could differentiate the effects without errors, which was clearly not the case in other conditions. In addition, the type of motor used had a clear effect on the recognition of force magnitude. With the small motor the small magnitudes were the easiest, while with the large motor the large magnitudes were the easiest. This is due to their mechanical construction. The gamepad also clearly outperformed the force feedback devices in differentiating frequency of the effects. Considering also the game type, easy availability, and inexpensive price, we chose to use a Logitech WingMan Rumblepad (Fig. 1) as the tactile controller for the new game. Its small size and two-handed (robust) use were also important criteria in selecting the device, as the game was designed for visually impaired small children. The gamepad was connected to the computer using a standard USB connector and it was interfaced in the software using Microsoft DirectX 9.0. The game cursor was controlled with the four-way digital ‘‘D’’ pad (Fig. 1) as it clearly represents physical directions. The cards were turned with the ‘‘C’’ button (Fig. 1) which is easily accessible with the player’s right thumb. The gamepad has two vibrating motors to produce tactile

effects. The larger motor is located within the left handle of the pad, and the smaller one within the right handle. Although the game was designed for blind users, we offered simple, high contrast visual feedback as well. Each card was represented with uniform visual feedback (a filled box). The game interface consisted of 12 dark blue cards in a 3 · 4 grid on a white background, as shown in Fig. 2. The width of a card was 270 pixels and the height was 150 pixels. The cursor was presented as a 385 · 215-pixel bounding black box around the card. The game area was 1280 · 925 pixels on a 1700 LCD monitor. A turned card was marked with the cyan color. As the pairs were found their visual and haptic presentations were removed from the game. Navigation information was presented with a digitized xylophone sound that was considered pleasant, but neutral. The column of the cursor was indicated by panning the sound from the left to the right audio channel based on the horizontal cursor position. The middle column divided the sound equally in both channels. The pitch of the xylophone indicated the row; the lowest row produced the lowest sound and the highest row produced the highest sound. A negative-sounding hoot sound indicated a wrong pair and a melodious chimes sound was played when the user found a right pair. The hoot sound was followed by a sound imitating a card turn to indicate that the cards were returned to the face-down position. As the cards disappeared when the pairs were found, an empty card slot was indicated by a sound with the corresponding pan and pitch, but with 50% of the volume of a full card slot sound. The edges of the game grid were indicated by a louder knocking sound. The tactile effects to be memorized were designed with Immersion Studio 4.1 (Immersion, 2006). Initially we assumed that plain tactile effects without sounds would be too hard for the children to recognize. So in the pilot tests we used effects imitating the feel of animals with sounds associated to them. These effects consisted of, e.g., dog barking and chicken cackling. In the pilot tests the animal sounds turned out to be too dominant, so we

Fig. 1. Logitech WingMan Rumblepad.

Fig. 2. The tactile memory game interface.

200

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

left them out from the actual tests to focus on tactile feedback. Yet audio cues for navigation were found to be working. Therefore, they were kept unchanged in the game. For the actual test we designed a new tactile effect set without sounds, making use of earlier empirical research on similar devices (Raisamo et al., 2005), as the animal effects worked well with the corresponding sounds, but were not the most easily recognizable effects if only tactile feedback were used. The exact parameter values that are reported in earlier studies of tacton research (e.g., Chan et al., 2005; van Erp and Spape´, 2003; Brown, 2005) were not usable for us, since the results are specific to the devices reported in the papers. We tried to use the results based on high-quality SensAble PHANTOM devices (SensAble, 2006), but the cheap device was not able to produce the frequencies determined to work best in these studies without causing disturbing resonance or other technical problems. So the effects needed to be specifically designed for the device used. Nevertheless, these studies gave us useful background information and basic rules when designing the tactons used. The criteria for selecting the effects in the game was that they had to be very short and at the same time clearly distinguishable from each other. In addition, the effects had to be designed so that they could have clear names that ease up their memorability in the game, as Brown et al. (2005) suggested in their work. They also suggested that instead of vibro-tactile parameters (frequency, magnitude, waveform or duration) more complex parameters should be used. They found roughness and rhythm to be usable as such parameters, level of roughness being produced with magnitude-modulated signals and rhythms with pulses that have different durations. Accordingly, four of our effects were based on altering magnitude and two were based on rhythmic pulses. To make the effects clearly distinctive and therefore easy to label, we designed three effect pairs and named them as follows: decreasing and increasing, two-rhythm and four-rhythm, and gentle and intense. Characteristics of the effects are listed in Table 1. All the effects used were based on square waveform. When the game ended, a score window appeared on the screen. The number of pairs turned was written on the screen and for every pair turn a red circle appeared sequentially to the screen with a ping sound so that the blind child could understand the score. As there were six pairs, the best

possible score would have been six, if only it would have required a lot of luck. Scores of 12 pairs or less were considered as very good. 4. Usability and playability evaluation Although our final users for the memory game were visually impaired young children, in this study the target group consisted of visually impaired teenagers, since they were able to tell us more accurately what they feel with the device to help us in iterative development of the tactile game, and we did not want to overload our final test users with testing of versions of the application that might not have been optimal. Testing of the final prototype with the young children is left for future work. A special usability testing procedure developed in an earlier project (Patoma¨ki et al., 2004) was refined for these studies. It follows the principles of usability studies (Nielsen, 1993), but includes specific support for visually impaired young children, as they often cannot manage alone. It is highly important not to give too much support during the test, but to enable the child to use the computer system. 4.1. Testing environment In Finland, as in several other countries, the aim is that visually impaired children without additional handicaps go to a normal school with the sighted children. Twice in a year these blind children belonging to the same age group are invited to take part in a special teaching period lasting for a week in Jyva¨skyla¨ School for the Visually Impaired (Jyva¨skyla¨, 2006). These courses are also convenient for testing purposes, since an entire target user group is gathered at the same time in the same place from all over the country. The testing was carried out in an empty classroom, where the test setup was built (see Fig. 3). The equipment consisted of a computer, a flat display, a pair of loudspeakers, a vibro-tactile gamepad and the equipment for recording and videotaping the test situation. 4.2. Test users Before testing the game with the target users it was pilot tested several times with sighted adults and children with and without the aid of graphical feedback. The two sighted

Table 1 The effects used in the tactile memory game Effect name

Description

Magnitude

Motor

Duration (ms)

Gentle Intense Increasing Decreasing 2- Rhythm 4- Rhythm

Gentle constant vibration Strong constant vibration Increasing vibration Decreasing vibration Two 250 ms impulses separated with a 250 ms break Four 250 ms impulses in a row

2000 10000 From 2000 to 10000 From 10000 to 2000 10000 10000

Small Large Large Large Large Large

2000 2000 2000 2000 750 1000

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

201

reader and screen magnifier. Every child had tried out the PHANTOM haptic feedback device (SensAble, 2006) in the tests carried out earlier in the same school (Patoma¨ki et al., 2004; Saarinen et al., 2006). Some of the children were also familiar with haptic gaming devices such as a force feedback wheel, a force feedback joystick, or a tactile gamepad. 4.3. Testing procedure

Fig. 3. A blind girl is playing the tactile memory game with the vibrotactile gamepad in her hands.

pilot child-users were at ages of 9 and 12 years. A blind 9year-old girl was also invited to pilot test the game. A group of seven 12–13-year-old visually impaired children participated in the tests: we had the whole age class of Finnish visually impaired children without any other major disabilities. All the children were visually impaired but they had different levels of impairment. Three of the children were totally blind and four were partially sighted. We asked the parents to write down a description of the level of remaining eyesight for those who were partially sighted. Two of the partially sighted children saw to a degree with one eye while their other eye was blind. The other two partially sighted children had some accurate sight ability when they looked at the target at a close distance. There were three boys and four girls (see Table 2). Four of the children were congenitally blind or partially sighted. Three of the children had gone blind or partially sighted later in their life. It is common for a blind child to have other disabilities as well. As the school has specialized in education of blind pupils, exclusively focusing attention to children not being multiply disabled, there was only one child in the group who had other disabilities. The child in question suffered from problems in concentration and in spatial conceptualization. All the children had frequently used ordinary keyboards and assistive technology, such as Braille keyboard, screen Table 2 Basic information on users Age

Gender

Visual disability

Inception age of blindness

Other handicaps

13 12 13 12 12 12 13

Boy Girl Boy Girl Girl Boy Girl

Blind Partially Partially Partially Blind Partially Blind

Congenital Congenital Congenital 2 11 Congenital 9

Yes No No No No No No

sighted sighted sighted sighted

The testing was carried out in Jyva¨skyla¨ School for the Visually Impaired (Jyva¨skyla¨, 2006). Demographic information on the children was gathered with a questionnaire that their parents filled in before the tests. In the beginning of each session the test assistant chatted casually with the child to suppress the tension that situations like this often cause. Our test assistant was a nursery school teacher whose task was to motivate and interact with the children based on the needs of the situation. An informal interview consisting of questions about the child’s background followed next based on the information acquired with the questionnaire. It was extremely beneficial to go through the questions with the child considering his or her skills and experiences on applications and haptic devices, since clearly the parents had not been able to describe them accurately. Based on earlier experiences in studies with visually impaired children (Patoma¨ki et al., 2004) we provided the children with visual stimuli. Before testing, the seeing ability of partially sighted children was evaluated by observing their use of graphical feedback. The best distance and position of the display was determined together by the child and the test assistant. The length of one test session was normally less than 45 min. If needed, a session could be extended up to 60 min. First, the child got used to playing the memory game with a tangible model. Following the advice by Eriksson and Ga¨rdenfors (2006) we organized training to ensure that the children had a correct mental model of the game. We also used elements of tangible interfaces. The purpose and rules of the memory game were rehearsed with a tangible model having a similar form as we had implemented as a virtual multi-modal application. The virtual cards in the tactile memory game were presented in the exactly same layout as in the tangible model. A tangible memory game model was constructed of small boxes with covers that were glued on the cardboard plane (see Fig. 4). Fabrics with various textiles were attached inside the covers of the boxes. In the tangible memory game an overturn of a cover and active exploration of the fabric texture in it corresponds to a card turn and seeing of a picture in the memory game of the sighted. Next, each child was introduced to the vibro-tactile gamepad (Fig. 1). The proper operating position and the use of push buttons in the gamepad were shown to the child. In the training mode of the game the cards were pre-arranged. In this way, the test assistant could

202

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

Fig. 4. Mental model of the game field was understood with the aid of a tangible model.

explain and show all the various action sounds that occur during the game play. The idea and rules of the game were gone through once more in this virtual gaming context. After the rehearsal of playing the virtual memory game the child continued with the actual testing. The goal was that the child would play the game independently, without any help. The child was expected to complete the game with as few card turns as possible. After completing the testing phase each child was asked a set of questions, but the interview that was carried out as a normal conversation was not limited to them. In short, the issues discussed included the tactile device, the auditory and tactile feedback of the game, and other game ideas that arose in the child’s mind. 5. Results and discussion In this section, we present the results of the usability study in the following main categories: use of visual and auditory feedback, localizing in the game, success in the game play, navigation strategies, and memorizing vibrations. The results are based on video logs, observations of the children, event logs, and the children’s answers in questionnaires. 5.1. Visual and auditory feedback In the tests it was noticed that all four partially sighted children could make use of the high-contrast graphical user interface of the game. The flat display proved to be practical for them, since it can be placed on the very edge of the desk. It is important that the display is easily adjustable for every child to have an optimal distance to the source of visual feedback. Two of the partially sighted children were able to use visual feedback in a normal sitting position head turned directly towards the display. The two children who could see partially only with one eye leaned forward

and turned their head to turn their better eye towards the display. After the test the children were asked about the various sounds the game contained. The children recalled the audio cues fairly accurately and could explain the meaning of sounds in the game to the test assistant. The game contained sound illustrations and audio cues for navigation. The knocking sound was remembered by all seven children. Location sounds of the cards were remembered by four of them. Five of the children remembered the melodious chimes sound. Four of the children remembered the hoot sound. The card turning sound was remembered by two of the children. The empty card slot sound was remembered by three. Two of the children mentioned the ending sound of the game and three of the children noted about the stereo sound used for localizing. The navigation sounds, knocking sound and location sounds, were remembered more often than other sounds in the game. This indicates that the navigation sounds were useful for the children. 5.2. Localizing in the game The tactile memory game sets different demands for a blind user in contrast to a partially sighted user. In the game the first challenge for a blind person is to form a mental model of a 3 · 4 matrix. This means that the spatial information gained with the tangible model is represented and symbolized in memory. When moving in the game area the user has to constantly keep his or her position in mind. Another possible way for a blind user to localize is to concentrate in the aid of location sounds. To get benefits of these audio cues a blind person has to have a fairly accurate ability to discriminate between different pitches of tone. With a good ear for tones it is possible to know the exact position in the matrix. One adult pilot user said that he could make localization based on the location sounds rather than keeping constant track of movements. The game play does not put as great a cognitive load for partially sighted users as it does for blind users. Partially sighted users actually see their own position and at the time of a card turn they see the cell being touched. During the game play partially sighted users can concentrate on recalling the vibration and use the visual memory to localize it, whereas blind users do not have these external visual aids for remembering. A blind user has to substitute visual memory aids with a mental image and/or with the location sounds of the game. We considered using a 4 · 4 grid in the game, but decided against it. Based on our pilot studies a larger number of tactile effects would have been recognizable. Navigation would have become more difficult. As our final target user group was visually impaired small children, we aimed at finding a game configuration that would work well for 12–13-year-old visually impaired teenagers. This way the game would become challenging, but presumably not too challenging for the 5–6-year-old children.

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

5.3. Success in the game play Using the tangible model blind children understood the notion of the play area matrix: the row and column layout in which the touchable boxes had been arranged. The choice of the interaction device was a success: there were not any difficulties in operating the push buttons and handling the gamepad. We considered a tactile memory game with 12 cards, that is six different vibrations, as rather demanding, especially for totally blind children. Our assumption was that memorizing various vibrations is more difficult than memorizing plain sounds or real materials that the children are already familiar with in ordinary memory games. However, the result was that blind children managed to finish the tactile memory game in fewer steps than expected. With adult pilot users the average result was 16 pair turns, but there was a great variation in the results between the users. With the sighted pilot child-users the results were 24 and 38 pair turns. The 9-year-old blind pilot user played the game with 21 pair turns, but from time to time she had to be assisted. In the actual tests the children typically completed the game with 15 pair turns, the two extremes being 11 and 24 pair turns (see Table 3). It was evident that the children could play the game without external help, since six of them played the game effortlessly by themselves. The blind child who had a problem with conceptualizing the space had serious difficulties with the game. In the tests he concentrated well and played the game with great enthusiasm, but he clearly could not manage in visualizing the game mentally. The test assistant helped him to find pairs card by card in the game that led to a result of 24 pair turns. With the other six children there was no great difference in results. Even total blindness seemed not to be a crucial factor in succeeding in the game. Results of the partially sighted children were 11, 12, 15 and 15 pair turns, whereas blind children had 13 and 15 pair turns. Eriksson and Ga¨rdenfors (2006) had a similar memory game to ours, but they used only sound feedback. However, our experiences on the usability of the memory game differed quite much from theirs. In their user tests blind children, aged from 9 to 12, did not manage well with the game, whereas in our tests children managed better than expected. Eriksson and Ga¨rdenfors made an assumption Table 3 The pair turns the users made during the game Subject

1

2

3

4

5

6

7

Gender Vision Distant pairs Close pairs Matching pairs Pair turns in total

M B* 11 7 6 24

F P 3 2 6 11

M P 7 2 6 15

F P 8 1 6 15

F B 3 4 6 13

M P 4 2 6 12

F B 2 7 6 15

Abbreviations in the table: M, male; F, female; B, blind, P, partially sighted and *, other handicaps.

203

that the difficulties in game play derived from the reason that children did not comprehend the play area. One significant difference between our and their test procedure is that we taught the children the game idea and layout with the tangible model (Fig. 4). This may be the most important reason for the better results. Of course, other reasons can be the age of the children, occurrence of other undiagnosed disabilities, and the level of visual impairment in the test group. There may also have been differences in memory games and testing procedures, since they did not give details on these. 5.4. Navigation strategies As the adult users played the game we noticed that the majority of them navigated in the matrix row by row. Some preferred to navigate from left to right, while some navigated every other row from left to right and every other row from right to left. At the same time when moving forward in the matrix they systematically turned cards over. They returned back only to pick up a card as they found a card that they recalled having a vibration felt earlier. It was somewhat surprising that the children did not have this kind of systematic navigation strategy. Only one child navigated in the game row by row from left to right. She was partially sighted and got the best result in the whole test series, 11 pair turns. The rest of the children seemed to wander around in the matrix and turned the cards every now and then in a seemingly random order. When investigating the game area, the user can choose to turn the cards so that they are located either closely or distantly in the matrix in relation to each other. In our analysis, the pairs the user chose during the game play were grouped in three categories: distant, close and matching pairs (see Table 3). The term ‘distant pair’ means that the cards were scattered in the matrix. The term ‘close pair’ means that the chosen cards were side by side, next to each other. The term ‘matching pair’ means that the cards had the same effect in them, forming a right pair. Distant pairs and close pairs tell about the spatial separation of the cards, whereas ‘matching pair’ has been added to the chart to show the total number of pair turns. With five of the children the separation can be made based on their preferences on choosing the cards when investigating the game area. The users 3, 4 and 6 tended to choose the cards that were distant in the matrix whereas the users 5 and 7 chose to more often turn cards that were close to each other. Game play of the users 1 and 2 differed from the other five distinctively. The user 1 obviously could not locate the vibrations and for the most part the game was played together with the assistant. The user 2 navigated the cards sequentially from left to right row by row. Part of the cards in pairs were situated distantly because the other card was the last in a row and the other one the first in a row. It can be noted that the totally blind users 5 and 7 tended to slightly more often choose to turn cards that were

204

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

close to each other in the matrix. The children with partial vision were more able to choose cards that were distant in the matrix. This can be explained with visual memory that is a stronger aid in positioning than audio cues (Goldstein, 1999).

was found last by three of seven children. Two-rhythm and gentle vibrations were the easiest to remember, whereas the similar characteristics of decreasing, increasing and intense vibrations made them feel alike. 6. Conclusions

5.5. Memorizing vibrations Before the tests we assumed that there might be a difference in memorability of various vibration effects. In the game there were six different vibration effects: decreasing, increasing, two-rhythm, four-rhythm, gentle and intense (see Table 1). The memorability was measured in two ways: the order the vibration effects were correctly matched in the game, and the number of times a vibration effect was tried out (Fig. 5). On average, gentle vibration was tried out the smallest number of times and intense vibration the largest number of times. During the game each child tried out a gentle vibration card from two to six times, while an intense vibration was tried out from 3 to 10 times. The reason for the gentle vibration demanding less tries than the intense vibration may be that the decreasing and increasing vibrations had common qualities with the intense vibration. All these three effects had the large gamepad motor in use and they had the same magnitude value (10,000). In this sense the gentle vibration making use of the small motor was a distinctive effect in this group. Gentle, intense, decreasing and increasing vibrations had an equal duration distinguishing them from two rhythmic effects. Both decreasing and intense vibrations were tried out from three to ten times. Typically the first pair found was the two-rhythm vibration. As many as four children out of seven found the tworhythm vibration pair first. The decreasing vibration pair

In this paper, we have presented design and evaluation of a tactile memory game that is played with a tactile gamepad by visually impaired children. The gamepad proved to be a usable device for 12–13-year-old children. The device had limited possibilities to produce the effects, but it is an example of devices that are widely available and can be bought by average computer users. The effects for the game were created by using more complex parameters like rhythm. The duration of the effects had to be short, yet the effects had to be distinguishable. To make the effects easily memorized they were given names that described the feel of them. Although one might consider the quality of tactile feedback of the gamepad to be inadequate it was found to be sufficient for the games like the one studied. Thus, tactile low-cost devices have potential in game design for visually impaired children. Since these children differ from each other in many aspects, the equipment setup, particularly the position of the display, should be easily adjustable. The tangible model of the game was found to be highly important in training, since it clearly aided in the conceptualization of the game space. The positioning in the game was enhanced with navigational audio cues that were remembered well by the children. In general, the children got excellent scores in the game and succeeded in game play better than expected. Between blind and partially sighted children there did not seem to be any notable difference in the results. Still, there was a difference in investigation strategies of the play area between blind and partially sighted children. The totally blind children tended to turn cards that were close to each other more often than the partially sighted children, who chose the cards that were scattered in the area. Based on the results gained we have adjusted the game to be suitable for younger, under 7-year-old visually impaired children. Our further research will also continue with designing, implementing and testing of various other multimodal games, including educational games. Acknowledgements

Fig. 5. Average memorability of different vibrations in the game.

This work was funded by the National Technology Agency of Finland (Tekes), Grant 40263/04, by the Academy of Finland (Grant 105555), and by Nordic Development Centre for Rehabilitation Technology (NUH). We thank the children and their parents for participating in the study and the personnel of the Jyva¨skyla¨ School for the Visually Impaired for helping to organize testing.

R. Raisamo et al. / Interacting with Computers 19 (2007) 196–205

References Baptiste-Jessel, N., Tornil, B., Encelle, B., 2004. Using SVG and a force feedback mouse to enable blind people to access ‘‘Graphical’’ Web based documents. In: Proceedings of Ninth International Conference on Computers Helping People with Special Needs (ICCHP’04). Lecture Notes in Computer Science, vol. 3118, pp. 228–235. Barnens, 2006. The Swedish Library of Talking Books and Braille. . Brown, L.M., Brewster, S.A., Purchase, H.C., 2005. A first investigation into the effectiveness of tactons. In: WorldHAPTICS 2005, IEEE Press, pp. 167–176. Chan, A., MacLean, K.E., McGrenere, J., 2005. Learning and identifying haptic icons under workload. In: WorldHAPTICS 2005, IEEE Press, pp. 432–439. Eriksson, Y., Ga¨rdenfors, D., 2006. Computer games for partially sighted and blind children. Project report. . Feintuch, U., Rand, D., Kizony, R., Weiss, P.L., 2004. Promoting Research and Clinical Use of Haptic Feedback in Virtual Environments. In: Proceedings of Fifth International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT’04). University of Reading, UK, pp. 141–147. Flexi Forum 2006. Meeting place for Flexiboard users. . Goldstein, E.B., 1999. Sensation and Perception, fifth ed. Brooks/Cole Publishing Company, Pacific Grove, CA. Hammarlund, J., 1999. Computer Play for Children Who Are Severely Visually Impaired. TRC report nr. 20, 1999. . Holmes, E., Jansson, G., 1997. A touch tablet enhanced with synthetic speech as a display for visually impaired people’s reading of virtual maps. In: Proceedings of CSUN 12th Annual Conference on Technology for People with Disabilities. California State University, Northridge. IGDA Accessibility Group, 2004. Accessibility in games: motivations and approaches. White Paper, International Game Developers Association. . Immersion, 2006. Immersion Studio for Gaming: Software Development Kit. . Inclusive Technology, 2006. IntelliKeys Transparent Overlays. . Jyva¨skyla¨, 2006. Jyva¨skyla¨ School for the visually impaired. . Landau, S., Russell, M., Gourgey, K., Erin, J., Cowan, J., 2003. Use of the talking tactile tablet in mathematics testing. Journal of Visual Impairment and Blindness 97 (2), 85–96. Lumbreras, M., Sa´nchez, J., 1999. Interactive 3D sound hyperstories for blind children. In: Proceedings of Conference on Human Factors in Computing Systems (CHI’99). ACM Press, pp. 318–325. Meijer, P.B.L., 2006. The vOICe Java applet. . Nemec, V., Sporka, A.J., Slavik, P., 2004. Haptic and spatial audio based navigation of visually impaired users in virtual environment using low cost devices. In: Proceedings of User-Centered Interaction Paradigms

205

for Universal Access in the Information Society: Eighth ERCIM Workshop on User Interfaces for All. Lecture Notes in Computer Science, vol. 3196, pp. 452–459. Nielsen, J., 1993. Usability Engineering. Academic Press. Patoma¨ki, S., Raisamo, R., Salo, J., Pasto, V., Hippula, A. 2004. Experiences on haptic interfaces for visually impaired young children. In: Proceedings of Sixth International Conference on Multimodal Interfaces (ICMI’04). ACM Press, pp. 281–288. Raisamo, R., Patoma¨ki, S., Raisamo, J., Hasu, M., Pasto, V., Kosonen, K., Pakkanen, T., 2005. Determining the usefulness of affordable haptic feedback devices: from games to applications helping the visually impaired. In: Proceedings of HCI International 2005. Lawrence Erlbaum Associates. Ramloll, R., Yu, W., Brewster, S.A., Riedel, B., Burton, A.M., Dimigen, G. 2000. Constructing sonified haptic line graphs for the blind student: first steps. In: Proceedings of ASSETS 2000. ACM Press, pp. 17–25. Roth, P., Petrucci, L., Pun, T., 2000. From dots to shapes: an auditory haptic game platform for teaching geometry to blind pupils. In: Proceedings of International conference on computers helping people with special needs (ICCHP 2000), pp. 603–610. Roth, P., Kamel, H., Petrucci, L., Pun, T., 2002. A comparison of three nonvisual methods for presenting scientific graphs. Journal of Visual Impairment and Blindness 96 (6), 420–428. Saarinen, R., Ja¨rvi, J., Raisamo, R., Tuominen, E., Kangassalo, M., Peltola, K., Salo, J., 2006. Supporting visually impaired children with software agents in a multimodal learning environment. Virtual Reality 9 (2–3), 108–117, Springer, 2006. Sa´nchez, J., Baloian, N., Hassler, T., Hoppe, U., 2003. AudioBattleship: blind learners collaboration through sound. In: Conference on Human Factors in Computing Systems (CHI’03), Extended abstracts. ACM Press, pp. 798–799. SensAble, 2006. SensAble Technologies, Inc. PHANTOM Devices. . Sjo¨stro¨m, C., 2002. Non-visual haptic interaction design: guidelines and applications. Doctoral dissertation, Certec, Lund Institute of Technology. TIM (2001), Tactile Interactive Multimedia Computer Games for Visually Impaired Children, project IST-2000-25298. Deliverable 12: Game function report. . Tiresias.org (2006). Guidelines for the Design of Accessible Information and Communication Technology Systems. . Touch Graphics, 2006. Touch graphics, inc. Talking tactile tablet. . van den Doel, K. (2003) SoundView: sensing color images by kinesthetic audio. In: Proceedings of the International Conference on Auditory Display (ICAD 2003), Boston University, USA. . van Erp, J.B.F., Spape´, M.M.A., 2003. Distilling the underlying dimensions of tactile melodies. In: Proceedings of Eurohaptics 2003, Dublin, Ireland, pp. 111–120. VisionConnection (2006). Accessibility and assistive technology, a compendium of available assistive technology products. VisionConnection, Lighthouse International, Inc. .