Design of Interactive Toy As Support Tool in STEM Education For Children With Special Needs
Design of Interactive Toy As Support Tool in STEM Education For Children With Special Needs
1 Introduction
STEM works with active learning and is therefore related to the design and
implementation of projects that encourage the student to acquire a set of aspects such as
dialogue, collaboration, development and knowledge construction. To work with
STEM in the classroom, it is necessary for the teacher to relate two or more areas
connected to a real-life problem [2]. It is very often selected for use in primary edu-
cation, since a single teacher is responsible for teaching the same students most or all of
their subjects, meaning that it can be integrated within a multidisciplinary treatment. It
also has a greater impact in primary since many high school students have already lost
interest in science, a situation even more the case among female students [3].
Tsupros et al. [4] define STEM education as an interdisciplinary approach to
learning, in which complex concepts applied to real-life problems are taught, where
science, technology, engineering and mathematics is applied, and which occurs in
contexts that manage to connect school, community, work and business in order to
foster STEM competencies. STEM education can furthermore impact jobs, productivity
and skills in different sectors such as health, technology, innovation and production.
In Colombia, ever-decreasing numbers of enrollments in scientific careers remains
cause for concern. One difficulty lies in the lack of popularity of areas such as math-
ematics, physics, programming and chemistry. A report by the Ministry of Education
shows that less than 30% of people graduate from technical or engineering subjects [5],
which today generates a lack of interest among students enrolling in different areas of
knowledge related to engineering, more so if these relate to electronics or systems.
Disinterest is currently perceived not only nationally but in the international arena
and the term STEM education therefore began to be used as an option for promoting
scientific literacy and encouraging a greater interest in the young. A study conducted in
2006, meanwhile, showed a strong positive relationship between students taking sci-
ence subjects and choosing higher education study to continue in STEM-related dis-
ciplines [6].
As regards the evolution of education, another aspect to bear in mind is that
children use technology daily to communicate, play or learn. Technology in this
context would refer not only to a Tablet, for example, but also to books, games, toys
and dolls that involve digital and electronic elements. Nowadays, children are growing
up with interactive technologies. The ways in which they learn and interact with others
is changing [7]. It is therefore important to understand the child in order to know how
to design and integrate all of these technologies.
In this article we present a number of ways in which interactive systems applied to
STEM education aimed at children with special needs can be designed. Section 2
describes a set of related work that has been done based on STEM education; Sect. 3
briefly describes concepts related to this work, such as interactive systems and STEM
education; Sect. 3.1 outlines the problems that children with special needs face and
how technology and interaction may change depending on their need; Sect. 3.5
describes the procedure of this work and the methodology that used in designing this
proposal; and finally, Sect. 4 presents some conclusions and anticipated future work.
Design of Interactive Toy as Support Tool in STEM Education for CSN 115
2 Background
2.1 STEM Education
The principle of STEM education aimed at children may be considered to be the
initiation of Logo Turtles in 1967 by Seymour Papert [8], who brought out a high-level
programming language aimed at children in normal conditions. Papert developed an
approach based on Piaget’s cognitive constructivism [9, 10], involving different areas
of knowledge such as philosophy, educational theory, artificial intelligence, cognitive
science, neuroscience, robotic programming and computer science. Another work
proposed by Mitch Resnick and MIT [11] developed a visual programming language
built by blocks, called Scratch [12]. This language is designed especially for ages
between 8 and 16 years, but is used by people of all ages.
This is also the case of research carried out by Tufts University, which presented a
robot called KIBO for children’s education from 4 to 7 years old [13]. The KIBO robot
features light, sound and movement sensors by which the child can develop pro-
gramming skills using ten wooden cubes, which he must order according to the
instructions he wants the robot to carry out. Another work, by the MIT laboratory, is
called LittleBits [14]. In LittleBits, electronics is offered as learning material, using
interactive objects such as electronic components (switches, batteries, buttons and
sensors, etc.) and electrical circuits with which different science experiments can be
built. This electronic system enables projects to be built using modular blocks. It
features four types of blocks, each identified with a specific color: Source (blue), Inputs
(pink), Wired (orange) and Outputs (green).
Piaget meanwhile has had a great influence on STEM education, since he indicates
in his studies the way to conceive development in children [9], stating that they are
typically extremely curious in their behavior because they are trying to discover and
interpret their world. Children seek knowledge through the different interactions they
conduct with the environment. Piaget was thus very interested in how they think about
problems and their solution.
STEM education involves several areas of knowledge and is beginning to be
included in primary education in different research topics such as educational robotics
and computational thinking. The main interest in involving STEM education in the
classrooms from the very youngest children is so that when they become teenagers they
understand the different roles of engineering within society and how they can partic-
ipate in it, to motivate them and to solve the problem of contextualizing areas such as
mathematics or science [15]. It can currently be seen that computer classes in schools
are taught by teachers with knowledge in the area of technology, and they usually limit
themselves to teaching routines or development instructions. However, programming
can be applied to other areas of knowledge and must not necessarily only be within the
subject of computer science. Carnevale et al. [16] have proposed a set of cognitive
competencies related with STEM, shown in Table 1.
116 S. Cano et al.
3 Proposal
The proposal is based on building a principal character, with one or several missions to
fulfill, with whom the child can interact in areas of knowledge such as mathematics,
literacy and computational thinking. The methodology chosen, Design Thinking [21],
involves five stages: Empathize, Define, Devise, Prototype and Test. Below we
describe what each stage consists of and what was done in each stage.
118 S. Cano et al.
3.1 Empathize
In this stage it is important to understand the needs and the way in which the children
interact with other children and the teacher in the classroom. It is also vital to
understand more about the methods and learning strategies teachers use with their
students (Invariant method, among others); we therefore attended various classes, in
literacy and mathematics.
The participants were children with a cochlear implant (11 children between 7 and
11 years, from the Institute of Blind and Deaf Children in the Valle del Cauca,
Colombia) and children with intellectual disability (7 children between 8 and 16 years,
from the Tobias Emanuel Institute, Cali, Colombia). The analysis consists of identi-
fying different variables that can affect interaction and these should be indicated
according to the needs of the child, according to his/her disability. The characteristics
analyzed are: level of disability, learning styles, age, academic level, gender, interests
and behaviors. Therefore, the children with cochlear implants have to rely on the visual
channel, as they are in the process of acquiring the language.
The children with intellectual disabilities on the other hand can rely on both visual
and auditory channels to memorize concepts. Profile analysis of the child with special
needs has been analyzed in prior research [22] in which different methods are used to
evaluate the user experience in children with a cochlear implant, such as direct
observation, Drawing Intervention [23], interviews, and Picture Card [24]. Direct
observation and interviews were applied to children with intellectual disabilities. Some
of these children were already 16-year-olds while others were 8-year-olds, so doing a
single activity that involved everyone was difficult. It was therefore decided to apply
only direct observation in the use of technologies and to conduct interviews with
teachers.
In the interviews, questions addressed to the teachers included: What problems do
the children have in developing basic skills (attention, perception, orientation, etc.)?
What technological elements do you use in the classroom with the children? What are
the biggest problems faced by the children in the areas of literacy and mathematics? Do
the children have any didactic material to work with outside the classroom on specific
problems? Do you use a particular learning method?.
Attention is a problem for children when given their cochlear implant, since they
must learn to recognize each of the sounds. During that process the children use their
visual channel to communicate, so teachers have a lot of visual material to explain the
meaning of the words. Meanwhile, with the aim of analyzing the orientation of the
children, a test was performed, consisted in them having to interact with a game
through the movement of a snail climbing a tree with branches in different orientations
(Fig. 1). None of the children could finish the first level of the game as it proved
difficult for them to locate the snail in the positions of up, down, left or right.
Another question that gathered information was the use of technologies, where it
was observed that most of the children, through belonging to a low or medium socio-
economic level, do not have a Tablet or a PC. But they came to make good use of it in
the classroom during their computer class, where the children interact with related
applications in learning literacy through existing educational games. The Tablets that
are available are iPads and Android, but in the IoT most of the applications are not free
Design of Interactive Toy as Support Tool in STEM Education for CSN 119
and with Android they were not able to find an application that could be useful for their
teaching strategy. In addition, the support material they have is either physical or
digital, but not a combination of both.
The answers to another question, this time about the teaching method used,
revealed that the Invariant method [25] was used for children with a cochlear implant,
while for children with cognitive disabilities, the ABCD-Spanish method referred to in
[18, 26] was favored. The focus of each of these methods is auditory and visual
attention; auditory attention especially so for children with cochlear implant, while for
children with cognitive disabilities, auditory and visual attention play equal roles.
Another problem that children face is their poor acquisition of vocabulary when they
want to express something and cannot find the words to do so. Furthermore, most of
these activities work only in the classroom, because at home parents often fail to
continue reinforcing this type of activity with them so that they can memorize
vocabulary or identify each of the sounds aurally.
Fig. 2. Designing a character. Institute of Blind and Deaf Children of the Valle del Cauca.
120 S. Cano et al.
Meanwhile, in work done with the evaluation methods applied to children with
cochlear implant [22] we worked with the Drawing Intervention method, where an
activity was carried out with the children, which consisted in constructing a character
and giving each character a name. It was difficult, however, to assign names, because
these students were only in the process of acquiring language and communicating with
them was difficult (Fig. 2).
In the same work, the Picture Cards method was also employed. Emotion cards
were used so that children could assign a scale of emotion to certain questions they
were asked about technology or to relate a color with an emotion. However, although at
the beginning a scale of five emotions was used, this had to be changed to only two
emotions - happy and sad - since it proved difficult for them to recognize the other three
emotions [28].
Teacher interviews were also conducted, in which a set of questions relating to the
child’s interaction within the classroom were formulated along with strategies and
learning methods used in the classroom with the children. In addition, teachers were
asked an open question, describing their experiences of the difficulties faced by chil-
dren in the teaching of literacy and mathematics.
3.2 Define
Once the needs of the children and teachers are known, all the collected information
obtained through the methods used in the previous stage is analyzed. The information
collected is used to define the needs of the children [18, 29], which may vary
depending on their disability, as well as their interaction.
Children with cochlear implants have greater difficulties in the area of literacy,
because they are faced with first having to learn to listen so that they can then learn to
speak and write. It is therefore a lengthy process that requires them to work not only
within the classroom, but outside of it [29]. This problem can become greater, since
some children receive their implants at a later age, which is reflected in a retarded
learning and academic level.
Children with intellectual disabilities on the other hand face problems at the level of
literacy. For them, the activities must be very repetitive to get to memorize the words
[17]. Although they have visual and auditory memory they present intellectual retar-
dation compared to a child in normal conditions.
3.3 Ideate
The objective in this stage is to have several alternative solutions and choose the one
that best fits according to the needs identified in the children. The idea that was selected
is that the child can interact with a physical and digital environment, which allows him
to interact with the other children in his class according to his needs.
Therefore, as a design, an interactive physical system was proposed that might be
an animal, since a toy animal would likely generate greater confidence in the child.
A teddy bear was chosen, so that the children could adopt it as their pet or inseparable
friend and have more confidence in themselves. It was decided to call the bear “Tobi”.
The advantage of offering smart toys is that they provide a rich playable environment in
Design of Interactive Toy as Support Tool in STEM Education for CSN 121
terms of use of multimedia and interactive usability, integrating the digital with tra-
ditional dolls [30]. As indicated by Lampe & Hinske [31], adding multimedia content
to these traditional dolls can increase the fantasy quality of play and provide a more
creative environment for children.
Furthermore, interaction ought to be considered with real elements of interaction
with the physical character and suitable communication channels. Therefore, the
auditory and visual channel was taken into account, integrating mobile technology.
3.4 Prototype
In this stage, the purpose is to convert the proposed idea into a functional prototype, so
that an analysis was carried out of the different sensors and components that Tobi will
feature incorporated into his body.
An RC-522 RFID sensor, a GY-50 Gyroscope and an HC06 Bluetooth sensor were
incorporated in the body of the toy bear Tobi (Fig. 2) to establish communication
between the teddy bear and the mobile device. For visual feedback, a neopixel was
used that acts as the bear’s heart. Its function is to exert positive feedback to the child if
it has done the activity correctly, so that all the LEDs turn red and the application
reproduces a spoken text, “My heart is happy now”. Two feedback types are thus
received - visual and auditory. If feedback is negative, the LEDs flash on and off,
indicating a negative emotion of the bear. At the same time, it reproduces the text “That
is not correct”, as the activity was not carried out correctly.
Fig. 3. Teddy bear adapted with Arduino and sensors (RFID, Gyroscope and Bluetooth) and a
Neopixel
In Fig. 3, the Neopixel is seen to be visible in the chest of the bear, while the RFID
(Radio-Frequency Identification) sensor located internally will allow the child to interact
by means of RFID cards and validate whether a card selection is correct or not. The
nano-Arduino, Gyroscope and Bluetooth sensors are also placed internally at the back.
122 S. Cano et al.
Fig. 4. Tobi the bear with RFID cards and the neopixel with all LEDs lit.
The cards with which the child interacts are RFID cards as shown in Fig. 4. The
cards represent the pictogram that must be associated according to the activity, either in
the area of mathematics, literacy or basic programming instructions (start, up, right,
left, down and finish). The cards feature a pictogram very similar to what children draw
in their early stages of drawing, as it is intended that Toby act as any other child would
act and that the child can thus have the confidence to perform their activities. Each card
features the respective text in lowercase.
Fig. 5. Some Basic level cards for the areas of literacy, mathematics and programming. (Color
figure online)
Design of Interactive Toy as Support Tool in STEM Education for CSN 123
The cards are related to a level of difficulty (basic, medium, advanced and expert),
so that each level corresponds to a card of different color. The white cards correspond
to the basic level and basic activities are associated in the different areas of literacy,
mathematics and programming (Fig. 5). The colors then for each level are: Basic,
white; Medium, blue; Advanced, green, and Expert, orange.
The mobile application will connect through the Bluetooth device. In Fig. 6a, it can
be seen that once the device has established a Bluetooth connection, it shows a main
interface with the different areas (mathematics, reading, writing, location and pro-
gramming) into which the player seeks to enter, accompanied by a voice message “Hi,
I’m Toby. Let’s play!” The mathematics and literacy areas (Fig. 6b, d) have the same
operating structure, which by means of a voice instruction using the Google Text-
toSpeech library, Toby asks the child to search for a certain word or determine a
number of Elements. The child must select a card and place it on the chest of the bear to
validate if it is correct. If the answer is correct, a pictogram appears and the text appears
at the bottom, as shown in Fig. 6b and c. Pictograms are taken from ARASAAC [32].
These have been made for communication boards for people with special needs.
The area of location is worked using spatial orientation, where the child interacts
with the application and the physical object. This is done in such a way that the child
manages to make the movement, within his environment and real space, when the
124 S. Cano et al.
application indicates certain orders to be carried out with Toby, such as: “Turn him to
the left”, “Turn him to the right”, “Lift him up” and “Put him down again”. The
Gyroscope consequently has limits of each position that have been configured with
parameters in the application, and which serve to identify each position and work out
whether or not it has been done correctly.
In the area of programming, the interaction is a digital maze in which Toby must
find his way home. The child must therefore locate the set of RFID cards that make it
possible to carry out this activity. In this first level we work with algorithmic thinking,
where a set of RFID cards is selected in order to follow a correct sequence to find the
way out.
3.5 Test
One evaluation technique is heuristic evaluation [33], which consists of a small group
of evaluators who evaluate the system using a set of heuristics as guides for the
inspection of the system. However, there is a dearth of research in which a heuristic
evaluation has been applied for the design of interactive systems. The heuristics pro-
posed by Nielsen and Molich [33] and Scholtz [34] were therefore used.
In the book “Psychology of Everyday Objects” describes the principle of visibility
and information, which generates important questions to be taken into account in
design and interaction when working with physical and digital elements, questions such
as: Which parts are mobile and which are fixed? Which part should be handled? What
kind of movement is possible? What information should be communicated? How
should it be communicated? This set of questions should thus be taken into account to
improve the design of the interactive system for children whose functional and cog-
nitive conditions may affect the interaction and understanding of the system (Fig. 7).
The heuristics evaluated with the interactive system are:
1. Visibility and state of the system (Nielsen)
2. Coincidence between the system and the real world (Nielsen)
3. Control and freedom for the user (Nielsen)
4. Minimize the memory load (Nielsen)
5. Aesthetic and minimalist design (Nielsen)
6. Presence of the information and presentation (Scholtz)
7. Performance of the interaction (Scholtz)
The results obtained from the heuristic evaluation carried out with two people,
where it was identified that in the visibility and state of the system, the feedback
messages that are presented to the child should be improved. The same applies to the
prevention of errors in the application, so that users are able to return to a previous state
if they have made critical errors.
Design of Interactive Toy as Support Tool in STEM Education for CSN 125
An interactive system design that simultaneously integrates a real and a digital envi-
ronment is extremely useful in motivating children during their learning in different
approaches related to STEM education. STEM education is today being employed only
with children in regular conditions. It may also prove to be a very positive concept with
which to work with special needs children, so that some thinking might be stimulated
involving STEM education. Three important areas were worked on in the present
research: literacy, mathematics and computational thinking, so that as the children
progress through the levels they are able to acquire certain competences in each one.
The idea is then that finally in the expert level, all of the areas worked independently in
the earlier levels are integrated in one single activity.
In designing the Tobi interactive teddy bear, low-cost electronic components were
used with the aim of making this educational product easily accessible to parents in the
low and medium socio-economic brackets. As future work we would like to go a little
deeper into each difficulty level. We would also expect to design the activities and
ensure that it is entirely multi-modal, in order that any child with special needs is able
to interact with Toby. We also hope to test each of the areas with a group of children
with cochlear implants and children with cognitive disabilities. Finally, it is also
planned to make adjustments to Toby as a system in which it is not only a bear, but a
character that the child can condition as his or her classmate.
Acknowlegements. This work was part supported by the OCDS funded project OCDS-
CUD2016/08 of the University of Balearic Islands, together with the Institute of Blind and Deaf
Children of Valle del Cauca and Institute Tobias Emanuel, Cali – Colombia.
126 S. Cano et al.
References
1. Pequeños Científicos (Little Scientists) STEM-Colombia. [Link]
org. Accessed 11 Dec 2017
2. Moore, T., Stohlmann, M., Wang, H., Roehrig, G.: Implementation and integration of
engineering in K-12 STEM education. In: Purzer, S., Strobel, J., Cardella, M. (eds.)
Engineering in Pre-College Settings: Research into Practice, pp. 35–60. Purdue Press, West
Lafayette (2014)
3. Abell, S., Lederman, N.G.: Handbook of Research on Science Education. Lawrence
Erlbaium Associates, New Jersey (2006)
4. Tsupros, N., Kohler, R., Hallinen, J.: STEM education: a project to identify the missing
components, Intermediate Unit 1 and Carnegie Mellon, Pennsylvania (2009)
5. [Link]
Accessed 16 June 2018
6. Tai, R.H., Qi Liu, C., Maltese, A.V., Fan, X.: Planning early for careers in science. Science
312, 1143–1145 (2006)
7. Hourcade, J.P.: Child-Computer Interaction. CreateSpace Independent Publishing Platform,
North Charleston (2015)
8. Papert, S.: Teaching Children Thinking. Mathematics Teaching, no 58. Spring (1972)
9. Piaget, J., Bärbel, I.: The Psychology of Child. Basic Books, New York (1969)
10. Papert, S.: Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, New York
(1980)
11. Resnick, M., Kafai, Y., Maeda, J.: A Networked, Media-Rich. Programming Environment to
Enhance Technological Fluency at After-School Centers in Economically Disadvantaged
Communities. Proposal to National Science Foundation (2003)
12. Scratch. Programming language online. [Link] Accessed 16 June 2018
13. Sullivan, A., Elkin, M., Bers, M.U.: KIBO robot demo: engaging young children in
programming and engineering. In: Proceedings of the 14th International Conference on
Interaction Design and Children, pp. 418–421 (2015). [Link]
2771868
14. Bdeir, A., Ullich, T.: Electronics as material: LittleBits. In: Proceedings of the Fifth
International Conference on Tangible, Embedded and Embodied Interaction, pp. 341–344
(2010). [Link]
15. Brophy, S., Klein, S., Portsmore, M., Rodger, C.: Advancing engineering education in P-12
classrooms. J. Eng. Educ. 97(3), 369–387 (2008). [Link]
tb00985.x
16. Carnevale, A., Smith, N., Melton, M.: STEM, Georgetown University, Washington, D.C.
(2011)
17. Cano, S., Collazos, C., Fardoun, H.M., Alghazzawi, D.M., Albarakati, A.: Model based on
learning needs of children with auditory impairment. In: Meiselwitz, G. (ed.) SCSM 2016.
LNCS, vol. 9742, pp. 324–334. Springer, Cham (2016). [Link]
39910-2_30
18. Cano, S., Palta, A., Posso, F., Peñeñory, V.M.: Towards designing a serious game for
literacy in children with moderate cognitive disability. In: Proceedings of the XVIII
International Conference on Human Computer Interaction (Interacción 2017), Article 19, 5
pages. ACM, New York (2017). [Link]
19. Pascual-Espada, J., Sanjuan-Martinez, O., G-Bustelo, B.C.P., Lovelle, J.M.C.: Virtual
objects on the Internet of Things. Int. J. Interact. Multimed. Artif. Intell. 1(4) (2011). https://
[Link]/10.9781/ijimai.2011.144
Design of Interactive Toy as Support Tool in STEM Education for CSN 127
20. de la Guia, E., Camacho, V.L., Orozco-Barbosa, L., Brea Lujan, V.M., Penichet, V.M.R.,
Perez, M.L.: Introducing IoT and wearable technologies into task-based language learning
for young children. IEEE Trans. Learn. Technol. 9(4) (2016). [Link]
2016.2557333
21. Brown, T.: Change by Design: How Design Thinking Transforms Organizations and
Inspires Innovation. Harper Collins, New York (2009)
22. Florez-Aristizabal, L., Cano, S., Collazos, C., Moreira, F., Alghazzawi, D., Fardoun, H.:
Tools and methods applied in interactive systems to evaluate the user experience with
deaf/hard of hearing children. In: Conference: Technological Ecosystems for Enhancing
Multiculturality, TEEM (2017). [Link]
23. Barendregt, W., Bekker, T.: Exploring the potential of the drawing intervention method for
design and evaluation by young children. In: CHI ’13 Extended Abstracts on Human Factors
in Computing Systems (CHI EA 2013), pp. 193–198 (2013). [Link]
2468356.2468392
24. Barendregt, W., Bekker, M.M., Baauw, E.: Development and evaluation of the problem
identification picture cards method. Cogn. Technol. Work 10(2), 95–105 (2008). [Link]
org/10.1007/s10111-007-0066-z
25. Solovieva, Y., Rojas, L.Q.: Método de formación de lectura para la corrección de
dificultades en el desarrollo. Tesis de maestría en diagnóstico y rehabilitación neuropsi-
cológica, Universidad Autónoma de Puebla, México (2012)
26. Cano, S., Alghazzawi, D.M., Arteaga, J.M., Fardoun, H.M., Collazos, C.A., Amador, V.B.:
Applying the information search process model to analyze aspects in the design of serious
games for children with hearing impairment. Univ. Access Inf. Soc. 17(1), 83–95 (2018).
[Link]
27. Cano, S., Collazos, C.A., Aristizábal, L.F., Gonzalez, C.S., Moreira, F.: Towards a
methodology for user experience assessment of serious games with children with cochlear
implants. Telematics Inform. 35(4), 993–1004 (2018). [Link]
011
28. Flórez Aristizábal, L., Cano, S., Collazos, C.A., Moreira, F., Alghazzawi, D.M., Fardoun,
H.: Tools and methods applied in interactive system to evaluate the user experience with
deaf/hard of hearing children. In: Proceedings of the 5th International Conference on
Technological Ecosystem for Enhancing Multiculturality (2017). [Link]
3144826.3145365
29. MBOT, Robótica Educativa. [Link] Accessed 16 June 2018
30. Kara, N., Aydin, C.C., Cagiltay, K.: Investigating the activities of children toward a smart
storytelling toy. Educ. Technol. Soc. 16(1), 28–43 (2013)
31. Lampe, M., Hinske, S.: Integrating interactive learning experiences into augmented toy
environments. In: Proceedings of the Workshop on Pervasive Learning 2007, Toronto,
pp. 1–9 (2007)
32. Portal Aragonés de la Comunicación Aumentativa y Alternativa, ARASAAC. [Link].
org. Accessed 16 June 2018
33. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the
ACM CH 1990, pp. 249–256 (1990)
34. Scholtz, J.: Evaluation methods for human-system performance of intelligent systems. In:
Proceedings of PERMIS 2002 (2002)