The Matrix Decoded

Teacher Information

Background Information
Have you ever noticed that when you really think about what you are reading, your lips
sometimes move and you silently “speak” the words? There’s a reason for that. The
brain sends out signals that connect what we think and what we say. We usually don’t
“read along with our lips” because it slows us down when we read. Still, it is a natural
connection. Researchers at NASA are learning how that thought-speech connection
can be used to help us communicate even better.

To make this technology work, researchers place electrodes on the larynx of the user.
The electrodes record sound signals. The device doesn’t record words, though, it
records sounds. And each sound is assigned a pair of numbers. Researchers use
numbers to code the alphabet into a matrix. For example 1,1 is used to record the
letter a. The letter b is shown as 1,2 and so on. This coding is used because
subvocal speech technology doesn’t recognize the alphabet. In a test, NASA
researchers used the new technology to control a small Mars rover. They were able
to direct the rover to different locations without any audible sounds. Commands for
the rover were thought, but not spoken. Using the new technology, the rover moved in
the correct direction.

In this lesson, students will learn how the researchers use a matrix to assign numbers
to each sound. They will then use this code to decode a series of words and sayings.

Key Concepts and Student Information

• Matrix – a table of intersecting rows and columns
• Subvocal speech – communicating without actually talking

Materials
NASAexplores 5-8 article, “Do You Hear What I’m Saying?” (1 copy per student)
Student Sheets (1 copy per student)

Procedure
1. Read the NASAexplores 5-8 article, “Do You Hear What I’m Saying?” Discuss.
2. Revisit the paragraph that discusses how researchers use a matrix to assign
number pairs to different letters of the alphabet.
3. Explain that a matrix is a grid of intersecting rows and columns. Distribute the
Student Sheets to give students an example. Be sure to point out that letter a is
represented by the pair 1,1 just like in the article.
4. Go over Student Sheet instructions. Answer questions before students begin.

Enrichment Activities
• Discuss other uses for the matrix coding system.
• Have students come up with their own puzzles to be decoded.
• Discuss possible uses for subvocal speech technology.

Answer to Student Sheet Message: In the future, subvocal technology could be

used by SCUBA divers to speak to each other while under water.
The Matrix Decoded

Do You Hear What I’m Saying?

NASAexplores article, August 25, 2004
[Link]

Have you ever noticed that when you

concentrate on what you’re reading,
your lips sometimes move and
silently speak the words? There’s a
reason for that. The brain sends out
electrical signals that connect
thinking and speech. We often avoid
“reading with our lips” because it
slows down the process, but it’s a
natural connection. Researchers at
NASA are learning how that thought-
speech connection can be used for
better communication.

Imagine being able to speak to someone without ever making a sound. That’s the idea
behind the subvocal speech project at NASA’s Ames Research Center. Researchers
have found that when people think of something to say, the brain sends out faint
electrical signals with that same message—even if the person never speaks the words
out loud. When researchers magnify those electrical signals and translate them into
real words, people can think a message and the message will be turned into words.
This is a form of voice recognition. It sends a message while cutting out the breath
and sound vibration parts of speech. This technology is called subvocal speech.
Subvocal means “below the spoken voice,” referring to the nerve signals to the
muscles that we use when we produce sounds.

Yelling is one way of sending a message

to others. Why would it be useful to speak
without using your voice? “People
working in high-noise areas, such as
construction sites or airports, could use
subvocal communication to send
commands and instruction for tasks when
they otherwise couldn’t be heard,” says
Chuck Jorgensen, chief scientist for
neuroengineering at NASA Ames.
“People who have suffered damage to
their vocal cords could communicate by
thinking, rather than speaking, their thoughts. Space explorers who may get injured
while on Mars could use subvocal speech to call for help. And when sitting in a
crowded room, people could ask each other a private question without bothering the
rest of the people.” Passwords could be entered without anyone else hearing the
word. Deep sea divers could communicate to diving partners in an environment where
words are impossible. Telephone calls could be conducted silently.
The Matrix Decoded

Do You Hear What I’m Saying? (Continued)

Don’t get the wrong idea about subvocal speech, though. It’s not mind reading. It’s just
like typing a word on a computer keyboard, but the message is sent by thinking the
idea rather than by typing the words. Subvocal speech is totally voluntary; nobody
forces anyone to think on demand.

To make this technology work, small

electrodes are placed on the throat near the
larynx. The signal that is created by thinking
is captured and recorded. The list of
recorded nerve patterns creates a type of
dictionary that a computer remembers. So
far, the system is able to recognize a few
words, and only words spoken by a
particular voice. As the system improves, it
will be able to recognize more words and
more speech patterns.

Researchers use numbers to code the alphabet into a matrix, so that 1,1 is the letter
a, and 1,2 is b, and so on. They do this because subvocal speech technology cannot
yet recognize the alphabet. In a demonstration, NASA researchers were able to
control a small Mars rover by directing the rover without any audible sound to go to
different locations. Commands such as right and left were thought,
but not spoken, and the rover responded.

Everyone's voice creates a unique [Link] electronic

“signature” put out by your muscles is unique, just like fingerprints,
Jorgensen says. Identifying the various signals that translate into
different sounds and words is the challenge the researchers face
now. Not only is every word different, but each person pronounces
those words a bit differently, too. Regional accents and individual
pronunciation create big differences in the signal.

Because English has so many different ways to pronounce the

same letters, the researchers decided to use Latin when they first
experimented with subvocal speech. English has
diphthongs—combinations of vowels that create unique sounds,
such as the word “voice”—and depending on how the letters are
arranged, letters can have several different pronunciations. “Nasal
languages, such as English and French, turned out to be much harder to develop into
subvocal speech,” says Jorgensen. “Languages with hard, guttural sounds, like
German and Japanese, are easier. They are more consistent in how each letter is
pronounced. And, Latin was the easiest tool for pulling out sounds and making
consistent pronunciation. So, there’s one more reason to know another language!”
The Matrix Decoded

Student Sheets

Objective
To understand how to decode a message using a matrix.

Materials
Matrix Decoded Worksheet
Do You Hear What I’m Saying article

Procedure
1. Read the “Do You Hear What I’m Saying?” article from NASAexplores.
2. Practice identifying letters using the matrix code on the Matrix Decoded student
worksheet
3. When you have finished the Decoding Practice and Coding Practice, go on to
decode the message on worksheet #2.
4. After decoding the message, create a message of your own, and code it according
to the matrix.
5. Have someone else in your group decode your message.
The Matrix Decoded

Student Worksheet

Subvocal speech technology cannot recognize the alphabet yet, so researchers use
numbers to code the alphabet into a matrix, so that (1,1) is the letter A, and (1,2) is B, and
so on.

The box below is called a matrix; it is a grid with intersecting rows and columns.
Researchers at NASA use a matrix, like the one below, to assign a letter to a pair of
numbers. For example, the pair of numbers (3,2) is found on the matrix by finding the
number 3 in the top row and seeing where it intersects with the row containing the
number two in the left vertical column.

1 2 3 4 5
1 A B C D E
2 F G H I J
3 K L M N O
4 P Q R S T
5 U V W X Y
6 Z

In the matrix above the horizontal and vertical rows intersect at the letter H. Therefore,
whenever the pair of numbers (3,2) is seen, the computer will interpret the code as the
letter H. Some other examples are:
(4,4) = S
(1,5) = U

DECODING PRACTICE:

( 2, 4) = _________ (5, 3) = __________ (3, 5) = __________

You can also use the matrix backward to create your own secret message! In order to do
this first, create your own secret message. Then, for each letter in the message find the
corresponding pair of numbers. For example, if your message you wanted code is the
word NASA, then your secret code would be:
NASA = (4,3) (1,1) (4,4) (1,1)

CODING PRACTICE:

R = ( ____, ____) C = ( ____, ____) I = ( ____, ____)

The Matrix Decoded

Student Worksheet #2

Try out your new decoding skills, use this matrix to decode the secret messages below.

1 2 3 4 5
1 A B C D E
2 F G H I J
3 K L M N O
4 P Q R S T
5 U V W X Y
6 Z

___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___,
(4,2) (4,3) (5,4) (3,2) (5,1) (1,2) (1,5) (5,4) (1,5) (3,4) (5,1)

___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___
(4,4) (1,5) (2,1) (2,5) (5,3) (3,1) (1,1) (2,3) (5,4) (5,1) (3,1) (3,2) (4,3) (5,3) (2,3) (5,3) (2,2) (5,5)

___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___
(3,1) (5,3) (1,5) (2,3) (4,1) (2,1) (5,1) (1,5) (4,4) (5,1) (4,1) (2,1) (5,5)

___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___
(4,4) (3,1) (1,5) (2,1) (1,1) (4,1) (4,2) (2,5) (5,1) (3,4) (4,4) (5,4) (5,3)

___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___
(4,4) (1,4) (5,1) (1,1) (1,3) (5,4) (5,3) (5,1) (1,1) (3,1) (3,2) (5,3) (5,4) (3,2) (5,1) (3,4)

___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___.
(3,5) (3,2) (4,2) (2,3) (5,1) (1,5) (4,3) (4,1) (5,1) (3,4) (3,5) (1,1) (5,4) (5,1) (3,4)

Use the space below to write out your own short secret message

____________________________________________________________________

Now, using the matrix above code your secret message into pairs of numbers.