Virtual Reality World Teaches Deaf Children Math Skills

Armen Hareyan's picture

Deaf Children and Math Skills

Purdue University is using technology employed in the films "King Kong" and "Lord of the Rings" to create colorful characters in a virtual reality world who teach deaf children math.

Computer graphics technology students are working in Purdue's Envision Center for Data Perceptualization with high-tech cameras and "cybergloves" that can translate body and hand motions into digital images. The resulting cartoon rabbit, robot and pig use sign language in fun, interactive environments projected on the walls and floor of a "cave" of screens that surrounds the deaf students.

They wear lightweight stereoscopic glasses so the virtual reality images appear three-dimensional. A device monitors the student's head position so the environment is consistently redrawn to match the user's perspective. A wrist tracker and telemetric "pinch gloves" monitor the student's hand and finger movements allowing interaction with the virtual environment and prompting responses from the characters.

The virtual reality program is designed to provide early elementary school age students with disabilities with a number of active, individualized learning conditions:

  • The ability to control their environment.

  • The ability to engage in learning activities at their own pace.

  • The ability to repeat activities as needed.

  • The ability to see or feel items or processes in concrete terms.

  • The ability to practice daily living tasks in a safe and barrier-free environment.

  • Motivation to succeed.


For example, in a virtual candy store environment the student communicates to the storekeeper in sign language, some of which is specific to mathematics. Pinch gloves allow students to count candies and to add and subtract by putting candy on or off the counter. The task can be repeated over and over at the student's own pace while providing consistent and understandable feedback.

"Learning is the development of one experience into a new experience," said Ronnie Wilbur, professor and chair of linguistics in the Department of Speech, Language and Hearing Sciences who has served as a consultant to the College of Technology during the project. "Immersive learning environments such as this are more effective than traditional computer software."

The project's supervising professor, Nicoletta Adamo-Villani, said virtual reality helps break down some of the barriers deaf children experience.

"Environments are more stimulating when students are able to interact with the subject and travel through the scenes," said Adamo-Villani, assistant professor of computer graphics. "Hands-on experiences equate to a better understanding of mathematical concepts in real-world situations."

Adamo-Villani said research shows that humans process visual information 60,000 times faster than textual information and that an eight-week virtual reality program can improve student math skill scores by 16 percent. She said that learning enthusiasm remains even after the novelty of virtual reality fades.

"Our mission is to be a worldwide leader in virtual learning environment development," said Laura Arns, the associate director of the Envision Center who is helping develop this virtual reality application. "We want to advance industry-leading concepts, software and services while fostering an environment of empowerment, creativity and commitment."

The program is being designed to overcome the barriers deaf children experience in learning math skills. Those barriers include:

  • Significant delay in reading comprehension.

  • Parents' inability to convey math concepts in sign language.

  • Difficulty taking advantage of supplemental learning opportunities, such as television shows and dinner-table conversation.

Adamo-Villani hopes by reducing the impact of these barriers, the program can help increase the number of deaf students who go on to study science, technology, engineering and mathematics