Touch is another sense that can be augmented with learning applications. A paper presented at the 2004 International Symposium on Mixed and Augmented Reality (ISMAR) discussed the development of augmented Standardized Patients (SPs) in medical training.
SPs are traditionally real people who portray patients in order for a medical student to practice such skills as getting a medical history, performing an examination, and so forth. However, it’s often difficult to find people who can realistically simulate various symptoms.
Completely virtual patients accessed via a computer raises a different set of concerns, the papers’ authors state. The medical students must learn not just how to diagnose patients but also how to interact and communicate with them. Human-to-computer interaction doesn’t provide the necessary skills transfer.
Fortunately, there’s a third option, thanks to new technology being tested and prototyped by the Eastern Virginia Medical School and the Virginia Modeling, Analysis, and Simulation Center at Old Dominion University. (Some funding also came from the Naval Health Research Center.)
Researchers rigged a system that tracks the movement of a student’s stethoscope on an augmented SP. When the stethoscope reaches the proper location, it triggers a sound file played over headphones the student wears. The system was developed using mannequins because of the long periods of immobility needed during testing, but it can be used just as easily on real people.
Other training situations that currently use actors, such as rescue worker or other emergency personnel training, could provide additional uses for haptic augmented reality.
Online learning and collaboration
Augmented reality can even be used with online learning. At the United Kingdom’s University Sussex, researcher Fotis Liarokapis developed “an interactive e-learning AR environment” in which users can view and interact with three-dimensional virtual objects aided by online instructors.
Part of an ongoing research project, the Multimedia Augmented Reality Interface for E-Learning (MARIE) uses many of the same technologies as visual augmented reality—the head-mounted display, camera, and computer—to present 3D multimedia information to the learner.
But the objects are part of synchronous e-learning: The instructor guides learners to view various them in sequence as part of his or her learning strategy, much as a traditional instructor might tell learners to turn to a certain illustration in a manual or textbook. However, with MARIE, the learner can interact with the object, rotating and manipulating it. Future development may enable the addition of other forms of augmentation, such as touch and smell, and he is continuing to develop the project in order to integrate it with e-learning platform WebCT.
Similarly, Kelly Dempski talks about a project he’s working on at Accenture that combines videoconferencing with augmented reality for augmented collaboration. Two people in different locations have cameras and display screens set up so they seem to be talking to each other from across a table. Then, the two of them can both interact with shared virtual objects in 3D for collaboration or learning.
Liarokapis says augmented reality tops virtual reality for learning and training “in terms of cost, realism, and human factors. Virtual reality completely replaces the real environment and this makes it much more difficult for users to adapt.” In 5 or 10 years, he says, we will be using AR for “a number of everyday applications,” including training and learning.
Personal Digital Assistants (PDAs)
In most of the previous examples, the augmented reality technology was under development and not in commercial use. For now, the technology is still too complicated and expensive for most companies. That will change as computer hardware gets smaller and less expensive. In the meantime, there is a workaround that has arisen over the past few years: augmented reality via personal digital assistant (PDA). The experience is obviously not as immersive, but it is a viable strategy being used with some success in learning, especially when combined with a game-based approach.
For example, the Massachusetts Institute of Technology’s Education Arcade (a group that promotes the use of computer and video games in education) joined with the Boston Museum of Science last February to create Mystery at the Museum, a “Hi-Tech Who Done It” they tested with pairs of middle school children and adults.
The premise: thieves have stolen a museum artifact and replaced it with a fake. The detective teams, three adult-child pairs, each received a handheld device, which they used in various galleries (along with the museum’s Wi-Fi network) to analyze objects with virtual instruments, interview characters, exchange information with other teams, and more. The children were not only highly engaged but also learned quite a bit.
College students in MIT’s Environmental Education courses play Environmental Detectives, an AR game in which they investigate a toxic spill and learn about observation, hypothesis testing, data gathering and analysis in a problem-based learning simulation untethered from the desktop.
Can these experiments translate to adult learning? Absolutely. Game-based learning is already being used with adult learners, and combining that with whole-body experiences as augmented reality does can not only make the learning more engaging but also more “sticky.”
The ADL (Advanced Distributed Learning) Academic Co-Lab is working to develop new augmented reality applications through the AR gaming platform developed by Education Arcade. Common elements include “engaging backstory, differentiated character roles, reactive third parties, guided debriefing, synthetic activities, and embedded recall/replay to promote both engagement and learning.”
The future of augmented reality
Where is augmented reality and learning going? For the answer to that question, we can look to the U.S. military, which has a head start on the technology as well as the educational pedagogy.
By 2014, more than 30 percent of mobile workers will be using augmented reality.
Source: Gartner, via Information in Place
The Army is developing the technology not only for performance-support information and graphics to be overlaid on soldiers’ vision—navigation lines or equipment repair instructions, for example—but also for thoroughly virtual characters, buildings, explosions, and so forth. Eventually, soldiers will be able to interact with these virtual items in the real world for sophisticated, realistic, whole-body training.
That kind of technology can cost millions of dollars and is still years off—Sonny Kirkley says at least five years and maybe as many as 10, depending on the rate at which the hardware matures. But the Army is getting ready now. As part of their Future Force Warrior program, they’ve contracted with Information in Place not just to work on the technology side but also to develop the instructional methodologies that will be needed in this new brand of training.
The Army said to Kirkley, “Great that you can make [the technology] work, but do people really learn?” So his company has been working to develop problem-based learning—they call it Problem-Based Embedded Training—and learner supports that will be used not only with the augmented reality training but also with the mix that is developing of augmented reality, virtual reality (simulations), and real-life training. Kirkley calls this “mixed reality” training.
In addition, Information in Place is developing an authoring tool that helps training designers think through the process of developing this new type of blending. “Instructional designers have enough trouble developing face-to-face and Web-based [training],” Kirkely says. “So we’ve been looking at an authoring support tool that focused on the instructional design process [for mixed reality training] from needs analysis to final product.”
Kirkley and his colleagues at Information in Place stress that the focus needs to be on what is effective, not the technology. Just as in selecting Web-based or classroom training, training designers need to look first at the training objectives and then decide which medium will best help learners meet those objectives. And that’s not always the expensive solution with the highest fidelity. In short, augmented reality training is no different than other types in that it needs to be focused on solid instructional design and well thought-out cost-benefit analysis.
Learning Circuits Articles
RSS: A Learning Technology
Content Copyright, the Commons, and the C Generation
Additional Information on Augmented Reality
Look, Listen, Walk, an article from MIT’s Technology Review
The Top Ten Technologies: #3 Augmented Reality
A PDF about Accenture Technology Labs’ work on virtual and augmented reality
More Augmented Reality Games
Savannah an AR game for children’s education developed in the United Kingdom
NetAttack, an indoor/outdoor AR game
PacManhattan, an AR twist on a classic game—and a Wired News article about it
The Invisible Train, a handheld multiplayer AR game
Published: December 2004