UNCLASSIFIED ADVANCED DISPLAYS FOR DISMOUNTED WARFIGHTERS 21-22 SEPTEMBER 2010 UNCLASSIFIED Outline • • • • Introduction Types of Advanced Displays Current Displays and Lessons Learned Occluded Helmet Mounted Displays • Head Mounted Displays • Fused Night Vision Goggles • Conformational Displays • Tactile Displays • Flex Displays • See-through Helmet Mounted Displays • Augmented reality displays and Enhanced Cognition • Conclusions Introduction The purposes for using displays are: • to present or hold up to view • to provide information or graphics on a screen • to provide a representation of information Now just a minute, earlier you said x equals 4! x+3=5 X=2 Introduction The purpose of advanced displays is to immerse warfighters in the operation so they can experience and convey critical information from real-time data feeds in an intuitive, recognition-based manner. Introduction Multifunction Displays (MFDs) Definition- a display surface which, through hardware or software, is capable of displaying information from multiple sources, in several different reference frames. It may display different groups of data one at a time or in a combined fashion. Multifunction, not multiple displays Types of Advanced Displays Headmounted Occluded See-through Body-worn or Carried Formed Configurable HMDs GMDs Augmented Reality Displays PDAs Tactile Displays Fused NVGs Notebooks Flexible Displays Current Displays and Lessons Learned Current Systems FBCB2 Color Helmet Mounted Display The Soldier is the most difficult “combat platform” to interface with! One Size never fits all Soldiers have different opinions Soldier Acceptability is critical AN/PVS-14 NVG Current Displays and Lessons Learned Land Warrior 2005 Land Warrior “Manchu” 2007 Land Warrior “Strike” 2008/2009 Nett Warrior 2012/2017 Helmet Mounted Display Headset & Mic GPS Antenna Battery Soldier Control Unit Navigation Module Soldier Radio/ Antenna Computer • • • Command & Control - Soldier: Voice & Data - Leader: Voice & Data Key Attributes - Weapon: Subsystem - Radio: EPLRS - Interoperability: w/ FBCB2 Size & Weight - 11 components - 16 pounds • • • The Future – Nett Warrior All BCTs • Command & Control - Soldier: No capability - Leader: Voice & Data Key Attributes - Weapon: Dropped - Radio: EPLRS - Interoperability: w/ limited external assets Size & Weight - 9 components - 11 pounds • • Command & Control - Soldier: No capability - Leader: Voice & Data • Key Attributes - Weapon: Dropped - Radio: EPLRS - Interoperability: - UGV interoperability for IED Defeat - Air/Ground Integration thru SADL • Size & Weight - 7 components - 9 pounds • Evolution From Land Warrior To Ground Soldier System (Nett Warrior): Current Path to Modernization Introduction Command & Control - Soldier: Interoperable with Soldier Voice & PLI - Leader: Digital Voice & Data Key Attributes - Weapon: Dropped - Radio: EPLRS/JTRS (P3I) and Open Architecture - Interoperability: increased w/ external sensors (JBC-P and Fire Control Systems) Size & Weight - Minimum essential - 10 pounds (Objective) Current Displays and Lessons Learned Icons Cognitive Walkthroughs with Target Audience Soldiers Several Iterations of User Juries for the Ground Soldier System • Ease of training • Intuitiveness • Speed • Errors Introduction Limited Hardware Evaluations Occluded Head Mounted Displays • • • • • • Currently planned for many systems (NETT Warrior, Common Controller, etc.) Lightweight Relatively high resolution and easier to read Require less power than many portable devices Less glare than many portable devices Hands free Occluded Head Mounted Displays POTENTIAL PROBLEMS • • • • • • • • • • Both real world and HMD imagery must be within the user’s depth of field, dark focus and dark vergence Restricted field of view can impair performance and adversely affect the ability to see the “whole picture” and reduce SA Binocular rivalry Competition for the attention of the wearer/attentional tunneling which adversely affects dual task performance Eyestrain and blurry vision As HMD wearers move their heads, displayed objects in front continue to be in front and this is unnatural Motion sickness, often explained as being caused by sensory conflict. Interference with night vision devices and weapon sights Fogging due to weather Potential for placing the musculoskeletal system of the head and neck under increased levels of stress Occluded Head Mounted Displays Helmet Mounted Displays • Scaling Robotic Displays: Displays and Techniques for Dismounted Movement with Robots – Soldiers performed significantly worse with the GMD than they did with the HHD on course completion times, driving errors, and the number of times they drove off course. – Soldiers also preferred the HHD to the GMD and rated the workload with the HHD lower. Occluded Head Mounted Displays Helmet Mounted Displays Dismounted Land Navigation Study PTN Tactile System: GPS sensor, electronic compass, processing unit, battery pack, eight tactors placed equidistant, on a belt around the torso, worn over T-shirt LAND WARRIOR HMD GPS SYSTEM HANDHELD GPS PLGR 1. 24 Soldiers (N = 21) 2. 3 routes, each with 2 waypoints & endpoint, each approx 1800m 3. 3 systems: Tactile, PLGR, LW HMD 4. Order of routes and systems counterbalanced Occluded Head Mounted Displays Helmet Mounted Displays Dismounted Land Navigation Outcomes System % Reached PTN 100 PLGR 100 LW 100 17 16.5 16 15.5 15 14.5 14 13.5 13 12.5 10 8 PTN LW 4 PLGR 2 0 Targets LW PLGR Nav Speed Target Detection 6 PTN Occluded Head Mounted Displays Helmet Mounted Displays Dismounted Land Navigation Soldier Feedback Very good 7 Good 6 Somewhat good 5 PTN Neutral 4 LWS PLGR Somewhat bad 3 Bad 2 Very bad 1 Easy to learn Easy to use Easy to tell where located Ease to stay on Route Accuracy of guidance Occluded Head Mounted Displays Fused Night Vision Goggles Urban Enhanced Night Vision Goggle Enhanced Night Vision Goggle Occluded Head Mounted Displays Fused Night Vision Goggles Scene Interpretation OVERLAY COLOR • Contrasting color for FLIR 2 image vs. green I image improves target detection speed and range. Bonnett, Redden, & Carstens, 2003 • Contrasting color also assists in differentiating terrain characteristics. Occluded Head Mounted Displays Fused Night Vision Goggles High Thermal Target Detection Optimal Mix of Patrolling I2 and Thermal Depends on Purpose Navigation High I 2 Urban Enhanced Night Vision Goggles (UENVG) • • UENVG is a prototype system used to determine the impact of adding a Short Wave InfraRed (SWIR) capability into the existing ENVG system. SWIR – works in darker conditions than I2 – can see objects with great clarity on moonless nights because night sky radiance (nearly all in SWIR wavelengths) emits 5 to 7 times more illumination than starlight ) – is not visible to the human eye but interacts in a similar manner as visible wavelengths – has shadows and contrast and can see through glass Occluded Head Mounted Displays Flexible Displays • Near term Objectives –Compare indoor and outdoor sunshine readability – Collect Soldier opinions on the technology • Recent Efforts – Evaluations at Fort Benning (2009, 2010) Handheld and Forearm Instructors from Warrior Training Center – Evaluation at Fort Bliss (2009) • Benefits Survey of 9 prototypes – Much less glare, lighter, rugged – Much less power consumption – Longer battery life Conformational Displays Flexible Displays Near-term: Rugged, Low Power, Compact, Lightweight Far-term vision: Novel form-factors Enabled by FDC’s Unique GEN II (37x47cm) Pilot Line toolset and People 21 Partners Representing: Display Technology Manufacturing Tool Suppliers Materials Developers Defense Contractors Conformational Displays Tactile Displays Scalability of Robotic Displays: An Evaluation of Controller Display Options • Evaluated 3 options for TALON controller Display: - 6.5 inch Split Screen - 3.5 inch Toggle Screen - 3.5 inch Toggle Screen plus Tactile belt Results • 3.5 Toggle Screen associated with slower performance and higher workload than either (A) or (C). No difference between (A) and (C) • Tactile belt enabled a smaller screen while providing cues that supported performance. Conformational Displays Tactile Displays 3.5”Toggle Display 6.5” Split Screen 3.5” Toggle Display with Tactile Belt Conformational Displays Tactile Displays • Subjective Measures – Task difficulty and SA ratings were worse for the toggle display. – Some Soldiers preferred the toggle/tactile display because they felt they were able to pay more attention to the driving display while wearing it. – Others preferred the split screen display because of the wealth of information provided. – A fixed position camera and latency cause some driving difficulties. – The egocentric GPS caused problems when Soldiers glanced away from the screen. See-through Displays Augmented Reality Display • • The Last 18 Inches: How can you improve perception, squad coordination, and decision making in tactical, high stress urban operations? Squads and platoons are not well served by current information systems – – – – • Most equipment today distracts during operations – – – – • Overloaded: too much 2D, 3D, video data Confusing: irrelevant data, old, out-of-scale, inaccurate Optimized for strategic levels, not tactical block-to-block, not flowing from bottom-up Its getting worse: soldiers/vehicles as sensors, geospatial & human network models Pain in the neck: heavy, cumbersome, ill-suited for on the move Can’t make use when and where needed, intended for desktop Need for “on the go” interfaces, computing, sensing Independent module development, proliferation of multiple methods (the TV remote problem) High stress tactical decision making – – – Hard to remember important stuff Too much to pay attention to Need tools to aid attention and memory See-through Displays Augmented Reality Display Squads Need Omniscience & Telepathy within the City Block “I’ve got your back!” • Extend superiority into short-range combat inside urban and jungle environments. – Extend collaborative planning, rehearsal, and execution capabilities from company to squad level. – Enable quieter, non-linear, distributed, increased op tempo, 3D operations (e.g. take down building from 3 directions) – Enable effective, quick dynamic replanning – Reduce chaos, fratricide, avoid surprises See-through Displays Augmented Reality Display Make data useful to platoon/squad/soldier in context Geospatial registration of annotations Agent-based systems to recognize intent, state Tools to leverage models for preview, execution, debrief Relevance at the level of seconds and meters Make communication for squads and platoons more effective Integrate planning, execution, and debrief in one system Improve remote (higher command) understanding of local ops/data/history by virtual experience Make interaction with information “on the go” Overwatch feeding/filtering information selectively Interface sensors respond to natural body actions, non-screen interfaces Advances in smaller, lighter, lower power, higher resolution displays, computing, sensing enable head/body worn approaches Make overwatch automatic Plan via analogy Fuse, filter, prioritize information automatically (learn) See-through Displays Augmented Reality Display • Civilian technology offers examples to learn from and leverage… – Madden Board and first down line – graphics registered in scene images for enhanced communication – Nintendo Wii as “natural interaction” – NFL offensive play caller – in booth above field – overwatch feeds info to coach and QB – Nascar crew chief communication to driver – “Mission Impossible” controller feeding data and info to agents in mission/field – Blackberry, iPhone, iPod, HUD (Private Eye) displays and interfaces – Social information systems like GoogleEarth, Facebook, Wikipedia, wwmx.org, rich info sources with engaging interfaces • GoogleEarth is an example of local users creating overlays on global 3D base model maintained/updated centrally Conclusions • The Dismounted Warfighter is the most difficult customer for displays. • Display technology continues to advance and today’s failures may be tomorrows successes. • Displays should be chosen based on mission requirements, echelon level and on environmental considerations. • Human factors considerations and experimentation are critical for effective display design.