Millimeter-accurate Augmented Reality enabled by
Carrier-Phase Differential GPS
Ken Pesyna, Daniel Shepard, Todd Humphreys
ION GNSS 2012 Conference, Nashville, TN | September 21, 2012
 Augmented Reality (AR) Definition
 Motivation for millimeter-accurate AR
 Our AR Hardware
 Our AR Software
 Demonstration
 Conclusions
2 of 26
What is Augmented Reality
 Augmenting a live view of the world with
computer-generated sensory input to enhance
one’s current perception of reality[1]
[1] Graham, M., Zook, M., and Boulton, A. "Augmented reality in urban places: contested content and the duplicity of
code." Transactions of the Institute of British Geographers.
3 of 26
Augmented Reality Today
4 of 26
Augmented Reality Today
 Landmark identification
 Stargazing
5 of 26
Augmented Reality: How it works
Visual Overlays
Textual Information
Other Sensory Inputs
6 of 26
Essential Components for AR
 Incorporates seven primary pieces of hardware
 GPS receiver
 Accelerometer
 Gyroscope
 Magnetometer
 Camera
 Computer
 Screen
7 of 26
Carrier-phase GPS (CDGPS) Positioning
Very Accurate!
8 of 26
Ultra-Precise Augmented Reality
 Incorporate CDGPS-generated hyper-precise
location information
Incorporate in IMU (gyroscopic, magnetometer,
accelerometer) orientation measurements
--Ultra-precise Position
--Ultra-precise Velocity
--Orientation information
9 of 26
Prior Art: Subsurface data visualization
 RTK capable receiver: Leica Geosystems
 Gyroscope + Magnetometer
 Did not couple GPS and INS together
[2] Roberts, G.W. and Evans, A. “The use of augmented reality, GPS and INS for subsurface data visualization.” FIG
XXII International Congress, 2002, University of Nottingham
10 of 26
Great Potential: Google Glass
11 of 26
Ultra-precise Google Glass
12 of 26
Ultra-precise Google Glass
13 of 26
Ultra-precise Google Glass
14 of 26
Our Augmented Reality System
15 of 26
Ultra-Precise Augmented Reality System
FOTON Software-Defined GPS Receiver
 Runs GRID software receiver
developed by UT and Cornell
 Dual frequency (L1 C/A and L2C)
 Data bit wipe-off capable
 5 Hz output of observables (carrier
phase and psuedorange)
 Enables rapid testing
16 of 26
Ultra-Precise Augmented Reality System (cont.)
IMU, Medium-grade, from Xsens
 accelerometer, gyro, and
magnetometer measurements
 100 Hz output rate
Single Board Computer (SBC)
 Handles communications with
Software-receiver over Ethernet
17 of 26
Ultra-Precise Augmented Reality System (cont.)
Antcom Active L1/L2 GPS Antenna
 HD webcam from FaceVision
 22 fps 720P Video
Rechargeable Lithium Ion Battery
 Provides power for over 3 hours
18 of 26
Ultra-Precise Augmented Reality System
 System hosted by a
tablet computer
 Retrieves data from the
reference station
 Records and processes
the GPS+IMU data
(currently postprocessing)
 Overlays the webcam
video with virtual objects
19 of 26
Extended Kalman Filter
 Goal: Provide sub-centimeter level positioning and degree-level
 IMU provides orientation to degree-level accuracy
 Tightly coupled IMU and CDGPS EKF will provide the desired positioning
 EKF state, :
  ≡ ECEF position of the IMU
  ≡ ECEF velocity of the IMU
  ≡ accelerometer biases
  ≡ 


  ≡ integer ambiguities from double-differenced
carrier phase
20 of 26
Block Diagram of our EKF
Visual Overlay Stage
K =======>
INS Stage
21 of 26
Augmented Reality Overlay
1. Take the precise user position and orientation from the EKF
2. Use MATLAB 3d toolbox to obtain correct perspective of
3. Object overlayed on camera feed
22 of 26
System Demonstration
East-North Position
Orientation vs Time
23 of 26
System Demonstration
24 of 26
 Demonstrated ultra-precise augmented reality is
possible by coupling together CDGPS and IMU
Discussed applications, specifically those tailored
toward the mobile arena
Future challenges include overcoming
interference and power constraints that will be
present in small-mobile systems
25 of 26
26 of 26
Augmented Reality Overlay
1. Take the precise
position and
orientation from the
2. Create of virtual
overlay (Using
MATLAB 3d toolbox)
3. Place overlay onto the
camera feed (using
MATLAB 3d toolbox)
28 of 26
GRID Software Receiver
29 of 26
30 of 26

similar documents