Structured_Hough_Voting

Report
GM-Carnegie Mellon Autonomous Driving CRL
Structured Hough Voting for Visionbased Highway Border Detection
Zhiding Yu
Carnegie Mellon University
1
GM-Carnegie Mellon Autonomous Driving CRL
Autonomous Driving: Not If, But When
2
GM-Carnegie Mellon Autonomous Driving CRL
GM-CMU Collaborative Research
GM-Carnegie Mellon Autonomous Driving CRL
Sensors Setup on SRX Platform
Images from: Junqing Wei et al., “Towards a Viable Autonomous Driving
Research Platform,” IEEE Intelligent Vehicles Symposium (IV), 2013
GM-Carnegie Mellon Autonomous Driving CRL
Information
Sensors: Price vs Information
Camera
Lidar
Radar
Price
GM-Carnegie Mellon Autonomous Driving CRL
Computer Vision Applications
 Object detection (pedestrian, vehicle, bicycle…)
 Road parsing (lane/border detection, road segmentation,
vanishing point estimation…)
 Localization and tracking
 Driver status monitoring
 Many other applications……
GM-Carnegie Mellon Autonomous Driving CRL
Motivation, Description and Goal
Goal
–
Development for future driving assistance system and autonomous driving system
–
Robust detection within 0.5 to 6 meters detection range. Achieve near 100%
accuracy in daytime and over 90% in nighttime on the right most lane
–
Handling various scenarios including highway entrance and exit
–
Extend to the joint system with front view
7
GM-Carnegie Mellon Autonomous Driving CRL
High-Level Idea: Learning based Method
Concrete Barrier
Guard Rail
Soft Shoulder
Concrete
Barrier
Guard
Rail
Soft
Shoulder
Lane
Marking
Densely Fired
scanning windows
Returned Voting
Points
Structured
Hough Voting
Border / lane
marking hypotheses
GM-Carnegie Mellon Autonomous Driving CRL
Dataset Collection
Overall 1592 training images:
1. Concrete Barrier (839 images)
2. Guard Rail (300 images)
3. Soft Shoulder (453 images)
Overall 2638 testing images:
GM-Carnegie Mellon Autonomous Driving CRL
Training Patch Alignment
Concrete
Positive
Samples:
Negative
Samples:
Natural
Steel
Lane Marker
GM-Carnegie Mellon Autonomous Driving CRL
Feature Extraction
HOG
Filter Bank
Concatenated Filter
Bank Feature
Patches that are discriminative to HOG
Patches that are discriminative to filter banks
Concatenated
HOG Feature
GM-Carnegie Mellon Autonomous Driving CRL
Classification & Detection
 Extract features from all training patches (based on previous page)
 Perform Fisher discriminant analysis
 Train an RBF kernel SVM
 Scanning window detection (Deliberately having a lot of positive firing)
Concrete
Barrier
Guard
Rail
Soft
Shoulder
Lane
Marking
GM-Carnegie Mellon Autonomous Driving CRL
Hough Voting
GM-Carnegie Mellon Autonomous Driving CRL
Structured Hough Voting: Intuitions
 Basic philosophy: A model that assumes voting results are correlated rather
than independent
 Inter-frame structural info on hypotheses (Temporal smoothness)
 Intra-frame structural info (Geometric relationship)
 Multiple candidate hypotheses generation (Proposals with diversity)
1. Constrained Hough Voting on detected voting points (Detection + Tracking)
2. Arbitrary Hough Voting on detected voting points (Detection)
3. Constrained Hough Voting on image gradients (Pure Tracking)
GM-Carnegie Mellon Autonomous Driving CRL
Purpose of Candidate 1
 Deals most of the frames where hypotheses from consecutive frames have
strong correlation.
GM-Carnegie Mellon Autonomous Driving CRL
Purpose of Candidate 2
 Automatically corrects result through searching for “much better” voting
configurations (This is the power of detection, avoids error from tracking)
GM-Carnegie Mellon Autonomous Driving CRL
Purpose of Candidate 3
 In the worst case where Type 1 voters fail, perform tracking by gradients
from previous pose configuration.
GM-Carnegie Mellon Autonomous Driving CRL
Modeling under CRF: Background
 A Conditional Random Field (CRF) discriminatively defines the joint posterior
probability as the product of a set of potentials
Unary Potential Pairwise Potential
H1
H2
X1
X2
…
HN
XN
 The potentials are functions with hypotheses Hi being the variables. They are modeled
in such a way that a larger potential value generally indicates a better hypothesis
configuration.
 CRF inference seeks to find the joint hypothesis configuration H that maximizes
GM-Carnegie Mellon Autonomous Driving CRL
Modeling under CRF: Intuition
 What are the hypothesis Hi?
 E.g.: image pixel labels (FG/BG, Object Class, etc.), if it is a segmentation problem.
 In our problem, Hi is the Hough Voting hypothesis: Hi = (r, θ).
 X is the observation of voting point coordinates and their weights.
 The unary potential
weights: exp(v(Hi)).
corresponds to the exponential of Hough voting
 The pairwise potential
(tracking) constraint.
corresponds to the inter-frame smoothness
H1
H2
X1
X2
…
HN
XN
GM-Carnegie Mellon Autonomous Driving CRL
No Structural Information
Hbd,1
X1
Hbd,2
X2
Hln,1
X1
…
XN
Hln,2
X2
Hbd,N
…
Hln,N
XN
Simplest Case: frame-wise independent Hough voting
GM-Carnegie Mellon Autonomous Driving CRL
Adding Inter-frame Structural Info.
Hbd,1
X1
Hbd,2
X2
Hln,1
X1
…
XN
Hln,2
X2
Hbd,N
…
Hln,N
XN
Adding temporal smoothness: Hough voting constrained
by neighboring frames
GM-Carnegie Mellon Autonomous Driving CRL
Adding Intra-frame Structural Info.
Hbd,1
X1
Hbd,2
X2
Hln,1
X1
…
XN
Hln,2
X2
Hbd,N
…
Hln,N
XN
Adding Geometric Constraint: Hough voting constrained by
both neighboring frames and intra-frame hypotheses
GM-Carnegie Mellon Autonomous Driving CRL
The Structured Hough Voting Model
Candidate Hypotheses Generation Unit
•••
Coupled
Structure
Potential
•••
Mode Selection Potential
GM-Carnegie Mellon Autonomous Driving CRL
The Structured Hough Voting Model
GM-Carnegie Mellon Autonomous Driving CRL
Candidate Hypotheses Generation Unit
GM-Carnegie Mellon Autonomous Driving CRL
Mode Selection Potential
 Use decision tree to guide the mode selection.
 The mode selection basically forces the output to be one of the candidate hypotheses,
but allows discrepancy with the decision tree prediction with a penalty.
GM-Carnegie Mellon Autonomous Driving CRL
Coupled Structure Potential
 The coupled structure potential captures two most important relations between a border
hypothesis and a lane hypothesis
 Parallelism
 Distance
GM-Carnegie Mellon Autonomous Driving CRL
Inference
 Conducting a whole inference each time given a new frame is computationally infeasible.
 Relaxation: Initialize with the inferred state variable configuration of the previous t-1
frames and infer the current state variables, updating in an incremental way.
 Inference procedure at t = 1:
1. Perform Hough voting for both border and lane marking
2. Perturbate hypotheses if geometric relationship violated (optional)
 Inference procedure at t > 1:
1. Generate the 3 candidate hypotheses for both border and lane marking
2. Use decision tree to help selecting the best candidate
3. Perturbate candidate hypotheses if geometric relationship violated (optional)
4. Re-select the best candidate
GM-Carnegie Mellon Autonomous Driving CRL
Experiments: Adding Coupled Structure
GM-Carnegie Mellon Autonomous Driving CRL
Experiments: Qualitative Results
Ground Truth and Baseline methods:
1.
2.
3.
4.
5.
6.
Ground Truth
Independent Hough voting in each frame using the fired detector voting points
Hough voting using the triggered detector voting points constrained by previous frame
Adding gradient tracking to Baseline 2.
Kalman filter.
Proposed Method
GM-Carnegie Mellon Autonomous Driving CRL
Experiments: Quantitative Results
GM-Carnegie Mellon Autonomous Driving CRL
Highway Entrance Detection and Lane
State Tracking
GM-Carnegie Mellon Autonomous Driving CRL
Summary
 Proposed the Structured Hough Voting Model
 The proposed model can be theoretically formulated under a CRF
 Fast real-time feature extraction and online inference
 Achieves very robust and good performance under challenging scenarios
and low quality inputs from production camera
GM-Carnegie Mellon Autonomous Driving CRL
Thank You!
Q&A

similar documents