Aerospace and Electronic Systems Magazine October 2017 - 54

Gyro-Aided Visual Tracking

Figure 2.

Tracking results of the iEMD tracker. The first row shows results using the shopping center sequence (1,800-2,100 frames from the WalkByShop1cor
data set of the Context Aware Vision Using Image-Based Active Recognition (CAVIAR) Project, http://homepages.inf.ed.ac.uk/rbf/CAVIAR/); the bottom row shows the results of the gyro-aided iEMD tracker (100 frames from the data set provided by Carnegie Mellon University [17]), the magenta
boxes indicate the results of the iEMD tracker without the gyroscope information, and the cyan boxes indicate the results of the gyro-aided iEMD tracker.

NT + NC - 1 constraints, which could be considered as redundant and
discarded [19]. On the basis of the optimal solution to the linear programming problem, the flow vector is separated into basic variables
T T
] ∈  NT NC, and the ground disand non-basic variables as f = [f BT , f NB
N N
T
T
T
tance vector d and H will be transformed as d = [d B , d NB ] ∈  T C
( N T + N C ) × NT N C
T
NT + NC −1
and H = [H B , H NB ] ∈ 
, where dB ∈ 
, and HB
∈  ( NT + NC )×( NT + NC −1). To derive the EMD as a function of the weights
of the candidate model, the matrix transformation is performed.
Considering the rank of the matrices, the last row of the constraint
matrices (1) is deleted, which is considered as the redundant constraint, and then the matrices HB, H, and w(y) are formulated as
( N + N −1)× NT N C
∗
∗
T
and
H ∗B ∈  ( NT + NC −1)×( NT + NC −1), H* = [H B , H NB ] ∈  T C
NT + NC −1
T
T
T
w*(y) = [qˆ (y ) NC , pˆ NT −1 ] ∈ 
.
The value of the EMD D* is represented as a function of
N + N −1
T
T
T
weights w*(y) = [qˆ (y ) NC , pˆ NT −1 ] ∈  T C , where the weights
are a function of the displacement y. The gradient method is then
used to find the displacement y of the target candidate. When
a new frame is obtained, the candidate template is obtained assuming the displacement y0 = [0, 0]T. Then the value of EMD
D* is calculated on the basis of the linear programming, and the
new displacement y1 is obtained by moving the template 1 pixel
along the gradient direction. The iEMD algorithm alternates between finding the smallest EMD between template target and the
template candidate on the basis of the current position yk by the
transportation simplex method and finding the best position yk+1,
leading to the smallest EMD by gradient method. The template
corresponding to the smallest EMD is the location of the target
in the new frame.
The general idea of the gyro-aided iEMD tracking algorithm
is to combine the image frames from the camera with the angular
rate generated by the gyroscope for visual tracking. Synchronization of the camera and the gyroscope in time is required. The
spatial relationship between the camera and the gyroscope must
also be pre-calibrated. Then, the angular rate generated by the
gyroscope is applied to compensate for the ego motion of the
camera. After the compensation of the ego motion of the camera,
the iEMD tracker is applied for tracking. The tracking results are
shown in Figure 2.
54

The contributions of this work are summarized as follows:
C

An efficient iEMD algorithm is developed. By iteratively
moving the template candidate by a small displacement along
the gradient direction of the EMD and using the EMD to
estimate the match between the template target and the new
template candidate, the moving object is tracked successfully.
This algorithm separates the process of representing the EMD
as the function of the weights of the template candidate into
two phases. First, the EMD problem is solved by transportation simplex method, which is more efficient for the number
of operations compared with the standard simplex method
(compare [20]) used by the DEMD algorithm. Second, on the
basis of the special structure of the transportation problem, the
redundant constraint is automatically selected and discarded.
Thus, the process of reforming the EMD as the function of the
weights of the target candidate is more efficient.
Gyro measurements are used to compensate for the pan, tilt,
and roll of the camera. Then the iEMD visual tracking algorithm is used to track the target after compensating for the
movement of the camera. By this method, the convergence
of the algorithm is improved, thus, providing a more robust
tracker, which is more capable of real-world tracking tasks.

REFERENCES
[1]

[2]

[3]

IEEE A&E SYSTEMS MAGAZINE

Yang, J., Dani, A. P., Chung, S.-J., and Hutchinson, S. Vision-based
localization and robot-centric mapping in riverine environments.
Journal of Field Robotics, Vol. 34, 3 (2017), 429-450.
Dani, A., McCourt, M., Curtis, J. W., and Mehta, S. Information fusion
in human-robot collaboration using neural network representation. In
Proceedings of the IEEE International Conference on Systems, Man
and Cybernetics, San Diego, CA, 2014, 2114-2120.
Ravichandar, H., and Dani, A. P. Intention inference for human-robot
collaboration in assistive robotics. In Human Modeling for Bio-Inspired Robotics: Mechanical Engineering in Assistive Technologies, J.
Ueda and Y. Kurita, Eds. London, United Kingdom: Academic Press.
2017, pp. 217-249.

OCTOBER 2017

http://homepages.inf.ed.ac.uk/rbf/CAVIAR/

Table of Contents for the Digital Edition of Aerospace and Electronic Systems Magazine October 2017