Aerospace and Electronic Systems Magazine September 2016 - 20

Onboard Visual Sense and Avoid System for Small Aircraft
Also, the algorithm is capable of detecting aircraft only in the sky
region and only videos with dark targets were involved in the tests.
Finally, the development of a monocular camera-based system
is targeted at applying similar image processing techniques as in
[7], but with stringent power and hardware needs. The next step
should be the overview of possible collision risk estimation methods based-on the processed image data.
As a result of image processing, in general the center of mass
and size of the identified object (intruder aircraft) is calculated
for an image. In the case of a monocular camera, either this data,
or optical-flow data are available to decide collision risk. In this
work, the focus is on the object centroid and size-based methods.
The focus of this work excludes the possibility to apply additional
sensory systems such as LADAR, or Millimeter Wave radar, and
others, as published in [8]. The application of optic-flow is also
excluded.
UtopiaCompression corporation [9] developed a Maneuverless
Monocular Passive Ranging System which might be suitable for
the goals of this work, but the system details are unknown and
the authors decided to apply available algorithms which are completely known instead.
Possible alternatives include other active and passive methods.
Active methods include own craft maneuvering to make intruder
distance and flying direction observable. These parameters are unobservable without own craft motion as pointed out in [10].
Extended Kalman Filters can be effectively applied to estimate
intruder position and velocity as shown in [11]-[13]. From this
data, collision cone approach [12], or the calculation of probability
of Near Mid Air Collision (NMAC) can be done. Both methods
lead to a decision about possible collision.
Considering the passive methods without own craft additional
motion, the following examples can be observed in the literature.
Using only centroid and size information [14] publishes an
HMM filter-based method, which estimates intruder size and
range, but assumes that the real size of the intruder is known. This
is not the case if the own craft meets an unknown intruder and so
the application of this method is not possible.
A possibility to detect and estimate intruder position in an image based on its a priori known color is published in [14] and an
avoidance strategy is proposed based on this method. However, the
color of the intruder is usually also unknown.
The authors published a method based on intruder closest point
of approach (CPA), which can be formalized based on image data
in [15]. If the intruder is on a collision course, this CPA value goes
to zero; otherwise, it increases by approaching the intruder. This
way a threshold can be selected below which there is risk of collision. Finally, this last method is further developed by the authors.
After deciding about the possibility of collision, an avoidance
maneuver should be executed if required. The maneuver should
guarantee the safe avoidance of the intruder, which means provision of a minimum distance between own craft and intruder. However, limited field-of-view (FOV) of the vision system can cause
a problem because own craft can lose the intruder from the FOV
(see [11], for example). This means that the intruder cannot be further tracked and there is a chance to turn into the intruder instead
of avoiding it. However, during the avoidance maneuver the own
20

craft should lose the intruder from the FOV at a given time because
it should fly "behind" it and go back to its original track (see [16],
for example).
The literature applies several different avoidance strategies
from which some examples are shortly summarized here.
Salazar et al. applied a differential geometry algorithm to guarantee avoidance with a given distance, but this requires the knowledge of the intruder position and velocity [8].
Sahawneh et al. applied a chain-based strategy which designs
the path of own craft between start and end waypoints and the intruder as a chain hung between the two waypoints and repelled by
the intruder [17]. However, this also assumes knowledge of the
intruder position and flight direction.
Grocholsky et al. published a Dubins path based own craft
avoidance trajectory generation if intruder position and velocity
are known, and a force field-based strategy when only monocular image data (intruder bearing) is available [18]. It generates the
force field in the image frame with repelling force from the intruder and the ground (to avoid ground crashes) and an attractive
force from the image center, because this points the original flight
direction of own craft.
Watanabe et al. applied minimum effort guidance with an
Euler-Lagrange equation-based optimal solution also assuming
known intruder position and velocity [12].
The authors applied four predefined maneuvers (up/down or
left/right motion/turn) and evaluate the probability of NMAC for
all of them. The lowest probability gives the required maneuver
direction [13]. This algorithm also considers estimated intruder
position and velocity.
Most of the algorithm assumes known intruder position and
flight direction, which is not true if a simple CPA-based decision is
made about a collision.
Degen et al. applied a CPA-based decision and so, the avoidance is done based on the measured bearing of the intruder [19].
This concept is a good starting point, however, it has its own drawbacks as will be pointed out later. Proposing an improved and feasible avoidance solution is a goal of the author's developments.
Summarizing all of the introduced topics, the goal of the authors is threefold. First, to propose and implement a hardware and
software solution for intruder aircraft detection based-on a monocular camera system considering power, weight, and size constraint, while providing real time applicability. Second, to propose
a simple and effective collision risk estimation algorithm possibly
based on simple image-based data, such as intruder image size and
position. Covering all possible intruder categories from small UAV
to large transport aircraft without any rescale of the algorithm is
another goal. Third, to propose an effective avoidance strategy
considering only non-cooperative intruders without an SAA system. Meanwhile the development focuses on UAVs; the detection
and collision risk estimation part of the system can also be used
on small manned aircraft such as ultra-lights. The final goal is to
present preliminary flight test results achieved by the developed
system. The next section summarizes the proposed intruder detection camera system hardware and software solutions. Then, the
proposed image processing-based collision risk estimation strategy
is described together with the characterization of possible intruder

IEEE A&E SYSTEMS MAGAZINE

SEPTEMBER 2016

Table of Contents for the Digital Edition of Aerospace and Electronic Systems Magazine September 2016