Aerospace and Electronic Systems Magazine September 2016 - 21

Zsedrovits et al.
vehicles. Finally, the avoidance strategy and preliminary flight test
results are presented considering two aircrafts in near mid-air collision and no threat of collision scenarios. The conclusions and possible further developments end the paper.

PROPOSED CLOSED-LOOP SENSE-AND-AVOID SYSTEM
The vision-based SAA system proposed in [20] is designed to be
small, cost-effective, and low power (see Figure 2). This is a closedloop SAA system without humans in the loop. The inputs of this
SAA system are cameras and sensors of a conventional inertial navigation system. Based on the detected intruders, the collision risk
is estimated, and if it is necessary an avoidance maneuver is run.

Figure 2.

Vision based closed-loop SAA system.

SENSOR-PROCESSOR SETUP FOR AIRCRAFT DETECTION
In order to fulfill the low power and real time criteria, the onboard
aircraft detection is run on two kinds of kilo-processor architectures. One is the GPU, another is the Field Programmable Gate Array (FPGA). The GPU implementation is for testing and parameter
tuning, and the FPGA is for the final system.
The advantage of the GPU is that the implementation time for a
new module is much faster than one on an FPGA. Our target system
is the nVidia Jetson TK1 development board which consists of the
TK1 System on a Chip with the necessary peripherals (Serial AT
Attachment (SATA), Gigabit Ethernet, High Definition Multimedia
Interface, Universal Serial Bus (USB), General-Purpose Input/Output) and can handle two cameras. This is a low power system with
a quad-core ("4-Plus-1") Advanced RISC Machine (ARM) Cortex
A15 and a Kepler GPU with 192 Compute Unified Device Architecture (CUDA) cores. The typical power consumption is around
6 W which is suitable for a small UAV. On the Jetson, the implementation can be done mostly in C/C++ with opencv::gpu and only
some special function has to be in CUDA language.
The input video stream comes from two cameras (2 × 1280 ×
960) and the LEFT/RIGHT/STRAIGHT command is sent through
RS232 to the flight control unit. For the image processing, the convolutions and morphologic calculations can be accelerated by the
Kepler GPU, and the Input/Output handling and other algorithmic
parts are done by the ARM. A solid state drive (SSD) is used as
a black box for the system. It stores the recorded images and the
telemetry data. The SSD drive connects to the system through the
SATA interface. Our TK1 implementation reaches 8 frames per
second (FPS) at 10 W (maximized operating frequency), which
is suitable for algorithm testing in real situations. The GPU test
system is shown in Figure 3. A new hardware setup with two USB
3.0 cameras is being tested right now.
As it is reported in [21], some parts of the aircraft detection algorithm are already implemented on a special FPGA architecture. The
main advantage of that system is that it is fast and has a low power
requirement. The field of view is approximately 220° ´ 78° with the
resolution of 2250 ´ 752 pixels reached by five micro cameras. The
processing architecture is implemented on a Xilinx Spartan6 L45
FPGA and the frame rate is 56 FPS. The system has an SSD drive
to store the images and telemetry data. The total weight is 450 g and
the power consumption is 4.2 W including the SSD (see Figure 4).
SEPTEMBER 2016

Figure 3.

The nVidia Jetson TK1 development board with cameras.

Figure 4.

Five camera system with FPGA processing.

AIRCRAFT DETECTION ALGORITHM
In our system, three different situations in the distant airplane detection problem are distinguished depending on the image background. This allows us to run algorithms with different strategies
for different situations. This way the overall performance can be
better as each part can be tailored for different needs. In the first
and most trivial case, the aircraft is against a clear sky (unstructured background). Naturally this allows the detection in the largest distance. In this case, the aircraft is robustly detectable already
when it is larger than 3 pixels. The detection range can be 3.7 km
for a Cessna 172 [22].
The second case is when there are clouds behind the aircraft.
In this case, the more structured the clouds are, the more difficult
the detection is. In front of dense, quasi homogeneous clouds, an
aircraft sized at 6 pixels can be typically detected. The third case is
the terrain background, where aircraft sized larger than 12 pixels
can be detected.

IEEE A&E SYSTEMS MAGAZINE

Table of Contents for the Digital Edition of Aerospace and Electronic Systems Magazine September 2016