Aerospace and Electronic Systems Magazine September 2016 - 13

Wilson et al.

Figure 7.

The SOC curves show the detection range and false alarm performance
for a selection of detection thresholds. SOC curves are shown for three
analysis configurations: 1) the original morphological filter's structuring
element (blue); 2) with the nondetection encounters removed (green);
and 3) the new structuring element for the morphological filter (red).
Detection thresholds are labelled at a selection of points on the SOC
curves.

the false alarm rate. Thus, the zero false alarm point on the SOC
curve provides the practical detection range of the SAA system.
No-target data are essential for SAA system characterisation.
The strongest false alarm performance claim that a data set can
make is the reciprocal of the duration of the no-target data. That
is, we assume that a false alarm would have occurred just after the
last frame was recorded. Based on the 16.2 min of no-target data
recorded, we can claim a false alarm rate of 3.7 false alarms per
hour. The real false alarm rate of the system is likely to be significantly lower than this, however, more data are required to make
any stronger claim.
Minimising the false alarm rate is important. One study has
shown that as the false alarm rate of a sensor increases the trust in
the sensor system decreases [34]. Also, a false alarm rate of one
detection per minute, for example, would provide an SAA system
that is impractical, especially if an automated collision-avoidance
manoeuvre is programmed after each detection.
During post flight-test analysis, a change was made to the morphological filter: the size of the structuring element was reduced
from 5×5 to 3×3. We found that the 3×3 structuring element improved the detection performance in all 15 encounters (5 mm lens).
The two encounters that proved particularly problematic with the
5×5 structuring element were easily detected with the 3×3 structuring element. The SOC curve that results from the 3×3 structuring
element shows consistently better detection-range performance
when compared with the flight tested SAA system.
Due to the significant performance improvement we would
recommend a 3×3 pixel structuring element for the morphological filter in future flight tests. We do note, however, that this is
a change from the previously well-tested 5×5 structuring element
[10]. We also note that this change may reduce detection performance if the threat aircraft occupies a large area of the image as
it enters the FOV. Finally, we note that the smaller structuring element is likely to have stricter lens-focussing requirements. A poorly focussed lens will lead to poorer SAA detection performance.
SEPTEMBER 2016

Figure 8.

The SAA detection-range performance versus the take-off weight of the
associated aircraft. A reference for each data point is also shown. The
ScanEagle's SAA detection range is shown by the red diamond. For
comparison purposes the performance of low-time pilots is indicated by
the dashed green horizontal line [20].

The real-time SAA detection results were generated using a
manually set detection threshold that was tuned to the prevailing
imaging conditions. This threshold was set by experienced personnel during the FT&E campaign. The ability to automatically set
the detection threshold will contribute to the readiness of a fully
autonomous SAA solution for unmanned aircraft operations in civilian airspace.

SYSTEM OPTIMISATION
A comparison of the detection ranges provided by the two lenses is
important for the optimisation of the SAA system. The 5 mm lens
provides a wider FOV at the expense of a shorter detection range.
The 8 mm lens provides longer detection ranges at the expense of
a narrow FOV. The 5 mm lens also has less stringent stabilisation
requirements than the 8 mm lens due to the larger area imaged by
each pixel. In general, less stringent stabilisation requirements can
lead to significant cost savings if GPS/INS-based stabilisation is
used.
The SAA system will detect collision-course aircraft within the
FOV. There is, however, no agreement on the FOV requirements
for an SAA system. One option is to mimic the FOV of a pilot, although this is often limited by the cockpit [8]. Nevertheless, multicamera SAA systems are being developed (see, for example, [35]).
SWaP limitations are always an important consideration when
installing embedded systems on unmanned aircraft. Whilst the
SAA system is a critical enabler for the integration of unmanned
aircraft into civilian airspace, the system also consumes SWaP resources that could be used for an additional payload capacity and/
or fuel. SAA system costs are also an important consideration for
commercial operations. Smaller and cheaper SAA systems will always be desired by the customer. In the process of migrating the
SAA system from the Cessna to the ScanEagle we achieved a more
than 20 times reduction in weight, and approximately 10 times

IEEE A&E SYSTEMS MAGAZINE

Table of Contents for the Digital Edition of Aerospace and Electronic Systems Magazine September 2016