Aerospace and Electronic Systems Magazine July 2017 - 51

detailed comparison of their system specifications [3]. Matteoli,
Diani, and Corsini present a survey of methods for processing hyperspectral imagery to detect small human-made anomalies that
are relevant in defense and surveillance applications [10].
We refer to our proposed new approach for multispectral image
processing as LDspectral, where "LD" here stands for lightweight
dataflow [18], [17]. LD is a lightweight design methodology that
facilitates cross-platform prototyping, experimentation, and design
optimization of signal processing systems. LD is "lightweight" in
the sense that it is based on a compact set of application programming interfaces (APIs) that can be retargeted to different platforms
and integrated into different design processes relatively easily. The
lightweight dataflow environment (LIDE) is a software tool that
supports the LD design methodology [17].
We prototype LDspectral using LIDE together with OpenCV,
and present results of extensive experimentation with this prototype to demonstrate the utility of LDspectral. OpenCV provides
a large library of software components for video processing (e.g.,
see [14]), including specialized capabilities that are complementary to the capabilities of LIDE for model-based design and implementation. In particular, the dataflow-based embedded software
components (actors) that we employ to implement LDspectral incorporate calls to relevant OpenCV functions to perform specific
image processing operations.
We demonstrate and evaluate the performance of LDspectral
capabilities using a background subtraction application, along with
a recently introduced data set for experimenting with multispectral background subtraction techniques [1]. As compared with a
standard image processing pipeline, the dynamic, integrated adjustment of data flow and spectral band selection provides systematic trade-off optimization between computational efficiency and
multispectral video processing accuracy.

RELATED WORK
Benezeth et al. [1] present a publicly available collection of multispectral video sequences that includes ground truth annotation
JULY 2017

of moving objects. They also apply this data set to demonstrate
improvements in background subtraction accuracy when using
multispectral video streams compared with RGB (Red, Green, and
Blue) streams. Additionally, they provide an evaluation of alternative background subtraction techniques that operate on multispectral video.
Using the multispectral data set introduced in [1], the proposed
DDDAS motivated LDSpectral system is tested. Figure 1 shows
an example of the data associated with a single video frame within
the employed multispectral data set. Specifically, Figure 1 shows
7 different images corresponding to the 7 different bands for the
same scene and the foreground result of this scene.
Our work on LDspectral is different from the methods discussed in [1] in the emphasis on integrating DDDAS methods into
multispectral video processing, and specifically, on supporting
flexible optimization involving the subset of available multispectral bands that is processed, and the associated trade-offs between
accuracy and computational cost. Additionally, pixel-level fusion
in the front-end of the processing chain for background subtraction is investigated. Pixel-level fusion improves computational efficiency, and reduces the execution time costs incurred by incorporating additional bands (i.e., larger subsets of the available bands)
into the video processing pipeline.
While pixel-level fusion is applied in our demonstration of LDspectral, the LDspectral framework does not require use of pixellevel fusion, nor any other specific form of multiband processing.
This flexibility allows for integration and experimentation with
alternative methods for fusion and analysis of video data across
multiple bands (e.g., see [15], [13], [16]) that may enhance the
available operating points and overall system adaptivity in terms
of accuracy, throughput, and other relevant metrics. Exploration
of such alternative methods in the context of LDspectral is an interesting direction for further study to develop multispectral image
fusion systems with user interaction [9].
The developments in this article provide new models and
methods that are promising for integrating with cloud-computing
frameworks for information fusion, such as the class of frame-

IEEE A&E SYSTEMS MAGAZINE

Table of Contents for the Digital Edition of Aerospace and Electronic Systems Magazine July 2017