Aerospace and Electronic Systems Magazine July 2017 - 50

Feature Article:

DOI. No. 10.1109/MAES.2017.160132

Dynamic, Data-Driven Processing of Multispectral
Video Streams
Honglei Li, Kishan Sudusinghe, Yanzhou Liu, University of Maryland College Park,
College Park, MD, USA
Jinsung Yoon, Mihaela van der Schaar, University of California Los Angeles, Los
Angeles, CA, USA
Erik Blasch, Air Force Research Laboratory, Rome, NY, USA
Shuvra S. Bhattacharyya, University of Maryland College Park, College Park, MD,
USA and Tampere University of Technology, Tampere, Finland

INTRODUCTION
Video analytics plays an important role in a wide variety of defense-, monitoring- and surveillance-related systems for air and
ground environments. In this context, multispectral video processing is attracting increased interest in recent years, due in part to
technological advances in video capture. Compared with monochromatic video, multispectral video offers better spectral resolution, and different bands of multispectral video streams can enhance video analytics capabilities in different ways. For example,
the infrared bands can provide better separation of shadows from
objects, and improved spatial resolution in scenes that are impaired
by fog or haze [16].
Multispectral video acquisition technology introduces novel
opportunities and challenges for applying the paradigm of dynamic, data-driven applications systems (DDDAS) [5] to the design and implementation of video analytics systems. The subset
of available multispectral bands that is stored and processed, and
the hardware and software configurations that are used to perform
the processing introduce a complex design space. Furthermore, the
most effective operating point in this design space is dependent on
the specific application scenario and data characteristics that are
encountered at a given point in time during system operation. For
example, when system accuracy is of greatest importance, it may
Authors' current addresses: H. Li, K. Sudusinghe, Y. Liu, S. S.
Bhattacharyya, University of Maryland College Park, 7950 Baltimore Ave., College Park, MD 20742, USA. E-mail: (honglei@
umd.edu). J. Yoon, M. van der Schaar, University of California
Los Angeles, Los Angeles, CA; E. Blasch, Air Force Research
Laboratory, Rome, NY; S. S. Bhattacharyya, University of
Maryland College Park, 7950 Baltimore Avenue, College Park,
MD 20742, and with the Department of Pervasive Computing,
Tampere University of Technology, Tampere, Finland.
Manuscript received June 8, 2016, revised October 11, 2016,
and ready for publication December 12, 2016.
Review handled by M. Cardinale.
0885/8985/17/$26.00 © 2017 IEEE
50

be desirable to operate on the full set of available multispectral
bands, while in situations where resource constraints are critical
(e.g., due to failures in certain subsystems or limited energy capacity), it may be most effective to select a proper subset of the available bands and process the selected bands in a way that optimizes
accuracy subject to the given resource limitations.
Based on this view of selectively processed bands from multispectral video data, we introduce in this article a novel system
design framework for dynamic, data-driven video processing. A
central part of our framework is the application of model-based
design methods based on dataflow techniques to represent and
transform the functionality of multispectral video processing systems. Existing dataflow techniques are employed for design optimization in a wide variety of signal processing application areas,
including speech processing, wireless communications, and video
processing (e.g., see [2]).
The approach that we discuss supports the development of new
DDDAS methods to dynamically select subsets of multispectral
bands to process, and dynamically reconfigure the dataflow within
the targeted video processing system to achieve the required processing on the selected subset of bands. DDDAS is a paradigm
that unifies computational and instrumentation aspects of applications systems, and thereby promotes deeply integrated approaches
to modeling, sensing, control, and data processing. DDDAS principles have great relevance to aerospace applications (e.g., see [4],
[11], [19]).
Multispectral imaging is related to hyperspectral imaging in
that both provide increased spectral discrimination compared with
traditional imaging methods. The difference is primarily in the
number of bands employed and the degree of spectral resolution
(e.g., see [7]). Whereas multispectral imaging generally refers to a
set of 3-10 bands, hyperspectral imaging uses significantly larger
numbers of bands-e.g., hundreds, thousands, or more-and narrower bandwidths. Thus, this article is complementary to tutorials
in IEEE AESM that have covered aspects of hyperspectral imaging. For example, Birk and McCord provide a review of many different airborne hyperspectral sensing systems, and also provide a

IEEE A&E SYSTEMS MAGAZINE

JULY 2017



Table of Contents for the Digital Edition of Aerospace and Electronic Systems Magazine July 2017

No label
Aerospace and Electronic Systems Magazine July 2017 - No label
Aerospace and Electronic Systems Magazine July 2017 - Cover2
Aerospace and Electronic Systems Magazine July 2017 - 1
Aerospace and Electronic Systems Magazine July 2017 - 2
Aerospace and Electronic Systems Magazine July 2017 - 3
Aerospace and Electronic Systems Magazine July 2017 - 4
Aerospace and Electronic Systems Magazine July 2017 - 5
Aerospace and Electronic Systems Magazine July 2017 - 6
Aerospace and Electronic Systems Magazine July 2017 - 7
Aerospace and Electronic Systems Magazine July 2017 - 8
Aerospace and Electronic Systems Magazine July 2017 - 9
Aerospace and Electronic Systems Magazine July 2017 - 10
Aerospace and Electronic Systems Magazine July 2017 - 11
Aerospace and Electronic Systems Magazine July 2017 - 12
Aerospace and Electronic Systems Magazine July 2017 - 13
Aerospace and Electronic Systems Magazine July 2017 - 14
Aerospace and Electronic Systems Magazine July 2017 - 15
Aerospace and Electronic Systems Magazine July 2017 - 16
Aerospace and Electronic Systems Magazine July 2017 - 17
Aerospace and Electronic Systems Magazine July 2017 - 18
Aerospace and Electronic Systems Magazine July 2017 - 19
Aerospace and Electronic Systems Magazine July 2017 - 20
Aerospace and Electronic Systems Magazine July 2017 - 21
Aerospace and Electronic Systems Magazine July 2017 - 22
Aerospace and Electronic Systems Magazine July 2017 - 23
Aerospace and Electronic Systems Magazine July 2017 - 24
Aerospace and Electronic Systems Magazine July 2017 - 25
Aerospace and Electronic Systems Magazine July 2017 - 26
Aerospace and Electronic Systems Magazine July 2017 - 27
Aerospace and Electronic Systems Magazine July 2017 - 28
Aerospace and Electronic Systems Magazine July 2017 - 29
Aerospace and Electronic Systems Magazine July 2017 - 30
Aerospace and Electronic Systems Magazine July 2017 - 31
Aerospace and Electronic Systems Magazine July 2017 - 32
Aerospace and Electronic Systems Magazine July 2017 - 33
Aerospace and Electronic Systems Magazine July 2017 - 34
Aerospace and Electronic Systems Magazine July 2017 - 35
Aerospace and Electronic Systems Magazine July 2017 - 36
Aerospace and Electronic Systems Magazine July 2017 - 37
Aerospace and Electronic Systems Magazine July 2017 - 38
Aerospace and Electronic Systems Magazine July 2017 - 39
Aerospace and Electronic Systems Magazine July 2017 - 40
Aerospace and Electronic Systems Magazine July 2017 - 41
Aerospace and Electronic Systems Magazine July 2017 - 42
Aerospace and Electronic Systems Magazine July 2017 - 43
Aerospace and Electronic Systems Magazine July 2017 - 44
Aerospace and Electronic Systems Magazine July 2017 - 45
Aerospace and Electronic Systems Magazine July 2017 - 46
Aerospace and Electronic Systems Magazine July 2017 - 47
Aerospace and Electronic Systems Magazine July 2017 - 48
Aerospace and Electronic Systems Magazine July 2017 - 49
Aerospace and Electronic Systems Magazine July 2017 - 50
Aerospace and Electronic Systems Magazine July 2017 - 51
Aerospace and Electronic Systems Magazine July 2017 - 52
Aerospace and Electronic Systems Magazine July 2017 - 53
Aerospace and Electronic Systems Magazine July 2017 - 54
Aerospace and Electronic Systems Magazine July 2017 - 55
Aerospace and Electronic Systems Magazine July 2017 - 56
Aerospace and Electronic Systems Magazine July 2017 - 57
Aerospace and Electronic Systems Magazine July 2017 - 58
Aerospace and Electronic Systems Magazine July 2017 - 59
Aerospace and Electronic Systems Magazine July 2017 - 60
Aerospace and Electronic Systems Magazine July 2017 - 61
Aerospace and Electronic Systems Magazine July 2017 - 62
Aerospace and Electronic Systems Magazine July 2017 - 63
Aerospace and Electronic Systems Magazine July 2017 - 64
Aerospace and Electronic Systems Magazine July 2017 - Cover3
Aerospace and Electronic Systems Magazine July 2017 - Cover4
http://www.brightcopy.net/allen/aesm/34-2s
http://www.brightcopy.net/allen/aesm/34-2
http://www.brightcopy.net/allen/aesm/34-1
http://www.brightcopy.net/allen/aesm/33-12
http://www.brightcopy.net/allen/aesm/33-11
http://www.brightcopy.net/allen/aesm/33-10
http://www.brightcopy.net/allen/aesm/33-09
http://www.brightcopy.net/allen/aesm/33-8
http://www.brightcopy.net/allen/aesm/33-7
http://www.brightcopy.net/allen/aesm/33-5
http://www.brightcopy.net/allen/aesm/33-4
http://www.brightcopy.net/allen/aesm/33-3
http://www.brightcopy.net/allen/aesm/33-2
http://www.brightcopy.net/allen/aesm/33-1
http://www.brightcopy.net/allen/aesm/32-10
http://www.brightcopy.net/allen/aesm/32-12
http://www.brightcopy.net/allen/aesm/32-9
http://www.brightcopy.net/allen/aesm/32-11
http://www.brightcopy.net/allen/aesm/32-8
http://www.brightcopy.net/allen/aesm/32-7s
http://www.brightcopy.net/allen/aesm/32-7
http://www.brightcopy.net/allen/aesm/32-6
http://www.brightcopy.net/allen/aesm/32-5
http://www.brightcopy.net/allen/aesm/32-4
http://www.brightcopy.net/allen/aesm/32-3
http://www.brightcopy.net/allen/aesm/32-2
http://www.brightcopy.net/allen/aesm/32-1
http://www.brightcopy.net/allen/aesm/31-12
http://www.brightcopy.net/allen/aesm/31-11s
http://www.brightcopy.net/allen/aesm/31-11
http://www.brightcopy.net/allen/aesm/31-10
http://www.brightcopy.net/allen/aesm/31-9
http://www.brightcopy.net/allen/aesm/31-8
http://www.brightcopy.net/allen/aesm/31-7
https://www.nxtbookmedia.com