Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos

Yan, Yijun and Ren, Jinchang and Zhao, Huimin and Zheng, Jiangbin and Zaihidee, Ezrinda Mohd and Soraghan, John; Chen, Enqing and Gong, Yihong and Tie, Yun, eds. (2016) Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos. In: Advances in Multimedia Information Processing - PCM 2016. Lecture Notes in Computer Science . Springer, CHN, pp. 697-704. ISBN 9783319488950 (https://doi.org/10.1007/978-3-319-48896-7_69)

[thumbnail of Yan-etal-PMC2016-Fusion-of-thermal-and-visible-imagery-for-effective-detection-and-tracking]
Preview
 Text. Filename: Yan_etal_PMC2016_Fusion_of_thermal_and_visible_imagery_for_effective_detection_and_tracking.pdf
Accepted Author Manuscript
Download (849kB)| Preview

Abstract

In this paper, we present an efficient approach to detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by background-subtraction for foreground detection in visible images. Meanwhile, adaptively thresholding is applied for foreground detection in thermal domain as human objects tend to be of higher temperature thus brighter than the background. To deal with cases of occlusion, prediction based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. The proposed method is evaluated on OTCBVS, a publicly available color-thermal benchmark dataset. Promising results have shown that the proposed fusion based approach can successfully detect and track multiple human objects.

ORCID iDs

Yan, Yijun ORCID logoORCID: https://orcid.org/0000-0003-0224-0078, Ren, Jinchang ORCID logoORCID: https://orcid.org/0000-0001-6116-3194, Zhao, Huimin, Zheng, Jiangbin, Zaihidee, Ezrinda Mohd and Soraghan, John ORCID logoORCID: https://orcid.org/0000-0003-4418-7391; Chen, Enqing, Gong, Yihong and Tie, Yun
CORE (COnnecting REpositories)