Thermal infrared target tracking via total variation sparse denoising and spatiotemporal memory regularization

Objective Thermal infrared (TIR) target tracking plays an important role in surveillance and autonomous systems, but TIR images often suffer from low resolution, weak texture, sensor noise, and complex background interference. These limitations make it difficult for correlation filter trackers to obtain reliable target representations and maintain accurate localization, particularly under occlusion, rapid motion, and motion blur. In addition, background thermal interference and noise can affect filter updating and lead to tracking drift. This study aims to develop a robust and efficient correlation filter tracking method for challenging TIR scenarios. The objective is to improve image quality and target representation while enhancing spatial discrimination and temporal consistency, thereby achieving more accurate and stable target tracking without introducing excessive computational complexity.

Methods Based on the Adaptive Regression-based Correlation Filter (ARCF), this study proposes a Total Variation Sparse Denoising and Spatiotemporal Memory Regularization Correlation Filter (TVSTCF) tracker. The proposed method consists of three complementary components: total variation sparse denoising (TVSD), spatial-aware regularization (SAAR), and temporal memory regularization (TMMR). 1) The TVSD module is designed to alleviate the adverse effects of noise and weak structural information in TIR images. It combines total variation regularization with a non-local low-rank representation to exploit both local image smoothness and non-local structural similarity. The observed image is decomposed into low-rank structural components, sparse anomaly components, and residual noise, allowing the principal target and background structures to be preserved while suppressing noise. A log-sum non-convex low-rank regularizer is further approximated using an iteratively reweighted strategy, and the resulting subproblems are solved using the alternating direction method of multipliers (ADMM) and Split-Bregman optimization. 2) The SAAR module replaces the fixed spatial weighting strategy in the baseline CF tracker with an adaptive spatial regularization matrix. A spatial structural prior based on a two-dimensional Gaussian model is combined with the local dispersion of multi-channel features. The resulting regularization weights impose weaker constraints in the target-centered region and stronger penalties in background-dominated regions, thereby reducing the influence of background clutter and improving the discriminative capability of the filter.3) The TMMR module addresses the instability of single-frame temporal constraints by constructing a multi-frame response memory model. Historical response maps from the previous Kframes are weighted according to an exponential temporal decay strategy, with greater weights assigned to more recent frames. The resulting temporal memory response provides a more stable reference for constraining the current response and helps reduce abnormal responses caused by occlusion, noise, and motion blur. To efficiently optimize the resulting objective function, an auxiliary variable is introduced to decouple the convolution term from the spatial-aware regularization term, and ADMM is used to derive closed-form updates for the correlation filter, auxiliary variable, and Lagrange multiplier.

Results and Discussions Extensive experiments were conducted on four TIR tracking benchmarks, including LSOTB-TIR, PTB-TIR, VOT-TIR2015, and VOT-TIR2017. The results demonstrate that the proposed TVSTCF consistently improves tracking accuracy and robustness over the ARCF baseline. On LSOTB-TIR, TVSTCF achieves a precision of 0.778, a success score of 0.640, and a Normalized precision of 0.697, while on PTB-TIR, it obtains a precision of 0.836 and a success score of 0.636. On the VOT-TIR2015 and VOT-TIR2017 benchmarks, TVSTCF achieves the highest EAO scores of 0.346 and 0.330, respectively, demonstrating its strong robustness under challenging thermal infrared conditions. Ablation experiments further confirm that TVSD, SAAR, and TMMR provide complementary improvements by enhancing image quality, target-background discrimination, and temporal consistency, respectively. Qualitative and attribute-based evaluations show that the proposed method performs particularly well under occlusion, background clutter, motion blur, and rapid motion. Although the additional denoising and regularization procedures introduce computational overhead, TVSTCF maintains approximately 30 f/s on an NVIDIA RTX 4060 GPU, providing a favorable balance between tracking performance and computational efficiency.

Conclusions The proposed TVSTCF provides an effective CF-based solution for TIR target tracking by jointly addressing image degradation, spatial background interference, and temporal response instability. The TVSD module improves the quality and structural representation of TIR images through total variation and non-local low-rank modeling. The SAAR module adaptively regulates the spatial constraints according to target structure and multi-channel feature stability, thereby enhancing target-background discrimination. The TMMR module introduces multi-frame historical response information to provide a more stable temporal constraint and reduce the influence of abnormal responses. Extensive experiments on four TIR tracking benchmarks demonstrate that the proposed method consistently improves tracking accuracy and robustness over the ARCF baseline and achieves competitive or superior performance compared with representative CF, deep learning, Siamese-network, and Transformer-based trackers. In particular, TVSTCF achieves the best EAO on both VOT-TIR2015 and VOT-TIR2017 while maintaining approximately real-time processing speed. These results confirm the effectiveness of combining image denoising with adaptive spatial and temporal regularization for complex TIR target tracking. Although the proposed method achieves clear performance improvements, its computational efficiency remains affected by the iterative denoising procedure. Future work will therefore focus on more robust and lightweight optimization strategies to further improve real-time performance while maintaining tracking accuracy in extreme TIR scenarios.