E-mail Alert
RSS
| Citation: |
Yang K, Liu F, Liang SY et al. Data-driven polarimetric imaging: a review. Opto-Electron Sci 3, 230042 (2024). doi: 10.29026/oes.2024.230042
|
-
Abstract
This study reviews the recent advances in data-driven polarimetric imaging technologies based on a wide range of practical applications. The widespread international research and activity in polarimetric imaging techniques demonstrate their broad applications and interest. Polarization information is increasingly incorporated into convolutional neural networks (CNN) as a supplemental feature of objects to improve performance in computer vision task applications. Polarimetric imaging and deep learning can extract abundant information to address various challenges. Therefore, this article briefly reviews recent developments in data-driven polarimetric imaging, including polarimetric descattering, 3D imaging, reflection removal, target detection, and biomedical imaging. Furthermore, we synthetically analyze the input, datasets, and loss functions and list the existing datasets and loss functions with an evaluation of their advantages and disadvantages. We also highlight the significance of data-driven polarimetric imaging in future research and development.-
Keywords:
- deep learning /
- polarimetric imaging /
- image processing
-
-
References
[1] Ronchi V. Barocas V. The Nature of Light: An Historical Survey (Harvard University Press, Cambridge, 1970). [2] Huard S. Polarization of Light (Wiley, Chichester, 1997). [3] Schechner YY, Karpel N. Recovery of underwater visibility and structure by polarization analysis. IEEE J Oceanic Eng 30, 570–587 (2005). doi: 10.1109/JOE.2005.850871 [4] Li XB, Hu HF, Zhao L et al. Polarimetric image recovery method combining histogram stretching for underwater imaging. Sci Rep 8, 12430 (2018). doi: 10.1038/s41598-018-30566-8 [5] Liu TG, Guan ZJ, Li XB et al. Polarimetric underwater image recovery for color image with crosstalk compensation. Opt Laser Eng 124, 105833 (2020). doi: 10.1016/j.optlaseng.2019.105833 [6] Liang J, Ren LY, Qu ES et al. Method for enhancing visibility of hazy images based on polarimetric imaging. Photonics Res 2, 38–44 (2014). doi: 10.1364/PRJ.2.000038 [7] Liu F, Han PL, Wei Y et al. Deeply seeing through highly turbid water by active polarization imaging. Opt Lett 43, 4903–4906 (2018). doi: 10.1364/OL.43.004903 [8] Liu F, Wei Y, Han PL et al. Polarization-based exploration for clear underwater vision in natural illumination. Opt Express 27, 3629–3641 (2019). doi: 10.1364/OE.27.003629 [9] Wei Y, Han PL, Liu F et al. Enhancement of underwater vision by fully exploiting the polarization information from the Stokes vector. Opt Express 29, 22275–22287 (2021). doi: 10.1364/OE.433072 [10] Li X, Liu F, Han PL et al. Near-infrared monocular 3D computational polarization imaging of surfaces exhibiting nonuniform reflectance. Opt Express 29, 15616–15630 (2021). [11] Han PL, Cai YD, Liu F et al. Computational polarization 3D: new solution for monocular shape recovery in natural conditions. Opt Laser Eng 151, 106925 (2022). doi: 10.1016/j.optlaseng.2021.106925 [12] Cui ZP, Gu JW, Shi BX et al. Polarimetric multi-view stereo. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2017); http://doi.org/10.1109/CVPR.2017.47. [13] Kadambi A, Taamazyan V, Shi BX et al. Depth sensing using geometrically constrained polarization normals. Int J Comput Vis 125, 34–51 (2017). doi: 10.1007/s11263-017-1025-7 [14] Kong N, Tai YW, Shin JS. A physically-based approach to reflection separation: from physical modeling to constrained optimization. IEEE Trans Pattern Anal Mach Intell 36, 209–221 (2014). doi: 10.1109/TPAMI.2013.45 [15] Bronstein AM, Bronstein MM, Zibulevsky M et al. Sparse ICA for blind separation of transmitted and reflected images. Int J Imag Syst Technol 15, 84–91 (2005). doi: 10.1002/ima.20042 [16] Forssell G. Test and analysis of the detectability of personnel mines in a realistic minefield by polarization in the infrared LW region. Proc SPIE 5415, 187–195 (2004). doi: 10.1117/12.542700 [17] Forssell G. Passive IR polarization measurements applied to covered surface landmines. Proc SPIE 5089, 547–557 (2003). doi: 10.1117/12.487163 [18] Cremer F, de Jong W, Schutte K. Infrared polarization measurements and modelling applied to surface laid anti-personnel landmines. Opt Eng 41, 1021–1032 (2002). doi: 10.1117/1.1467362 [19] Aron Y, Gronau Y. Polarization in the LWIR: a method to improve target aquisition. Proc SPIE 5783, 653–661 (2005). doi: 10.1117/12.605316 [20] Ratliff BM, Lemaster DA, Mack RT et al. Detection and tracking of RC model aircraft in LWIR microgrid polarimeter data. Proc SPIE 8160, 816002 (2011). [21] Voss KJ, Fry ES. Measurement of the Mueller matrix for ocean water. Appl Opt 23, 4427–4439 (1984). doi: 10.1364/AO.23.004427 [22] Fry ES, Voss KJ. Measurement of the Mueller matrix for phytoplankton. Limnol Oceanogr 30, 1322–1326 (1985). doi: 10.4319/lo.1985.30.6.1322 [23] Svensen Ø, Stamnes JJ, Kildemo M et al. Mueller matrix measurements of algae with different shape and size distributions. Appl Opt 50, 5149–5157 (2011). doi: 10.1364/AO.50.005149 [24] Wang WF, Lim LG, Srivastava S et al. Investigation on the potential of Mueller matrix imaging for digital staining. J. Biophotonics 9, 364–375 (2016). doi: 10.1002/jbio.201500006 [25] Du E, He HH, Zeng N et al. Mueller matrix polarimetry for differentiating characteristic features of cancerous tissues. J Biomed Opt 19, 076013 (2014). doi: 10.1117/1.JBO.19.7.076013 [26] Le DL, Huynh TN, Nguyen DT et al. Characterization of healthy and nonmelanoma-induced mouse utilizing the Stokes-Mueller decomposition. J Biomed Opt 23, 125003 (2018). [27] Pierangelo A, Manhas S, Benali A et al. Multispectral Mueller polarimetric imaging detecting residual cancer and cancer regression after neoadjuvant treatment for colorectal carcinomas. J Biomed Opt 18, 046014 (2013). doi: 10.1117/1.JBO.18.4.046014 [28] Shukla P, Pradhan A. Mueller decomposition images for cervical tissue: potential for discriminating normal and dysplastic states. Opt Express 17, 1600–1609 (2009). doi: 10.1364/OE.17.001600 [29] Pierangelo A, Nazac A, Benali A et al. Polarimetric imaging of uterine cervix: a case study. Opt Express 21, 14120–14130 (2013). doi: 10.1364/OE.21.014120 [30] Egan WG. Polarization in remote sensing. Proc SPIE 0891 (1988). https://doi.org/10.1117/12.944289 [31] David G, Thomas B, Dupart Y et al. UV polarization lidar for remote sensing new particles formation in the atmosphere. Opt Express 22, A1009–A1022 (2014). doi: 10.1364/OE.22.0A1009 [32] Carotenuto V, de Maio A, Clemente C et al. Invariant rules for multipolarization SAR change detection. IEEE Trans Geosci Remote Sens 53, 3294–3311 (2015). doi: 10.1109/TGRS.2014.2372900 [33] Nagdimunov L, Kolokolova L, Mackowski D. Characterization and remote sensing of biological particles using circular polarization. J Quant Spectrosc Radiat Transfer 131, 59–65 (2013). doi: 10.1016/j.jqsrt.2013.04.018 [34] Wang F, Ainouz S, Lian CF et al. Multimodality semantic segmentation based on polarization and color images. Neurocomputing 253, 193–200 (2017). doi: 10.1016/j.neucom.2016.10.090 [35] Xiang KT, Yang KL, Wang KW. Polarization-driven semantic segmentation via efficient attention-bridged fusion. Opt Express 29, 4802–4820 (2021). doi: 10.1364/OE.416130 [36] Ni J, Zhang F, Ma F et al. Random region matting for the high-resolution PolSAR image semantic segmentation. IEEE J Sel Top Appl Earth Obs Remote Sens 14, 3040–3051 (2021). doi: 10.1109/JSTARS.2021.3062447 [37] Li ZS, Sun JS, Fan Y et al. Deep learning assisted variational Hilbert quantitative phase imaging. Opto-Electron Sci 2, 220023 (2023). doi: 10.29026/oes.2023.220023 [38] Wang YYD, Wang H, Gu M. High performance “non-local” generic face reconstruction model using the lightweight Speckle-Transformer (SpT) UNet. Opto-Electron Adv 6, 220049 (2023). doi: 10.29026/oea.2023.220049 [39] Guo YM, Zhong LB, Min L et al. Adaptive optics based on machine learning: a review. Opto-Electron Adv 5, 200082 (2022). doi: 10.29026/oea.2022.200082 [40] Li YX, Qian JM, Feng SJ et al. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021 [41] Chen YX, Zhang FY, Dang ZB et al. Chiral detection of biomolecules based on reinforcement learning. Opto-Electron Sci 2, 220019 (2023). doi: 10.29026/oes.2023.220019 [42] Hu HF, Lin Y, Li XB et al. IPLNet: a neural network for intensity-polarization imaging in low light. Opt Lett 45, 6162–6165 (2020). doi: 10.1364/OL.409673 [43] Zeng XL, Luo Y, Zhao XJ et al. An end-to-end fully-convolutional neural network for division of focal plane sensors to reconstruct S0, DoLP, and AoP. Opt Express 27, 8566–8577 (2019). doi: 10.1364/OE.27.008566 [44] Wu RY, Zhao YQ, Li N et al. Real-time division-of-focal-plane polarization imaging system with progressive networks. arXiv: 2110.13823 (2021). https://doi.org/10.48550/arXiv.2110.13823 [45] Li XB, Li HY, Lin Y et al. Learning-based denoising for polarimetric images. Opt Express 28, 16309–16321 (2020). doi: 10.1364/OE.391017 [46] Hu HF, Zhang YB, Li XB et al. Polarimetric underwater image recovery via deep learning. Opt Laser Eng 133, 106152 (2020). doi: 10.1016/j.optlaseng.2020.106152 [47] Ding XY, Wang YF, Fu XP. Multi-polarization fusion generative adversarial networks for clear underwater imaging. Opt Laser Eng 152, 106971 (2022). doi: 10.1016/j.optlaseng.2022.106971 [48] Zhou C, Teng MG, Han YF et al. Learning to dehaze with polarization. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS, 2021). [49] Zhu YM, Zeng TJ, Liu KW et al. Full scene underwater imaging with polarization and an untrained network. Opt Express 29, 41865–41881 (2021). doi: 10.1364/OE.444755 [50] Xu XY, Wan MG, Ge JY et al. ColorPolarNet: residual dense network-based chromatic intensity-polarization imaging in low-light environment. IEEE Trans Instrum Meas 71, 5025210 (2022). [51] Gao SH, Cao Y, Zhang WJ et al. Learning feature fusion for target detection based on polarimetric imaging. Appl Opt 61, D15–D21 (2022). doi: 10.1364/AO.441183 [52] Blin R, Ainouz S, Canu S et al. Road scenes analysis in adverse weather conditions by polarization-encoded images and adapted deep learning. In Proceedings of 2019 IEEE Intelligent Transportation Systems Conference 27–32 (IEEE, 2019); https://doi.org/10.1109/ITSC.2019.8916853. [53] Lei CY, Huang XH, Zhang MD et al. Polarized reflection removal with perfect alignment in the wild. In Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition 1747–1755 (IEEE, 2020); https://doi.org/10.1109/CVPR42600.2020.00182. [54] Fan W, Ainouz S, Meriaudeau F et al. Polarization-based car detection. In Proceedings of the 2018 25th IEEE International Conference on Image Processing 3069–3073 (IEEE, 2018); https://doi.org/10.1109/ICIP.2018.8451397. [55] Xie RC, Zu HY, Xue Y et al. Target detection method for polarization imaging based on convolutional neural network. Proc SPIE 11455, 114557Z (2020). [56] Zhang Y, Morel O, Blanchon M et al. Exploration of deep learning-based multimodal fusion for semantic road scene segmentation. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications 336–343 (SciTePress, 2019); https://doi.org/10.5220/0007360403360343. [57] Kalra A, Taamazyan V, Rao SK et al. Deep polarization cues for transparent object segmentation. In Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition 8599–8608 (IEEE, 2020); https://doi.org/10.1109/CVPR42600.2020.00863. [58] Si L, Huang TY, Wang XJ et al. Deep learning-based polarization feature retrieval from a single stokes vector. Proc SPIE 11963, 1196307 (2022). [59] Sun R, Sun XB, Chen FN et al. Polarimetric imaging detection using a convolutional neural network with three-dimensional and two-dimensional convolutional layers. Appl Opt 59, 151–155 (2020). doi: 10.1364/AO.59.000151 [60] Li XP, Liao R, Zhou JL et al. Classification of morphologically similar algae and cyanobacteria using Mueller matrix imaging and convolutional neural networks. Appl Opt 56, 6520–6530 (2017). doi: 10.1364/AO.56.006520 [61] Li XP, Liao R, Ma H et al. Polarimetric learning: a Siamese approach to learning distance metrics of algal Mueller matrix images. Appl Opt 57, 3829–3837 (2018). doi: 10.1364/AO.57.003829 [62] Zhao YQ, Reda M, Feng K et al. Detecting giant cell tumor of bone lesions using Mueller matrix polarization microscopic imaging and multi-parameters fusion network. IEEE Sens J 20, 7208–7215 (2020). doi: 10.1109/JSEN.2020.2978021 [63] Dong Y, Wan JC, Wang XJ et al. A polarization-imaging-based machine learning framework for quantitative pathological diagnosis of cervical precancerous lesions. IEEE Trans Med Imaging 40, 3728–3738 (2021). doi: 10.1109/TMI.2021.3097200 [64] Ba YH, Gilbert A, Wang F et al. Deep shape from polarization. In Proceedings of the 16th European Conference on Computer Vision 554–571 (Springer, 2020); https://doi.org/10.1007/978-3-030-58586-0_33. [65] Zou SH, Zuo ZX, Qian YM et al. 3D human shape reconstruction from a polarization image. In Proceedings of the 16th European Conference on Computer Vision 351–368 (Springer, 2020); https://doi.org/10.1007/978-3-030-58568-6_21 [66] Li DK, Lin B, Wang XY et al. High-performance polarization remote sensing with the modified U-Net based deep-learning network. IEEE Trans Geosci Remote Sens 60, 5621110 (2022). [67] Lei CY, Qi CY, Xie JX et al. Shape from polarization for complex scenes in the wild. In Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition 12622–12631 (IEEE, 2021); https://doi.org/10.1109/CVPR52688.2022.01230. [68] Zhang JC, Shao JB, Luo HB et al. Learning a convolutional demosaicing network for microgrid polarimeter imagery. Opt Lett 43, 4534–4537 (2018). doi: 10.1364/OL.43.004534 [69] Yang KL, Bergasa LM, Romera E et al. Predicting polarization beyond semantics for wearable robotics. In Proceedings of the 2018 IEEE-RAS 18th International Conference on Humanoid Robots 96–103 (IEEE, 2018); https://doi.org/10.1109/HUMANOIDS.2018.8625005. [70] Zhang R, Gui XY, Cheng HY et al. Underwater image recovery utilizing polarimetric imaging based on neural networks. Appl Opt 60, 8419–8425 (2021). doi: 10.1364/AO.431299 [71] Kondo Y, Ono T, Sun LG et al. Accurate polarimetric BRDF for real polarization scene rendering. In Proceedings of the 16th European Conference on Computer Vision 220–236 (Springer, 2020); https://doi.org/10.1007/978-3-030-58529-7_14. [72] Shi YJ, Guo EL, Bai LF et al. Polarization-based haze removal using self-supervised network. Front Phys 9, 789232 (2022). doi: 10.3389/fphy.2021.789232 [73] Wieschollek P, Gallo O, Gu JW et al. Separating reflection and transmission images in the wild. In Proceedings of the 15th European Conference on Computer Vision 90–105 (Springer, 2018); https://doi.org/10.1007/978-3-030-01261-8_6. [74] Lyu YW, Cui ZP, Li S et al. Reflection separation using a pair of unpolarized and polarized images. In Proceedings of the 33rd International Conference on Neural Information Processing Systems 1304 (Curran Associates Inc. , 2019); https://doi.org/10.5555/3454287.3455591. [75] Liu TR, de Haan K, Bai BJ et al. Deep learning-based holographic polarization microscopy. ACS Photonics 7, 3023–3034 (2020). doi: 10.1021/acsphotonics.0c01051 [76] Yang SL, Qu BW, Liu GS et al. Unsupervised learning polarimetric underwater image recovery under nonuniform optical fields. Appl Opt 60, 8198–8205 (2021). doi: 10.1364/AO.432994 [77] Liu HD, Zhang YZ, Cheng ZZ et al. Attention-based neural network for polarimetric image denoising. Opt Lett 47, 2726–2729 (2022). doi: 10.1364/OL.458514 [78] Yang XJ, Zhao QH, Huang TY et al. Deep learning for denoising in a Mueller matrix microscope. Opt Express 13, 3535–3551 (2022). doi: 10.1364/BOE.457219 [79] Wu XS, Zhang H, Hu XP et al. HDR reconstruction based on the polarization camera. IEEE Robotics Autom Lett 5, 5113–5119 (2020). doi: 10.1109/LRA.2020.3005379 [80] Deschaintre V, Lin YM, Ghosh A. Deep polarization imaging for 3D shape and SVBRDF acquisition. In Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition 15562–15571 (IEEE, 2021); https://doi.org/10.1109/CVPR46437.2021.01531. [81] Shao MQ, Xia CK, Yang ZD et al. Transparent shape from a single view polarization image. arXiv: 2204.06331 (2023)https://doi.org/10.48550/arXiv.2204.0633. [82] De S, Bruzzone L, Bhattacharya A et al. A novel technique based on deep learning and a synthetic target database for classification of urban areas in PolSAR data. IEEE J Sel Top Appl Earth Obs Remote Sens 11, 154–170 (2018). doi: 10.1109/JSTARS.2017.2752282 [83] Blanchon M, Morel O, Seulin R et al. Outdoor scenes pixel-wise semantic segmentation using polarimetry and fully convolutional network. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications 328–335 (SciTePress, 2019); https://doi.org/10.5220/0007360203280335. [84] Akkaynak D, Treibitz T, Shlesinger T et al. What is the space of attenuation coefficients in underwater computer vision? In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition 568–577 (IEEE, 2017); http://doi.org/10.1109/CVPR.2017.68. [85] Wang YH, Louie DC, Cai JY et al. Deep learning enhances polarization speckle for in vivo skin cancer detection. Opt Laser Technol 140, 107006 (2021). doi: 10.1016/j.optlastec.2021.107006 [86] Zhou XM, Ma L, Brown W et al. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and machine learning. Proc SPIE 11603, 116030Q (2021). doi: 10.1117/12.2582330 [87] Yao Y, Zuo M, Dong Y et al. Polarization imaging feature characterization of different endometrium phases by machine learning. OSA Continuum 4, 1776–1791 (2021). doi: 10.1364/OSAC.414109 [88] Azzam RMA. Division-of-amplitude photopolarimeter (DOAP) for the simultaneous measurement of all four Stokes parameters of light. Opt Acta Int J Opt 29, 685–689 (1982). doi: 10.1080/713820903 [89] Jellison GE. Four-channel polarimeter for time-resolved ellipsometry. Opt Lett 12, 766–768 (1987). doi: 10.1364/OL.12.000766 [90] Compain E, Drevillon B. Broadband division-of-amplitude polarimeter based on uncoated prisms. Appl Opt 37, 5938–5944 (1998). doi: 10.1364/AO.37.005938 [91] Ju HJ, Ren LY, Liang J et al. A Mueller matrix measurement technique based on a division-of-aperture polarimetric camera. Proc SPIE 10839, 108391F (2019). [92] Pezzaniti JL, Chenault DB. A division of aperture MWIR imaging polarimeter. Proc SPIE 5888, 58880V (2005). doi: 10.1117/12.623543 [93] Gao SK, Gruev V. Bilinear and bicubic interpolation methods for division of focal plane polarimeters. Opt Express 19, 26161–26173 (2011). doi: 10.1364/OE.19.026161 [94] York T, Gruev V. Calibration method for division of focal plane polarimeters in the optical and near-infrared regime. Proc SPIE 8012, 80120H (2011). [95] Hsu WL, Davis J, Balakrishnan K et al. Polarization microscope using a near infrared full-Stokes imaging polarimeter. Opt Express 23, 4357–4368 (2015). doi: 10.1364/OE.23.004357 [96] Goldstein DH. Polarized Light 3rd ed (CRC Press, Boca Raton, 2017). [97] Li XB, Liu TG, Huang BJ et al. Optimal distribution of integration time for intensity measurements in stokes polarimetry. Opt Express 23, 27690–27699 (2015). doi: 10.1364/OE.23.027690 [98] Mueller H. On the theory of scattering of light. Proc Roy Soc A Math Phys Eng Sci 166, 425–449 (1938). [99] Liu F, Zhang SC, Han PL et al. Depolarization index from Mueller matrix descatters imaging in turbid water. Chin Opt Lett 20, 022601 (2022). doi: 10.3788/COL202220.022601 [100] Ortega-Quijano N, Haj-Ibrahim B, García-Caurel E et al. Experimental validation of Mueller matrix differential decomposition. Opt Express 20, 1151–1163 (2012). doi: 10.1364/OE.20.001151 [101] Lu SY, Chipman RA. Interpretation of Mueller matrices based on polar decomposition. J Opt Soc Am A 13, 1106–1113 (1996). doi: 10.1364/JOSAA.13.001106 [102] He HH, Zeng N, Du E et al. A possible quantitative Mueller matrix transformation technique for anisotropic scattering media/Eine mögliche quantitative Müller-Matrix-Transformations-Technik für anisotrope streuende Medien. Photonics Lasers Med 2, 129–137 (2013). [103] Arteaga O, Garcia-Caurel E, Ossikovski R. Anisotropy coefficients of a Mueller matrix. J Opt Soc Am A 28, 548–553 (2011). doi: 10.1364/JOSAA.28.000548 [104] Ortega-Quijano N, Arce-Diego JL. Mueller matrix differential decomposition. Opt Lett 36, 1942–1944 (2011). doi: 10.1364/OL.36.001942 [105] Ossikovski R. Analysis of depolarizing Mueller matrices through a symmetric decomposition. J Opt Soc Am A 26, 1109–1118 (2009). doi: 10.1364/JOSAA.26.001109 [106] Cloude SR. Group theory and polarisation algebra. Optik 75, 26–36 (1985). [107] Carnicer A, Javidi B. Polarimetric 3D integral imaging in photon-starved conditions. Opt Express 23, 6408–6417 (2015). doi: 10.1364/OE.23.006408 [108] Chen C, Chen QF, Xu J et al. Learning to see in the dark. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 3291–3300 (IEEE, 2018); https://doi.org/10.1109/CVPR.2018.00347. [109] Tibbs AB, Daly IM, Roberts NW et al. Denoising imaging polarimetry by adapted BM3D method. J Opt Soc Am A 35, 690–701 (2018). doi: 10.1364/JOSAA.35.000690 [110] Gao SK, Gruev V. Gradient-based interpolation method for division-of-focal-plane polarimeters. Opt Express 21, 1137–1151 (2013). doi: 10.1364/OE.21.001137 [111] Zhang JC, Luo HB, Hui B et al. Image interpolation for division of focal plane polarimeters with intensity correlation. Opt Express 24, 20799–20807 (2016). doi: 10.1364/OE.24.020799 [112] Zhang JC, Luo HB, Liang RG et al. Sparse representation-based demosaicing method for microgrid polarimeter imagery. Opt Lett 43, 3265–3268 (2018). doi: 10.1364/OL.43.003265 [113] Ratliff BM, LaCasse CF, Tyo JC. Interpolation strategies for reducing IFOV artifacts in microgrid polarimeter imagery. Opt Express 17, 9112–9125 (2009). doi: 10.1364/OE.17.009112 [114] Wen SJ, Zheng YQ, Lu F et al. Convolutional demosaicing network for joint chromatic and polarimetric imagery. Opt Lett 44, 5646–5649 (2019). doi: 10.1364/OL.44.005646 [115] Sargent GC, Ratliff BM, Asari VK. Conditional generative adversarial network demosaicing strategy for division of focal plane polarimeters. Opt Express 28, 38419–38443 (2020). doi: 10.1364/OE.412687 [116] Sun YY, Zhang JC, Liang RG. Color polarization demosaicking by a convolutional neural network. Opt Lett 46, 4338–4341 (2021). doi: 10.1364/OL.431919 [117] Pistellato M, Bergamasco F, Fatima T et al. Deep demosaicing for polarimetric filter array cameras. IEEE Trans Image Process 31, 2017–2026 (2022). doi: 10.1109/TIP.2022.3150296 [118] Zhang JC, Chen JL, Yu HW et al. Polarization image demosaicking via nonlocal sparse tensor factorization. IEEE Trans Geosci Remote Sens 60, 5607210 (2021). [119] Hagen N, Otani Y. Stokes polarimeter performance: general noise model and analysis. Appl Opt 57, 4283–4296 (2018). doi: 10.1364/AO.57.004283 [120] Li XB, Hu HF, Liu TG et al. Optimal distribution of integration time for intensity measurements in degree of linear polarization polarimetry. Opt Express 24, 7191–7200 (2016). doi: 10.1364/OE.24.007191 [121] Lewis JJ, O’Callaghan RJ, Nikolov SG et al. Pixel- and region-based image fusion with complex wavelets. Inf Fusion 8, 119–130 (2007). doi: 10.1016/j.inffus.2005.09.006 [122] Nencini F, Garzelli A, Baronti S et al. Remote sensing image fusion using the curvelet transform. Inf Fusion 8, 143–156 (2007). doi: 10.1016/j.inffus.2006.02.001 [123] Li ST, Kang XD, Hu JW. Image fusion with guided filtering. IEEE Trans Image Process 22, 2864–2875 (2013). doi: 10.1109/TIP.2013.2244222 [124] Liu Y, Liu SP, Wang ZF. A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24, 147–164 (2015). doi: 10.1016/j.inffus.2014.09.004 [125] Li ST, Kang XD, Fang LY et al. Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33, 100–112 (2017). doi: 10.1016/j.inffus.2016.05.004 [126] Zhang JC, Shao JB, Chen JL et al. PFNet: an unsupervised deep network for polarization image fusion. Opt Lett 45, 1507–1510 (2020). doi: 10.1364/OL.384189 [127] Zhang JC, Shao JB, Chen JL et al. Polarization image fusion with self-learned fusion strategy. Pattern Recognit 118, 108045 (2021). doi: 10.1016/j.patcog.2021.108045 [128] Lin TY, Dollár P, Girshick R et al. Feature pyramid networks for object detection. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition 936–944 (IEEE, 2017); http://doi.org/10.1109/CVPR.2017.106. [129] Song YB, Ma C, Gong LJ et al. CREST: convolutional residual learning for visual tracking. In Proceedings of 2017 IEEE International Conference on Computer Vision 2574–2583 (IEEE, 2017); https://doi.org/10.1109/iccv.2017.279. [130] Xu Y, Wen J, Fei LK, Zhang Z. Review of video and image defogging algorithms and related studies on image restoration and enhancement. IEEE Access 4, 165–188 (2015). [131] Guo Y, Liu RW, Lu YX et al. Haze visibility enhancement for promoting traffic situational awareness in vision-enabled intelligent transportation. IEEE Trans Veh Technol 72, 15421–15435 (2023). doi: 10.1109/TVT.2023.3298041 [132] Nguyen K, Nguyen P, Bui DC et al. Analysis of the influence of de-hazing methods on vehicle detection in aerial images. Int J Adv Comput Sci Appl 13, 846–856 (2022). [133] Liu JP, Wang SJ, Wang X et al. A review of remote sensing image dehazing. Sensors 21, 3926 (2021). doi: 10.3390/s21113926 [134] Karavarsamis S, Gkika I, Gkitsas V et al. A survey of deep learning-based image restoration methods for enhancing situational awareness at disaster sites: the cases of rain, snow and haze. Sensors 22, 4707 (2022). doi: 10.3390/s22134707 [135] Song YF, Nakath D, She MK et al. Optical imaging and image restoration techniques for deep ocean mapping: a comprehensive survey. PFG J Photogramm Remote Sens Geoinf Sci 90, 243–267 (2022). [136] Islam J, Xia YY, Sattar J. Fast underwater image enhancement for improved visual perception. IEEE Robot Autom Lett 5, 3227–3234 (2020). doi: 10.1109/LRA.2020.2974710 [137] Negi A, Chauhan P, Kumar K et al. Face mask detection classifier and model pruning with keras-surgeon. In Proceedings of the 2020 5th IEEE International Conference on Recent Advances and Innovations in Engineering 1–6 (IEEE, 2020); http://doi.org/10.1109/ICRAIE51050.2020.9358337. [138] Narasimhan SG, Nayar SK. Vision and the atmosphere. Int J Comput Vis 48, 233–254 (2002). doi: 10.1023/A:1016328200723 [139] Schechner YY, Narasimhan SG, Nayar SK. Polarization-based Vision through haze. Appl Opt 42, 511–525 (2003). doi: 10.1364/AO.42.000511 [140] Akkaynak D, Treibitz T. A revised underwater image formation model. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 6723–6732 (IEEE, 2018); http://doi.org/10.1109/CVPR.2018.00703. [141] Akkaynak D, Treibitz T. Sea-Thru: a method for removing water from underwater images. In Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition 1682–1691 (IEEE, 2019); http://doi.org/10.1109/CVPR.2019.00178. [142] Hu HF, Zhao L, Li XB et al. Underwater image recovery under the nonuniform optical field based on polarimetric imaging. IEEE Photonics J 10, 6900309 (2018). [143] Wei Y, Han PL, Liu F et al. Polarization descattering imaging: a solution for nonuniform polarization characteristics of a target surface. Chin Opt Lett 19, 111101 (2021). doi: 10.3788/COL202119.111101 [144] Fabbri C, Islam J, Sattar J. Enhancing underwater imagery using generative adversarial networks. In Proceedings of 2018 IEEE International Conference on Robotics and Automation 7159–7165 (IEEE, 2018); http://doi.org/10.1109/ICRA.2018.8460552. [145] Anwar S, Li CY, Porikli F. Deep underwater image enhancement, arXiv: 1807.03528 (2018). https://doi.org/10.48550/arXiv.1807.03528 [146] Wang N, Zhou YB, Han FL et al. UWGAN: underwater GAN for real-world underwater color restoration and dehazing. arXiv: 1912.10269 (2019). https://doi.org/10.48550/arXiv.1912.10269 [147] He KM, Sun J, Tang XO. Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33, 2341–2353 (2011). doi: 10.1109/TPAMI.2010.168 [148] Ren QM, Xiang YF, Wang GC et al. The underwater polarization dehazing imaging with a lightweight convolutional neural network. Optik 251, 168381 (2022). doi: 10.1016/j.ijleo.2021.168381 [149] Li BY, Peng XL, Wang ZY et al. AOD-Net: all-in-one dehazing network. In Proceedings of 2017 IEEE International Conference on Computer Vision 4780–4788 (IEEE, 2017); http://doi.org/10.1109/ICCV.2017.511. [150] Peng YT, Cao KM, Cosman PC. Generalization of the dark channel prior for single image restoration. IEEE Trans Image Process 27, 2856–2868 (2018). doi: 10.1109/TIP.2018.2813092 [151] Fu XP, Liang Z, Ding XY et al. Image descattering and absorption compensation in underwater polarimetric imaging. Opt Laser Eng 132, 106115 (2020). doi: 10.1016/j.optlaseng.2020.106115 [152] Drews PLJ, Nascimento ER, Botelho SSC et al. Underwater depth estimation and image restoration based on single images. IEEE Comput Graph Appl 36, 24–35 (2016). [153] Fu XY, Zhuang PX, Huang Y et al. A retinex-based enhancing approach for single underwater image. In Proceedings of 2014 IEEE International Conference on Image Processing 4572–4576 (IEEE, 2014); http://doi.org/10.1109/ICIP.2014.7025927. [154] Fu XY, Fan ZW, Ling M et al. Two-step approach for single underwater image enhancement. In Proceedings of 2017 International Symposium on Intelligent Signal Processing and Communication Systems 789–794 (IEEE, 2017); http://doi.org/10.1109/ISPACS.2017.8266583. [155] Li BY, Ren WQ, Fu DP et al. Benchmarking single-image dehazing and beyond. IEEE Trans Image Process 28, 492–505 (2019). doi: 10.1109/TIP.2018.2867951 [156] Sakaridis C, Dai DX, Van Gool L. Semantic foggy scene understanding with synthetic data. Int J Comput Vis 126, 973–992 (2018). doi: 10.1007/s11263-018-1072-8 [157] Sakaridis C, Dai DX, Van Gool L. Model adaptation with synthetic and real data for semantic dense foggy scene understanding. In Proceedings of the 15th European Conference on Computer Vision (Springer, 2018); https://doi.org/10.1007/978-3-030-01261-8_42. [158] Zhang YF, Ding L, Sharma G. HazeRD: an outdoor scene dataset and benchmark for single image dehazing. In Proceedings of 2017 IEEE International Conference on Image Processing 3205–3209 (IEEE, 2017); http://doi.org/10.1109/ICIP.2017.8296874. [159] Zhao SY, Zhang L, Huang SY et al. Dehazing evaluation: real-world benchmark datasets, criteria, and baselines. IEEE Trans Image Process 29, 6947–6962 (2020). doi: 10.1109/TIP.2020.2995264 [160] Carlevaris-Bianco N, Mohan A, Eustice RM. Initial results in underwater single image dehazing. In Proceedings of OCEANS 2010 MTS/IEEE SEATTLE 1-8 (IEEE, 2010); http://doi.org/10.1109/OCEANS.2010.5664428. [161] Huang DM, Wang Y, Song W et al. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In Proceedings of the 24th International Conference on Multimedia Modeling 453–465 (Springer, 2018); https://doi.org/10.1007/978-3-319-73603-7_37. [162] Huang SC, Cheng FC, Chiu YS. Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans Image Process 22, 1032–1041 (2013). doi: 10.1109/TIP.2012.2226047 [163] Chao L, Wang M. Removal of water scattering. In Proceedings of the 2010 2nd International Conference on Computer Engineering and Technology V2-35–V2-39 (IEEE, 2010); https://doi.org/10.1109/ICCET.2010.5485339. [164] Liang J, Ren LY, Ju HJ et al. Visibility enhancement of hazy images based on a universal polarimetric imaging method. J Appl Phys 116, 173107 (2014). doi: 10.1063/1.4901244 [165] Hitam MS, Awalludin EA, Yussof WNJHW et al. Mixture contrast limited adaptive histogram equalization for underwater image enhancement. In Proceedings of 2013 International Conference on Computer Applications Technology 1–5 (IEEE, 2013); http://doi.org/10.1109/ICCAT.2013.6522017. [166] Fan R, Wang HL, Cai PD et al. SNE-RoadSeg: incorporating surface normal information into semantic segmentation for accurate freespace detection. In Proceedings of the 16th European Conference on Computer Vision 340–356 (Springer, 2020); https://doi.org/10.1007/978-3-030-58577-8_21. [167] Kazhdan M, Bolitho M, Hoppe H. Poisson surface reconstruction. In Proceedings of the Fourth Eurographics Symposium on Geometry ProcessING 61–70 (2006); https://dl.acm.org/doi/10.5555/1281957.1281965. [168] Qi XJ, Liao RJ, Liu ZZ et al. Geonet: geometric neural network for joint depth and surface normal estimation. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 283–291 (IEEE, 2018); http://doi.org/10.1109/CVPR.2018.00037. [169] Huang JW, Zhou YC, Funkhouser T, Guibas L. FrameNet: learning local canonical frames of 3D surfaces from a single RGB image. In Proceedings of 2019 IEEE/CVF International Conference on Computer Vision 8637–8646 (IEEE, 2019); http://doi.org/10.1109/ICCV.2019.00873. [170] Wang R, Geraghty D, Matzen K et al. VPLNet: deep single view normal estimation with vanishing points and lines. In Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition 686–695 (IEEE, 2020); http://doi.org/10.1109/CVPR42600.2020.00077. [171] Bansal A, Chen XL, Russell B et al. PixelNet: towards a general pixel-level architecture. arXiv: 1609.06694 (2016). https://arxiv.org/abs/1609.06694 [172] Li B, Shen CH, Dai YC et al. Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs. In Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition 1119–1127 (IEEE, 2015); http://doi.org/10.1109/CVPR.2015.7298715. [173] Atkinson GA. Polarisation photometric stereo. Comput Vis Image Underst 160, 158–167 (2017). doi: 10.1016/j.cviu.2017.04.014 [174] Fukao Y, Kawahara R, Nobuhara S et al. Polarimetric normal stereo. In Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition 682–690 (IEEE, 2021); http://doi.org/10.1109/CVPR46437.2021.00074 [175] Zhu DZ, Smith WAP. Depth from a polarisation + RGB stereo pair. In Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition 7578–7587 (IEEE, 2019); http://doi.org/10.1109/CVPR.2019.00777. [176] Ding YQ, Ji Y, Zhou MY et al. Polarimetric helmholtz stereopsis. In Proceedings of 2021 IEEE/CVF International Conference on Computer Vision 5017–5026 (IEEE, 2021); http://doi.org/10.1109/ICCV48922.2021.00499. [177] Atkinson GA, Hancock ER. Recovery of surface orientation from diffuse polarization. IEEE Trans Image Process 15, 1653–1664 (2006). doi: 10.1109/TIP.2006.871114 [178] Robles-Kelly A, Huynh CP. Imaging Spectroscopy for Scene Analysis (Springer, London, 2013). [179] Frankot RT, Chellappa R. A method for enforcing integrability in shape from shading algorithms. IEEE Trans Pattern Anal Mach Intell 10, 439–451 (1988). doi: 10.1109/34.3909 [180] Miyazaki D, Tan RT, Hara K et al. Polarization-based inverse rendering from a single view. In Proceedings of the Ninth IEEE International Conference on Computer Vision 982–987 (IEEE, 2003); https://doi.org/10.1109/ICCV.2003.1238455. [181] Mahmoud AH, El-Melegy MT, Farag AA. Direct method for shape recovery from polarization and shading. In Proceedings of the 2012 19th IEEE International Conference on Image Processing 1769–1772 (IEEE, 2012); https://doi.org/10.1109/ICIP.2012.6467223. [182] Loper M, Mahmood N, Romero J et al. SMPL: a skinned multi-person linear model. ACM Trans Graph 34, 248 (2015). [183] Han PL, Li X, Liu F et al. Accurate passive 3D polarization face reconstruction under complex conditions assisted with deep learning. Photonics 9, 924 (2022). doi: 10.3390/photonics9120924 [184] Levin A, Weiss Y. User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans Pattern Anal Mach Intell 29, 1647–1654 (2007). doi: 10.1109/TPAMI.2007.1106 [185] Li Y, Brown MS. Exploiting reflection change for automatic reflection removal. In Proceedings of 2013 IEEE International Conference on Computer Vision 2432–2439 (IEEE, 2013); https://doi.org/10.1109/ICCV.2013.302. [186] Zhou BL, Lapedriza A, Khosla A et al. Places: a 10 million image database for scene recognition. IEEE Trans Pattern Anal Mach Intell 40, 1452–1464 (2018). doi: 10.1109/TPAMI.2017.2723009 [187] Pang YX, Yuan MK, Fu Q et al. Progressive polarization based reflection removal via realistic training data generation. Pattern Recognit 124, 108497 (2022). doi: 10.1016/j.patcog.2021.108497 [188] Girshick R, Donahue J, Darrell T et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition 580–587 (IEEE, 2014); https://doi.org/10.1109/CVPR.2014.81. [189] Girshick R. Fast R-CNN. In Proceedings of 2015 IEEE International Conference on Computer Vision 1440–1448 (IEEE, 2015); https://doi.org/10.1109/ICCV.2015.169. [190] Ren SQ, He KM, Girshick R et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39, 1137–1149 (2017). doi: 10.1109/TPAMI.2016.2577031 [191] Redmon J, Divvala S, Girshick R et al. You only look once: unified, real-time object detection. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition 779–788 (IEEE, 2016); https://doi.org/10.1109/CVPR.2016.91. [192] Liu W, Anguelov D, Erhan D et al. SSD: single shot multibox detector. In Proceedings of the 14th European Conference on Computer Vision 21–37 (Springer, 2016); https://doi.org/10.1007/978-3-319-46448-0_2. [193] Law H, Deng J. CornerNet: detecting objects as paired keypoints. In Proceedings of the 15th European Conference on Computer Vision 765–781 (Springer 2018); https://doi.org/10.1007/978-3-030-01264-9_45 [194] Wolff LB. Polarization-based material classification from specular reflection. IEEE Trans Pattern Anal Mach Intell 12, 1059–1071 (1990). doi: 10.1109/34.61705 [195] Wolff LB. Surface orientation from polarization images. Proc SPIE 850, 110–121 (1988). doi: 10.1117/12.942866 [196] Tian Y, Zhang KB, Wang LY et al. Face anti-spoofing by learning polarization cues in a real-world scenario. In Proceedings of the 4th International Conference on Advances in Image Processing 129–137 (ACM, 2020); https://doi.org/10.1145/3441250.3441254. [197] Usmani K, Krishnan G, O'Connor T et al. Deep learning polarimetric three-dimensional integral imaging object recognition in adverse environmental conditions. Opt Express 29, 12215–12228 (2021). doi: 10.1364/OE.421287 [198] Shen Y, Lin WF, Wang ZF et al. Rapid detection of camouflaged artificial target based on polarization imaging and deep learning. IEEE Photonics J 13, 7800309 (2021). [199] Liu T, Lu M, Chen BG et al. Distinguishing structural features between Crohn’s disease and gastrointestinal luminal tuberculosis using Mueller matrix derived parameters. J. Biophotonics 12, e201900151 (2019). doi: 10.1002/jbio.201900151 [200] Shen YX, Huang RR, He HH et al. Comparative study of the influence of imaging resolution on linear retardance parameters derived from the Mueller matrix. Biomed Opt Express 12, 211–225 (2021). doi: 10.1364/BOE.410989 [201] Sun T, Liu T, He HH et al. Distinguishing anisotropy orientations originated from scattering and birefringence of turbid media using Mueller matrix derived parameters. Opt Lett 43, 4092–4095 (2018). doi: 10.1364/OL.43.004092 [202] Rehbinder J, Haddad H, Deby S et al. Ex vivo Mueller polarimetric imaging of the uterine cervix: a first statistical evaluation. J Biomed Opt 21, 071113 (2016). doi: 10.1117/1.JBO.21.7.071113 [203] Chue-Sang J, Bai YQ, Stoff S et al. Use of combined polarization-sensitive optical coherence tomography and Mueller matrix imaging for the polarimetric characterization of excised biological tissue. J Biomed Opt 21, 071109 (2016). doi: 10.1117/1.JBO.21.7.071109 [204] Novikova T, Pierangelo A, Manhas S et al. The origins of polarimetric image contrast between healthy and cancerous human colon tissue. Appl Phys Lett 102, 241103 (2013). doi: 10.1063/1.4811414 [205] Pierangelo A, Manhas S, Benali A et al. Ex vivo photometric and polarimetric multilayer characterization of human healthy colon by multispectral Mueller imaging. J Biomed Opt 17, 066009 (2012). doi: 10.1117/1.JBO.17.6.066009 [206] Dubreuil M, Babilotte P, Martin L et al. Mueller matrix polarimetry for improved liver fibrosis diagnosis. Opt Lett 37, 1061–1063 (2012). doi: 10.1364/OL.37.001061 [207] Wang WF, Lim LG, Srivastava S et al. Roles of linear and circular polarization properties and effect of wavelength choice on differentiation between ex vivo normal and cancerous gastric samples. J Biomed Opt 19, 046020 (2014). doi: 10.1117/1.JBO.19.4.046020 [208] Chen GH, Wang JY, Zhang AJ. Transparent object detection and location based on RGB-D camera. J Phys Conf Ser 1183, 012011 (2019). doi: 10.1088/1742-6596/1183/1/012011 [209] Liu YJ, Jourabloo A, Liu XM. Learning deep models for face anti-spoofing: binary or auxiliary supervision. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 389–398 (IEEE, 2018); https://doi.org/10.1109/CVPR.2018.00048. [210] Wang T, He XM, Barnes N. Glass object localization by joint inference of boundary and depth. In Proceedings of the 21st International Conference on Pattern Recognition 3783–3786 (IEEE, 2012). [211] Xu YC, Nagahara H, Shimada A et al. TransCut: transparent object segmentation from a light-field image. In Proceedings of 2015 IEEE International Conference on Computer Vision 3442–3450 (IEEE, 2015); https://doi.org/10.1109/ICCV.2015.393. [212] Zhang SF, Wang XB, Liu AJ et al. A dataset and benchmark for large-scale multi-modal face anti-spoofing. In Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition 919–928 (IEEE, 2019); https://doi.org/10.1109/CVPR.2019.00101. [213] Sun MH, He HH, Zeng N et al. Characterizing the microstructures of biological tissues using Mueller matrix and transformed polarization parameters. Biomed Opt Express 5, 4223–4234 (2014). doi: 10.1364/BOE.5.004223 [214] Li PC, Lv DH, He HH et al. Separating azimuthal orientation dependence in polarization measurements of anisotropic media. Opt Express 26, 3791–3800 (2018). doi: 10.1364/OE.26.003791 [215] Gil JJ. Invariant quantities of a Mueller matrix under rotation and retarder transformations. J Opt Soc Am A 33, 52–58 (2016). doi: 10.1364/JOSAA.33.000052 [216] Le QV. Building high-level features using large scale unsupervised learning. In Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing 8595–8598 (IEEE, 2013); https://doi.org/10.1109/ICASSP.2013.6639343. [217] Dy JG, Brodley CE. Feature selection for unsupervised learning. J Mach Learn Res 5, 845–889 (2004). [218] Dike HU, Zhou YM, Deveerasetty KK et al. Unsupervised learning based on artificial neural network: a review. In Proceedings of 2018 IEEE International Conference on Cyborg and Bionic Systems 322–327 (IEEE, 2018); https://doi.org/10.1109/CBS.2018.8612259. [219] Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng 22, 1345–1359 (2010). doi: 10.1109/TKDE.2009.191 [220] Pan SJ, Tsang IW, Kwok JT et al. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22, 199–210 (2011). doi: 10.1109/TNN.2010.2091281 [221] Stevens SY, Delgado C, Krajcik JS. Developing a hypothetical multi-dimensional learning progression for the nature of matter. J Res Sci Teach 47, 687–715 (2010). doi: 10.1002/tea.20324 [222] Xin M, Kundu S. Multi-task learning with high-dimensional noisy images. J Am Stat Assoc (2021). doi: 10.1080/01621459.2022.2140052. [223] Li T, Sahu AK, Talwalkar A et al. Federated learning: challenges, methods, and future directions. IEEE Signal Process Mag 37, 50–60 (2020). [224] Rubin NA, D’Aversa G, Chevalier P et al. Matrix Fourier optics enables a compact full-Stokes polarization camera. Science 365, eaax1839 (2019). doi: 10.1126/science.aax1839 [225] Zhang YX, Pu MB, Jin JJ et al. Crosstalk-free achromatic full Stokes imaging polarimetry metasurface enabled by polarization-dependent phase optimization. Opto-Electron Adv 5, 220058 (2022). doi: 10.29026/oea.2022.220058 [226] Tang DL, Shao ZL, Xie X et al. Flat multifunctional liquid crystal elements through multi-dimensional information multiplexing. Opto-Electron Adv 6, 220063 (2023). doi: 10.29026/oea.2023.220063 [227] Chen J, Wang DP, Si GY et al. Planar peristrophic multiplexing metasurfaces. Opto-Electron Adv 6, 220141 (2023). doi: 10.29026/oea.2023.220141 -
Access History
Figures(25)
Tables(6)
Article Metrics
Export File
Citation
Yang K, Liu F, Liang SY et al. Data-driven polarimetric imaging: a review. Opto-Electron Sci 3, 230042 (2024). doi: 10.29026/oes.2024.230042
Format
Content
DownLoad:
-
Figure 1.
Schematic of short history about data-driven polarimetric imaging.
-
Figure 2.
Schematics of the trends of existing data-driven polarimetric imaging. The solid black arrows refer to the data flow, the dotted black arrows refer to that the data may or not flow. The gradient blue arrows mean the trends of corresponding aspects. (a) The utilization of polarimetric information has been gradually deepened from raw polarimetric images to preprocessing features. (b) The physical modes have been crucial during the training of network from end-to-end architecture to physical-model-combined structure. (c) The various physical properties have been gradually fused to network. (d) The tasks have been developed from image processing to semantic tasks.
-
Figure 3.
Applications of data-driven polarimetric imaging.
-
Figure 4.
The considered tasks of restoration and enhancement of polarization information, including low-light imaging, Muller matrix denoising, high dynamic range (HDR) reconstruction, polarimetric parameters denoising, demosaicing, transformation between holographic amplitude & phase and polarization channels, transformation between Stokes vectors and MMPD images.
-
Figure 5.
Enhanced method of polarization information. (a) Architecture of the Hu et al.’s method. (b) Enhanced results and corresponding comparisons with mainstream methods. Figure reproduced with permission from ref.42, Optical Society of America Publishing AG.
-
Figure 6.
The convolutional demosaicing network proposed by Zhang et al. and results68. (a) Architecture of the Zhang et al. method. (b) Reconstructed DoLP images of different methods on the test images. (b1) The results of method113. (b2) The results of method110. (b3) The results of method111. (b4) The results of method112. (b5) and (b6) are respectively PDCNN and ground truth. (c) Reconstructed AoP images of different methods on the test images. (c5) and (c6) are respectively PDCNN and ground truth. Figure reproduced with permission from ref.68, Optical Society of America Publishing AG.
-
Figure 7.
Liu et al. method. (a) Architecture of the proposed method. (b) Enhanced results and corresponding comparisons with mainstream methods. Figure reproduced with permission from ref.77, Optical Society of America Publishing AG.
-
Figure 8.
The unsupervised deep network PFNet proposed by Zhang et al.126 (a) The architecture of PFNet. (b) Fused results compared with conventional methods. Figure reproduced with permission from ref.126, Optical Society of America Publishing AG.
-
Figure 9.
The flow chart and results of the Hu et al. method, which is a typical end-to-end descattering architecture46. (a) The architecture of proposed method. (b) Comparison of the enhanced images by different methods. Figure reproduced with permission from ref.46, Elsevier BV.
-
Figure 10.
The methods combining the physical formation model with deep learning methods and corresponding examples. (a1) Zhou et al. method and (a2) the corresponding example. (b1) Shi et al. method and (b2) the corresponding examples.
-
Figure 11.
Unsupervised underwater descattering method. (a) Architecture of Zhu et al. method49 and (b) visual comparison among different de-scattering methods. Figure reproduced with permission from ref.49, Optical Society of America Publishing AG.
-
Figure 12.
Unsupervised underwater descattering methods and corresponding results. For the equations and variables, see the ref.76. (a) The flow chart. (b) Proposed architecture of Yang et al. method. (c) The results of generated backscattering light; (c1) the captured polarized image; (c2) the generated backscattering image of proposed trained network; (c3) the backscattering image after smooth filtering; (c4) the corresponding ground truth. (d) Final descattering results and comparisons with other methods. Figure reproduced with permission from ref.76, Optical Society of America Publishing AG.
-
Figure 13.
Polarization of specular reflection and diffuse reflection. (a) Specular reflection. (b) Diffuse reflection.
- Figure 14.
-
Figure 15.
Han et al. passive 3D polarization face reconstruction method. (a) Overall schematic of the proposed method. (b) 3D polarization face reconstruction results. Figure reproduced with permission from ref.183, Multidisciplinary Digital Publishing Institute.
-
Figure 16.
The physical-based prior confidence map according to the differences of polarization characteristics between transparent object and background.
-
Figure 17.
The 3D shape reconstruction methods based on the pBRDF. (a) Y Kondo et al. method and the reconstruction examples. (b) V Deschaintre et al. method and corresponding results.
-
Figure 18.
The inputs of network in 3D shape reconstruction task. (a) The raw polarization images as well as the diffuse and specular normal prior. (b) The raw polarization images and two diffuse ambiguous normal. (c) The DoFP image consists of four polarization sub-images, viewing encoding, intensity, DoP image and encoded AoP image. (d) The chromatic intensity, phase and DoP image. (e) The raw polarization images, flash image, normalized color and Stokes map. (f) The maximum and minimum polarization images. (g) The raw polarization images, physics-based prior confidence, DoLP and AoLP.
-
Figure 19.
The typical data-driven polarimetric methods of reflection removal. (a1) P Wieschollek et al. method. (a2) The image-based data generation procedure of (a1). (b) Y Lyu et al. method.
-
Figure 20.
Shen et al. method. (a) The flow chart of proposed method. (b) The detected results. Figure reproduced with permission from ref.198, Institute of Electrical and Electronics Engineers.
-
Figure 21.
The architecture and results of multi-parameters fusion network (MPFN) and the corresponding results.
-
Figure 22.
Polarization-imaging-based ML framework for quantitative diagnosis of cervical precancerous lesions. Figure reproduced with permission from ref.174, Institute of Electrical and Electronics Engineers.
-
Figure 23.
Yang et al. method: proposed architecture and produced results of depth and segmented results.
-
Figure 24.
Transparent object segmentation. (a) The designed architecture. (b) The segmented results of intensity Mask R-CNN and polarized Mask R-CNN in several dataset.
-
Figure 25.
Input and utilization of polarization information. (a) Original polarization images (OPI). (b) Polarimetric parameter feature maps (PPFM). (c) Associated parameters maps (APM).
