volume | PIER Journals

Abstract

The black-box nature of machine learning (ML) models limits their adoption in industrial terahertz (THz) inspection systems where decision transparency is mandatory. We present a SHAP-based explainability framework for THz classification of five insulating materials using a 636-dimensional feature set (625 spectral channels plus 11 morphological descriptors) benchmarked across 11 classifiers. LightGBM achieved the highest performance (Macro F1 = 86.84%, Accuracy = 87.18%) and was selected for in-depth SHAP analysis and four validation experiments. SHAP-guided feature selection showed that 50 features (7.9% of the space) already match the full baseline and are the smallest tested subset that does so. Cross-model analysis showed Gradient_Std to be the single highest-ranked SHAP feature for LightGBM, XGBoost, and Random Forest alike, confirming its discriminative role to be a property of the data rather than of a single learner. Four SHAP-identified spectral windows (113 channels, 18.1%) combined with morphological features retained 95.7% of full-spectrum performance (F1 = 83.07%). A noise robustness sweep from 60 dB to 5 dB SNR, assessed against an explicit reliability criterion (Macro F1≥95% of each subset's own noise-free value), places the minimum usable SNR at 30 dB for the full feature set, shows that the morphological descriptors degrade first (falling below their criterion at 50 dB when used alone), and supports a 40 dB design margin. Engineered morphological descriptors contributed 39.8× more per feature than raw spectral channels. These results provide actionable guidance for compact narrowband sensor design and edge deployment in industrial insulation inspection.

1. Tonouchi, Masayoshi, "Cutting-edge terahertz technology," Nature Photonics, Vol. 1, No. 2, 97-105, February 2007.
doi:10.1038/nphoton.2007.3 Google Scholar

2. Jepsen, P. U., D. G. Cooke, and M. Koch, "Terahertz spectroscopy and imaging --- Modern techniques and applications," Laser & Photonics Reviews, Vol. 5, No. 1, 124-166, October 2010.
doi:10.1002/lpor.201000011 Google Scholar

3. Shen, Yao-Chun, "Terahertz pulsed spectroscopy and imaging for pharmaceutical applications: A review," International Journal of Pharmaceutics, Vol. 417, No. 1-2, 48-60, September 2011.
doi:10.1016/j.ijpharm.2011.01.012 Google Scholar

4. Mei, Hongwei, Lanxin Li, Fanghui Yin, Wei Hu, Jianben Liu, and Liming Wang, "Study on detecting dielectric properties of typical electrical insulation materials by terahertz wave spectroscopy," IEEE Access, Vol. 8, 226887-226896, 2020.
doi:10.1109/access.2019.2941798 Google Scholar

5. Guo, Chenjun, Wenlong Xu, Mingming Cai, Shufan Duan, Jianbiao Fu, and Xinlong Zhang, "A review: Application of terahertz nondestructive testing technology in electrical insulation materials," IEEE Access, Vol. 10, 121547-121560, 2022.
doi:10.1109/access.2022.3222860 Google Scholar

6. Bauer, Maris, et al. "Fast FMCW terahertz imaging for in-process defect detection in press sleeves for the paper industry and image evaluation with a machine learning approach," Sensors, Vol. 21, No. 19, 6569, September 2021.
doi:10.3390/s21196569 Google Scholar

7. Jiang, Yuying, Guangming Li, Hongyi Ge, Faye Wang, Li Li, Xinyu Chen, Ming Lu, and Yuan Zhang, "Machine learning and application in terahertz technology: A review on achievements and future challenges," IEEE Access, Vol. 10, 53761-53776, 2022.
doi:10.1109/access.2022.3174595 Google Scholar

8. Cielecki, Paweł Piotr, Michel Hardenberg, Georgiana Amariei, Martin Lahn Henriksen, Mogens Hinge, and Pernille Klarskov, "Identification of black plastics with terahertz time-domain spectroscopy and machine learning," Scientific Reports, Vol. 13, No. 1, 22399, December 2023.
doi:10.1038/s41598-023-49765-z Google Scholar

9. Arrieta, Alejandro Barredo, et al. "Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI," Information Fusion, Vol. 58, 82-115, June 2020.
doi:10.1016/j.inffus.2019.12.012 Google Scholar

10. Lundberg, S. M. and S.-I. Lee, "A unified approach to interpreting model predictions," Advances in Neural Information Processing Systems, Vol. 30, 2017.

11. Lundberg, Scott M., et al. "From local explanations to global understanding with explainable AI for trees," Nature Machine Intelligence, Vol. 2, No. 1, 56-67, January 2020.
doi:10.1038/s42256-019-0138-9 Google Scholar

12. Lundberg, Scott M., et al. "Explainable machine-learning predictions for the prevention of hypoxaemia during surgery," Nature Biomedical Engineering, Vol. 2, No. 10, 749-760, October 2018.
doi:10.1038/s41551-018-0304-0 Google Scholar

13. Elkhawaga, Ghada, Omar M. Elzeki, Mervat Abu-Elkheir, and Manfred Reichert, "Why should i trust your explanation? An evaluation approach for xai methods applied to predictive process monitoring results," IEEE Transactions on Artificial Intelligence, Vol. 5, No. 4, 1458-1472, April 2024.
doi:10.1109/tai.2024.3357041 Google Scholar

14. Sarjaš, Andrej, Blaž Pongrac, and Dušan Gleich, "Automated inorganic pigment classification in plastic material using terahertz spectroscopy," Sensors, Vol. 21, No. 14, 4709, July 2021.
doi:10.3390/s21144709 Google Scholar

15. Kristensen, Mathias Hedegaard, Paweł Piotr Cielecki, and Esben Skovsen, "High-accuracy material classification via reference-free terahertz spectroscopy: Revisiting spectral referencing and feature selection," Sensors and Actuators A: Physical, Vol. 402, 117648, May 2026.
doi:10.1016/j.sna.2026.117648 Google Scholar

16. Cacciari, Ilaria and Anedio Ranfagni, "Unlocking Terahertz technology with machine learning: A comprehensive review," Journal of Applied Physics, Vol. 137, No. 22, 220701, June 2025.
doi:10.1063/5.0272428 Google Scholar

17. Vera, E. R., J. Montoya-Cardona, V. G. Pineda, A. Valencia-Arias, N. Gomez-Cardona, F. Lopez-Giraldo, and E. Gonzalez-Valencia, "Advances in THz spectroscopy: A systematic review of material characterization applications," ASEAN Journal of Science and Engineering, Vol. 6, No. 2, 173-208, 2026. Google Scholar

18. Yu, Pengpeng, Chaoping Shen, Wenhui Zhu, Wenya Zhang, Junhui Cheng, and Jinxiu Song, "Recent developments and applications of terahertz spectroscopy in food analysis," Biosensors, Vol. 15, No. 10, 677, October 2025.
doi:10.3390/bios15100677 Google Scholar

19. Nsengiyumva, Walter, Shuncong Zhong, Longhui Zheng, Wei Liang, Bing Wang, Yi Huang, Xuefeng Chen, and Yaochun Shen, "Sensing and nondestructive testing applications of terahertz spectroscopy and imaging systems: State-of-the-art and state-of-the-practice," IEEE Transactions on Instrumentation and Measurement, Vol. 72, 1-83, 2023.
doi:10.1109/tim.2023.3318676 Google Scholar

20. Kristensen, Mathias Hedegaard, Paweł Piotr Cielecki, and Esben Skovsen, "Classification of non-referenced continuous-wave terahertz reflection spectra for remote material identification," Infrared Physics & Technology, Vol. 140, 105420, August 2024.
doi:10.1016/j.infrared.2024.105420 Google Scholar

21. Ke, G., Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, "LightGBM: A highly efficient gradient boosting decision tree," Advances in Neural Information Processing Systems, Vol. 30, 2017.

22. Chen, Tianqi and Carlos Guestrin, "XGBoost: A scalable tree boosting system," Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794, August 2016.
doi:10.1145/2939672.2939785

23. Prokhorenkova, L., G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, "CatBoost: Unbiased boosting with categorical features," Advances in Neural Information Processing Systems, Vol. 31, 2018.

24. Geurts, Pierre, Damien Ernst, and Louis Wehenkel, "Extremely randomized trees," Machine Learning, Vol. 63, No. 1, 3-42, March 2006.
doi:10.1007/s10994-006-6226-1 Google Scholar

25. Breiman, Leo, "Random forests," Machine Learning, Vol. 45, No. 1, 5-32, October 2001.
doi:10.1023/a:1010933404324 Google Scholar

26. Freund, Yoav and Robert E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," Journal of Computer and System Sciences, Vol. 55, No. 1, 119-139, August 1997.
doi:10.1006/jcss.1997.1504 Google Scholar

27. Cortes, Corinna and Vladimir Vapnik, "Support-vector networks," Machine Learning, Vol. 20, No. 3, 273-297, September 1995.
doi:10.1007/bf00994018 Google Scholar

28. Cox, D. R., "The regression analysis of binary sequences," Journal of the Royal Statistical Society Series B: Statistical Methodology, Vol. 20, No. 2, 215-232, July 1958.
doi:10.1111/j.2517-6161.1958.tb00292.x Google Scholar

29. Fisher, R. A., "The use of multiple measurements in taxonomic problems," Annals of Eugenics, Vol. 7, No. 2, 179-188, September 1936.
doi:10.1111/j.1469-1809.1936.tb02137.x Google Scholar

30. Rumelhart, David E., Geoffrey E. Hinton, and Ronald J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, No. 6088, 533-536, October 1986.
doi:10.1038/323533a0 Google Scholar

31. Zhang, Harry, "The optimality of naive bayes," Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2004), Vol. 1, No. 2, Florida, USA, 2004.

32. Tasumi, M. and S. Krimm, "Crystal vibrations of polyethylene," The Journal of Chemical Physics, Vol. 46, No. 2, 755-766, January 1967.
doi:10.1063/1.1840736 Google Scholar

33. Marchessault, R. H. and C. Y. Liang, "Infrared spectra of crystalline polysaccharides. III. Mercerized cellulose," Journal of Polymer Science, Vol. 43, No. 141, 71-84, March 1960.
doi:10.1002/pol.1960.1204314107 Google Scholar

34. Cataldo, Franco, Giancarlo Angelini, D. Aníbal García-Hernández, and Arturo Manchado, "Far infrared (terahertz) spectroscopy of a series of polycyclic aromatic hydrocarbons and application to structure interpretation of asphaltenes and related compounds," Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, Vol. 111, 68-79, July 2013.
doi:10.1016/j.saa.2013.03.077 Google Scholar

35. Mori, Keigo, Tomofumi Seki, Naoshi Hirai, and Yoshimichi Ohki, "Terahertz absorption spectra of several polymer nanocomposites indicating polymer-filler interactions," AIP Advances, Vol. 9, No. 10, 105109, October 2019.
doi:10.1063/1.5117182 Google Scholar

36. Li, Jiawei, Cong Zhang, Xiaohui Wang, Jinjing Zhang, Hanwen Liu, and Xu Wu, "Terahertz spectroscopy and density functional theory for non-destructive analysis of anticoagulant warfarin," Molecules, Vol. 30, No. 8, 1791, April 2025.
doi:10.3390/molecules30081791 Google Scholar

37. Chen, Quansheng, Jiewen Zhao, and Saritporn Vittayapadung, "Identification of the green tea grade level using electronic tongue and pattern recognition," Food Research International, Vol. 41, No. 5, 500-504, January 2008.
doi:10.1016/j.foodres.2008.03.005 Google Scholar

Dr. Lifeng Zhu

Dr. Jing Shen

Dr. Lei Zhang

Dr. Yongbin Sun

Dr. Xiaofeng Zhang

Dr. Luoyi Li

Dr. Xianliang Li

Dr. Chong Tian

Dr. Zhangchong Ye

Dr. Jie Cai

Dr. Qingqing Ge

Dr. Jiangyang Zhan