volume | PIER Journals

Abstract

This study addresses the challenge of detecting Autism Spectrum Disorder (ASD) in children, where clinical diagnostic scales used in practice suffer from subjectivity and high costs. Eye tracking (ET), as a non-contact sensing technology, offers the potential for objective ASD recognition. However, existing studies often use specially crafted visual stimuli, making them less reproducible, or rely on the construction of handcrafted features. Deep learning methods allow us to build more efficient models, but only a few studies simultaneously focused on visual behaviors of ASD in both temporal and spatial dimensions, and many studies compressed the temporal dimension, potentially losing valuable information. To address these limitations, this study employed a relatively lenient visual stimulus selection criterion to collect ET data of ASD in social scenes, enabling analyses to be conducted both temporally and spatially. Findings indicate that the spatial attention distribution of ASD is more dispersed, and gaze trajectories are more unstable in the temporal dimension. We also observed that children with ASD exhibit slower responses in gaze-following scenarios. Additionally, data loss emerges as an effective feature for ASD identification. We proposed an SP-Inception-Transformer network based on CNN and Transformer encoder architecture, which can simultaneously learn temporal and spatial features. It utilized raw eye-tracking data to prevent information loss, and employed Inception and Embedding to enhance the performance. Compared to benchmark methods, our model demonstrated superior results in accuracy (0.886), AUC (0.8972), recall (0.82), precision (0.95), and F1 score (0.8719).

1. Wei, Qiuhong, Huiling Cao, Yuan Shi, Ximing Xu, and Tingyu Li, "Machine learning based on eye-tracking data to identify Autism Spectrum Disorder: A systematic review and meta-analysis," Journal of Biomedical Informatics, Vol. 137, 104254, 2023.
doi:10.1016/j.jbi.2022.104254

2. Zeidan, Jinan, Eric Fombonne, Julie Scorah, Alaa Ibrahim, Maureen S. Durkin, Shekhar Saxena, Afiqah Yusuf, Andy Shih, and Mayada Elsabbagh, "Global prevalence of autism: A systematic review update," Autism Research, Vol. 15, No. 5, 778-790, 2022.
doi:10.1002/aur.2696

3. Jones, Warren, Cheryl Klaiman, Shana Richardson, Meena Lambha, Morganne Reid, Taralee Hamner, Chloe Beacham, Peter Lewis, Jose Paredes, Laura Edwards, et al. "Development and replication of objective measurements of social visual engagement to aid in early diagnosis and assessment of autism," JAMA Network Open, Vol. 6, No. 9, e2330145, 2023.
doi:10.1001/jamanetworkopen.2023.30145

4. Jones, Warren, Cheryl Klaiman, Shana Richardson, Christa Aoki, Christopher Smith, Mendy Minjarez, Raphael Bernier, Ernest Pedapati, Somer Bishop, Whitney Ence, et al. "Eye-tracking-based measurement of social visual engagement compared with expert clinical diagnosis of autism," JAMA, Vol. 330, No. 9, 854-865, 2023.
doi:10.1001/jama.2023.13295

5. Falck-Ytter, Terje, Sven Bölte, and Gustaf Gredebäck, "Eye tracking in early autism research," Journal of Neurodevelopmental Disorders, Vol. 5, No. 1, 28, 2013.
doi:10.1186/1866-1955-5-28

6. Wan, Guobin, Xuejun Kong, Binbin Sun, Siyi Yu, Yiheng Tu, Joel Park, Courtney Lang, Madelyn Koh, Zhen Wei, Zhe Feng, Yan Lin, and Jian Kong, "Applying eye tracking to identify Autism Spectrum Disorder in children," Journal of Autism and Developmental Disorders, Vol. 49, No. 1, 209-215, 2019.
doi:10.1007/s10803-018-3690-y

7. Fang, Yi, Huiyu Duan, Fangyu Shi, Xiongkuo Min, and Guangtao Zhai, "Identifying children with Autism Spectrum Disorder based on gaze-following," 2020 IEEE International Conference on Image Processing (ICIP), 423-427, Abu Dhabi, United Arab Emirates, 2020.
doi:10.1109/icip40778.2020.9190831

8. Fujioka, Toru, Keisuke Inohara, Yuko Okamoto, Yasuhiro Masuya, Makoto Ishitobi, Daisuke N. Saito, Minyoung Jung, Sumiyoshi Arai, Yukiko Matsumura, Takashi X. Fujisawa, et al. "Gazefinder as a clinical supplementary tool for discriminating between Autism Spectrum Disorder and typical development in male adolescents and adults," Molecular Autism, Vol. 7, No. 1, 19, 2016.
doi:10.1186/s13229-016-0083-y

9. Klin, Ami, Warren Jones, Robert Schultz, Fred Volkmar, and Donald Cohen, "Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism," Arch Gen Psychiatry, Vol. 59, No. 9, 809-816, 2002.
doi:10.1001/archpsyc.59.9.809

10. Wang, Shuo, Ming Jiang, Xavier Morin Duchesne, Elizabeth A. Laugeson, Daniel P. Kennedy, Ralph Adolphs, and Qi Zhao, "Atypical visual saliency in Autism Spectrum Disorder quantified through model-based eye tracking," Neuron, Vol. 88, No. 3, 604-616, 2015.
doi:10.1016/j.neuron.2015.09.042

11. Oliveira, Jessica S., Felipe O. Franco, Mirian C. Revers, Andréia F. Silva, Joana Portolese, Helena Brentani, Ariane Machado-Lima, and Fátima L. S. Nunes, "Computer-aided autism diagnosis based on visual attention models using eye tracking," Scientific Reports, Vol. 11, No. 1, 10131, 2021.
doi:10.1038/s41598-021-89023-8

12. Hedger, Nicholas and Bhismadev Chakrabarti, "Autistic differences in the temporal dynamics of social attention," Autism, Vol. 25, No. 6, 1615-1626, 2021.
doi:10.1177/1362361321998573

13. Atyabi, Adham, Frederick Shic, Jiajun Jiang, Claire E. Foster, Erin Barney, Minah Kim, Beibin Li, Pamela Ventola, and Chung Hao Chen, "Stratification of children with Autism Spectrum Disorder through fusion of temporal information in eye-gaze scan-paths," ACM Transactions on Knowledge Discovery from Data, Vol. 17, No. 2, 1-20, 2023.
doi:10.1145/3539226

14. Gutiérrez, Jesús, Zhaohui Che, Guangtao Zhai, and Patrick Le Callet, "Saliency4ASD: Challenge, dataset and tools for visual attention modeling for Autism Spectrum Disorder," Signal Processing: Image Communication, Vol. 92, 116092, 2021.
doi:10.1016/j.image.2020.116092

15. Tao, Yudong and Mei-Ling Shyu, "SP-ASDNet: CNN-LSTM based ASD classification model using observer scanpaths," 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 641-646, Shanghai, China, 2019.
doi:10.1109/icmew.2019.00124

16. Liaqat, Sidrah, Chongruo Wu, Prashanth Reddy Duggirala, Sen-Ching Samson Cheung, Chen-Nee Chuah, Sally Ozonoff, and Gregory Young, "Predicting ASD diagnosis in children with synthetic and image-based eye gaze data," Signal Processing: Image Communication, Vol. 94, 116198, 2021.
doi:10.1016/j.image.2021.116198

17. Praveena, K. N. and R. Mahalakshmi, "Classification of Autism Spectrum Disorder and typically developed children for eye gaze image dataset using convolutional neural network," International Journal of Advanced Computer Science and Applications, Vol. 13, No. 3, 2022.
doi:10.14569/ijacsa.2022.0130345

18. Carette, Romuald, Mahmoud Elbattah, Gilles Dequen, Jean-Luc Guérin, and Federica Cilia, "Visualization of eye-tracking patterns in Autism Spectrum Disorder: Method and dataset," 2018 Thirteenth International Conference on Digital Information Management (ICDIM), 248-253, Berlin, Germany, 2018.
doi:10.1109/icdim.2018.8846967

19. Kanhirakadavath, Mujeeb Rahman and Monica Subashini Mohan Chandran, "Investigation of eye-tracking scan path as a biomarker for autism screening using machine learning algorithms," Diagnostics, Vol. 12, No. 2, 518, 2022.
doi:10.3390/diagnostics12020518

20. Ahmed, Ibrahim Abdulrab, Ebrahim Mohammed Senan, Taha H. Rassem, Mohammed A. H. Ali, Hamzeh Salameh Ahmad Shatnawi, Salwa Mutahar Alwazer, and Mohammed Alshahrani, "Eye tracking-based diagnosis and early detection of Autism Spectrum Disorder using machine learning and deep learning techniques," Electronics, Vol. 11, No. 4, 530, 2022.
doi:10.3390/electronics11040530

21. Elbattah, Mahmoud, Jean-Luc Guérin, Romuald Carette, Federica Cilia, and Gilles Dequen, "Vision-based approach for autism diagnosis using transfer learning and eye-tracking," Proceedings of the 15th International Joint Conference on Biomedical Engineering Systems and Technologies, Vol. 5, 256-263, Online, 2022.
doi:10.5220/0010975500003123

22. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin, "Attention is all you need," 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 2017.

23. Wang, Yi-Wen, Kelsey J. Dommer, Sara Jane Webb, and Frederick Shic, "On the value of data loss: A study of atypical attention in Autism Spectrum Disorder using eye tracking," Proceedings of the 2023 Symposium on Eye Tracking Research and Applications, 1-2, New York, NY, USA, 2023.
doi:10.1145/3588015.3590127

24. Szegedy, Christian, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, "Going deeper with convolutions," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-9, Boston, MA, USA, June 2015.
doi:10.1109/cvpr.2015.7298594

25. Lord, Catherine, Susan Risi, Linda Lambrecht, Edwin H. Cook Jr., Bennett L. Leventhal, Pamela C. DiLavore, Andrew Pickles, and Michael Rutter, "The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism," Journal of Autism and Developmental Disorders, Vol. 30, No. 3, 205-223, 2000.
doi:10.1023/a:1005592401947

26. American Psychological Association, Diagnostic and Statistical Manual of Mental Disorders, 5th Ed., American Psychiatric Publishing, 2013.
doi:10.1176/appi.books.9780890425596

27. Keles, Umit, Dorit Kliemann, Lisa Byrge, Heini Saarimäki, Lynn K. Paul, Daniel P. Kennedy, and Ralph Adolphs, "Atypical gaze patterns in autistic adults are heterogeneous across but reliable within individuals," Molecular Autism, Vol. 13, No. 1, 39, 2022.
doi:10.1186/s13229-022-00517-2

28. Zhu, Huilin, Jun Li, Yuebo Fan, Xinge Li, Dan Huang, and Sailing He, "Atypical prefrontal cortical responses to joint/non-joint attention in children with Autism Spectrum Disorder (ASD): A functional near-infrared spectroscopy study," Biomedical Optics Express, Vol. 6, No. 3, 690-701, 2015.
doi:10.1364/boe.6.000690

29. Borji, Ali, "Saliency prediction in the deep learning era: Successes and limitations," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, No. 2, 679-700, 2021.
doi:10.1109/tpami.2019.2935715

30. Hochreiter, Sepp and Jürgen Schmidhuber, "Long short-term memory," Neural Computation, Vol. 9, No. 8, 1735-1780, 1997.
doi:10.1162/neco.1997.9.8.1735

31. Huang, Xun, Chengyao Shen, Xavier Boix, and Qi Zhao, "SALICON: Reducing the semantic gap in saliency prediction by adapting deep neural networks," 2015 IEEE International Conference on Computer Vision (ICCV), 262-270, Santiago, Chile, 2015.
doi:10.1109/iccv.2015.38

32. Ioffe, Sergey and Christian Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," Proceedings of the 32nd International Conference on Machine Learning, Vol. 37, 448-456, Lille, France, 2015.

33. Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna, "Rethinking the inception architecture for computer vision," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2818-2826, Las Vegas, NV, USA, 2016.
doi:10.1109/cvpr.2016.308

34. Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, "An image is worth 16 x 16 words: Transformers for image recognition at scale," International Conference on Learning Representations, Vienna, Austria, 2021.

35. Dwarampudi, Mahidhar and N. V. Subba Reddy, "Effects of padding on LSTMs and CNNs," arXiv:1903.07288, 2019.
doi:10.48550/arXiv.1903.07288

36. Han, Gyeo-Re, Artem Goncharov, Merve Eryilmaz, Shun Ye, Barath Palanisamy, Rajesh Ghosh, Fabio Lisi, Elliott Rogers, David Guzman, Defne Yigci, et al. "Machine learning in point-of-care testing: Innovations, challenges, and opportunities," Nature Communications, Vol. 16, No. 1, 3165, 2025.
doi:10.1038/s41467-025-58527-6

37. Amin, Youssef, Paola Cecere, Tania Pomili, and Pier Paolo Pompa, "Smartphone-integrated YOLOv4-CNN approach for rapid and accurate point-of-care colorimetric antioxidant testing in saliva," Progress In Electromagnetics Research, Vol. 181, 9-19, 2024.
doi:10.2528/PIER24120505

Dr. Deyu Guo

Dr. Yan Zhang

Dr. Tengfei Ma

Dr. Xinhua Zhu

Prof. Sailing He