Inferring players’ psychological states using a time-series model integrated with facial image quality
,
,
,
,
and
Article Sidebar
Full Text: 
PDF
Abstract
Analysing the latent emotional states of players in a real-world competitive environment is a challenging problem due to the inconsistent video quality and limitations of current methods for modelling emotional dynamics over time. This paper proposes an integrated system that assesses facial image quality (FIQA) as a reliability signal within a time-series model, allowing for continuous monitoring of facial expressions even under shaky, blurry, or partially obscured images. Based on the acquired latent state space, the study proposes three quantitative indices: Latent Stability Index (LSI), Temporal Drift Index (TDI), and Latent Variability Index (LVI) to characterise the stability and variability of emotions over time. Experimental results show that the system is capable of consistently reflecting the evolution of emotional states in real-world competition situations. The study is positioned as a proof-of-concept, laying the groundwork for analysing the underlying emotional states from sports videos and opening up potential applications in supporting performance analysis and tactical decision-making.
Tóm tắt
Việc phân tích trạng thái cảm xúc tiềm ẩn của cầu thủ trong môi trường thi đấu thực tế là một bài toán thách thức do chất lượng video không ổn định và hạn chế của các phương pháp hiện tại trong việc mô hình hóa động thái cảm xúc theo thời gian. Bài báo được thực hiện nhằm đề xuất một hệ thống tích hợp đánh giá chất lượng ảnh khuôn mặt (FIQA) như một tín hiệu độ tin cậy vào mô hình chuỗi thời gian, cho phép theo dõi liên tục biểu cảm khuôn mặt ngay cả trong các điều kiện hình ảnh bị rung, mờ hoặc che khuất một phần. Trên cơ sở không gian trạng thái tiềm ẩn thu được, ba chỉ số định lượng đã được đề xuất gồm Latent Stability Index (LSI), Temporal Drift Index (TDI) và Latent Variability Index (LVI) nhằm đặc trưng hóa mức độ ổn định và biến động cảm xúc theo thời gian. Kết quả thực nghiệm cho thấy hệ thống có khả năng phản ánh nhất quán diễn biến trạng thái cảm xúc trong các tình huống thi đấu thực tế. Nghiên cứu được định vị như một proof-of-concept, đặt nền tảng cho hướng phân tích trạng thái cảm xúc tiềm ẩn từ video thể thao và mở ra tiềm năng ứng dụng trong hỗ trợ phân tích hiệu suất và ra quyết định chiến thuật.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
Azizan, L. A., Aziz, N. H. A., & Abdullah, S. N. H. S. (2024). Advances in facial expression recognition: A survey of methods, benchmarks, models, and datasets. Information, 15(3), 135. https://doi.org/10.3390/info15030135
Chen, Z., Wang, M., Zhang, Y., Deng, W., Cui, X., Shi, H., Wen, D., & Zhao, J. (2024). Confidence-aware RGB-D face recognition via virtual depth synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (pp. 1421-1430). IEEE.
https://arxiv.org/abs/2403.06529
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020). RetinaFace: Single-shot multi-level face localisation in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5203-5212). IEEE. https://doi.org/10.1109/CVPR42600.2020.00525
Goodfellow, I. J., Boureau, Y.-L., & Bengio, Y. (2013). Challenges in representation learning: A report on three machine learning contests. In International Conference on Neural Information Processing (ICONIP) (pp. 117-124). Springer. https://doi.org/10.1007/978-3-642-42051-1_16
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770-778). IEEE. https://doi.org/10.1109/CVPR.2016.90
Hernandez-Ortega, J., Galbally, J., Fierrez, J., Haraksim, R., & Beslay, L. (2019). FaceQnet: Quality assessment for face recognition based on deep learning. In Proceedings of the 2019 International Conference on Biometrics (ICB) (pp. 1-8). IEEE. https://doi.org/10.1109/ICB45273.2019.8987255
Ivarsson, A., Kilhage-Persson, A., Martindale, R., Priestley, D., Huijgen, B., Ardern, C., & McCall, A. (2020). Psychological factors and future performance of football players: A systematic review with meta-analysis. Journal of Science and Medicine in Sport, 23(4), 415-420. https://doi.org/10.1016/j.jsams.2019.10.021
Jailani, N. L. M., Dhanasegaran, J. K., Alkawsi, G., Alkahtani, A. A., Phing, C. C., Baashar, Y., Capretz, L. F., Al-Shetwi, A. Q., & Tiong, S. K. (2023). Investigating the power of LSTM-based models in solar energy forecasting. Processes, 11(5), 1382. https://doi.org/10.3390/pr11051382
Khan, S. A., Khan, T. A., & Min-Allah, N. (2024). Learning face expression features from video using spatio-temporal feature extractor and CNN-LSTM. In 2023 International Conference on Computing, Intelligence and Data Analytics (ICCIDA) (pp. 1-6). IEEE. https://doi.org/10.1109/ICCIDA58661.2023.10537794
Kim, D. H., Baddar, W. J., Jang, J., & Ro, Y. M. (2019). Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition. IEEE Transactions on Affective Computing, 10(2), 223-236. https://doi.org/10.1109/TAFFC.2017.2695999
Lee, J., Kim, S., & Ro, S. (2023). Facial expression recognition in the wild using face graph and attention. IEEE Access, 11, 60324-60336. https://doi.org/10.1109/ACCESS.2023.3287639
Li, S., & Deng, W. (2022). Deep facial expression recognition: A survey. IEEE Transactions on Affective Computing, 13(4), 1195–1215. https://doi.org/10.1109/TAFFC.2020.2981446
Meng, Q., Zhao, S., Huang, Z., & Zhou, F. (2021). MagFace: A universal representation for face recognition and quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 14225-14234). IEEE. https://doi.org/10.1109/CVPR46437.2021.01400
Nguyen, X.-B., Nguyen, H.-T., Nguyen, T.-H., Do, N.-T., & Dinh, Q. V. (2024). Emotic masked autoencoder on dual-views with attention fusion for facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (pp. 4784-4792). IEEE. https://doi.org/10.1109/CVPRW63382.2024.00481
Ou, F., Liu, X., Zhang, Y., & Zhang, G. (2024). CLIB-FIQA: Face image quality assessment with confidence calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2554-2564). IEEE.
https://doi.org/10.1109/CVPR52733.2024.00248
Parthasarathy, V. B., Zafar, A., Khan, A., & Shahid, A. (2024). The ultimate guide to fine-tuning LLMs from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research challenges and opportunities. arXiv. https://doi.org/10.48550/arXiv.2408.13296
Rubanova, Y., Chen, R. T. Q., & Duvenaud, D. K. (2019). Latent ordinary differential equations for irregularly-sampled time series. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019) (pp. 5320-5330). Curran Associates.
https://dl.acm.org/doi/10.5555/3454287.3454765
Shawi, R. T., Abdulkadhim, A. A. A., & Abbas, F. N. (2025). Facial emotion recognition based on improved ResNet50 using hybrid pooling and adaptive Leaky ReLU. International Journal of Scientific Research in Science, Engineering and Technology, 12(3), 728-737. https://doi.org/10.32628/IJSRSET251290