Utilising transformers for American Sign Language fingerspelling recognition
Book chapter
Pinnington, J., Souag, A. and Azhar, H. 2024. Utilising transformers for American Sign Language fingerspelling recognition. in: 24th International Symposium on Computational Intelligence and Informatics (CINTI 2024)
Authors | Pinnington, J., Souag, A. and Azhar, H. |
---|---|
Abstract | This paper presents a machine learning approach to American Sign Language (ASL) fingerspelling recognition using Transformer models. Addressing the challenges of high variability in hand shapes, movement, and signing speed, the study utilises the new ASL Fingerspelling Recognition Corpus and the CRISP-DM methodology to develop and evaluate a proof-of-concept model. The model achieved a mean Levenshtein distance of 4.7, corresponding to an error rate of 16.37%. This research demonstrates the feasibility of using advanced AI techniques to enhance accessibility for the deaf and hard of hearing communities by translating ASL fingerspelling into text. |
Keywords | American Sign Language (ASL); Fingerspelling recognition; Transformer models; Machine learning; CRISP-DM; AI accessibility; Deaf and hard of hearing; Levenshtein distance; PyTorch; MediaPipe |
Year | 2024 |
Book title | 24th International Symposium on Computational Intelligence and Informatics (CINTI 2024) |
Output status | In press |
File | License File Access Level Restricted |
Publication process dates | |
Deposited | 27 Nov 2024 |
Related URL | https://conf.uni-obuda.hu/cinti2024/ |
Event | IEEE 24th International Symposium on Computational Intelligence and Informatics (CINTI 2024) |
https://repository.canterbury.ac.uk/item/99v5z/utilising-transformers-for-american-sign-language-fingerspelling-recognition
3
total views1
total downloads1
views this month0
downloads this month