Utilising transformers for American Sign Language fingerspelling recognition

This paper presents a machine learning approach to American Sign Language (ASL) fingerspelling recognition using Transformer models. Addressing the challenges of high variability in hand shapes, movement, and signing speed, the study utilises the new ASL Fingerspelling Recognition Corpus and the CRISP-DM methodology to develop and evaluate a proof-of-concept model. The model achieved a mean Levenshtein distance of 4.7, corresponding to an error rate of 16.37%. This research demonstrates the feasibility of using advanced AI techniques to enhance accessibility for the deaf and hard of hearing communities by translating ASL fingerspelling into text.

Publications router.