News and Topics

2021.05.31

The Centre for Sign Linguistics and Deaf Studies (CSLDS) of the Department of Linguistics and Modern Languages at The Chinese University of Hong Kong (CUHK), and The Nippon Foundation (Japan), in close partnership with Google LLC and the Sign Language Research Center (SLRC) of Kwansei Gakuin University, have jointly developed SignTown, the world’s first sign language online learning game featuring AI-based sign language recognition to promote understanding of sign language and deaf awareness. The SLRC was established in April 2015 as the first institution of sign language at a Japanese university, and it has been active since 2016, supported by The Nippon Foundation. The SLRC has been contributing to this project by selecting Japanese Sign Language (JSL) words and collecting JSL data for use in machine learning.

SignTown is a web-based, real-time sign recognition game, where users take control of a cute avatar who is a newcomer to SignTown, a town in a fictional world where everyone communicates in sign language to get things done. With the AI-based recognition technology, users can learn and express themselves in sign language, and get feedback about their signing simply with a computer and a webcam. SignTown also acts as a gateway to understanding sign language and Deaf culture by providing information about Deaf cultures throughout the game. Users can enjoy the experience of signing in Hong Kong Sign Language (HKSL) and Japanese Sign Language (JSL), also known as Shuwa in Japanese, while allowing them to switch between the two in the game. Throughout the game, users can collect various items after their signs are correctly recognised. Users can learn signs from various common categories in SignTown, which is designed based on various themes, ranging from placing orders in a café to finding accommodation during a trip. With the aim of making improvements based on feedback from users, the beta version of SignTown was released on May 21. The first complete version of SignTown will be available on September 23, 2021, the International Day of Sign Language.

The full press release can be viewed on  The Nippon Foundation's website.