VIPBoard: An intelligent touchscreen keyboard for visually impaired people
2020-04-21
Institute for Accessibility Develeopment
The Department of Computer Science and Technology of Tsinghua University and Sogou Inc. (NYSE: SOGO) released the world's first smart keyboard for visually impaired users during the ninth annual Global Accessibility Awareness Day (GAAD), which focuses on raising awareness of digital access and inclusion for people with disabilities and impairments.
The smart keyboard, VIPBoard, aims at improving the underlying keyboard algorithm without altering current input interaction.
Experiment setup and interface
Entering text on their smartphone remains hugely challenging for visually impaired users. While modern touchscreen keyboards are all powered by the word-level auto-correction ability to handle input errors, visually impaired users cannot enjoy this benefit. They rely on a screen-reader keyboard on their smartphones, which only offers character-level input that has no predictive text correction ability. When users are texting on their smartphones, they often miss the targeted key when touching without visual feedback.
The fitted general touch model
To tackle this problem, Professor Shi Yuanchun and Associate Researcher Yu Chun of the Department of Computer Science and Technology of Tsinghua University have led their research team to successfully design a character-level auto-correction method, achieving a 63% reduction in the character error rate (CER). This means that during typing, the algorithm can help reduce the missed clicks from an average of 3 to 1, for example.
Average text entry speed in each session
On each tap, the VIPBoard predicts the probability of each key considering both users' keyboard touch location and language model, and reads the most likely key, which saves calibration time when the user misses the target key. With a one-millisecond response time, VIPBoard allows users to correct a variety of text input errors and supports multi-language input.
The research team published their paper in the 2019 ACM CHI Conference “On Human Factors in Computing Systems” held in Glasgow, United Kingdom, the top academic conference in the field of human-computer interaction. The paper won Honorable Mention Award at the conference.
