Technical Report of IEICE Japan, Vol. SP2002-113, pp. 27-32, 2002 (in Japanese)
A study on education tools with an animated talking agent for the hearing impaired
T. Kitamura, T. Arai and P. Connors
Abstract: An education tool with an animated taking agent named Baldi, that has been developed at CSLU (Center for Spoken Language Understanding at the Oregon Graduate Institute) is used at Tucker-Maxon (Portland, Oregon), a school for deaf children using oral language. Tutors like the CSLU Toolkit have not been used in Japan. In order to create a Japanese language animated agent, we have tried to map visemes of Mexican Spanish phonemes to the corresponding visemes of the Japanese phonemes. In this report, we proposed the ways in which these resources can be used effectively in Japan. We focused on the following areas:
(1) We can obtain not only audible information but also visible information from an animated talking agent;
(2) By combining an animated talking agent, speech synthesis, speech recognition, and images, we can make any kind of tutor with the CSLU Toolkit.
The result of the examination showed that the CSLU Toolkit could be effective for training pronunciation, speech reading, and the acquisition of vocabulary in Japan if there were a Japanese voice, text to speech system, speech recognition and facial animation in Japanese within the CSLU Toolkit.
Keywords: animation, hearing impaired, speech reading, training of pronunciation
[PDF (952 kB)]