WithYou

Author

Zhang, Xinlei and Miyaki, Takashi and Rekimoto, Jun

Abstract

Speech shadowing, in which the subject listens to native narration sound and tries to repeat it immediately while listening, is a proven way of practicing speaking skills when learning foreign languages. However, since the narration is independent of user's speech, the playback cannot make an adjustment when the learner fails to catch up, and this makes shadowing difficult. We propose WithYou, a system based on Automated Speech Recognition (ASR) that is able to adjust narration playback during a live shadowing speech. WithYou compares the student's live speech with the narration playback to detect shadowing mistakes. In addition, WithYou is able to handle pauses and recognize repetitive phrases in shadowing practice. A user study shows that practicing shadowing with WithYou is easier and more effective compared with conventional methods.