Recent artificial intelligence manufactures based on voice recognition cannot be used by the deaf. In order to solve this problem, we present ‘Lip Analysis System’ using deep learning with lip movement. This system analyzes mouth shape and process time series data through the 3-dimensional convolution neural network and gated recurrent unit. Our Lip Analysis System deals with Korean vocabulary, and creates subtitles based on oral movements of the subjects in the video. This system recognizes individual words rather than the whole sentences. We achieved 91.8% accuracy. This system could be applicable for someone who being deaf, having the difficulty of hearing, or anyone who requires communication without the voice.

artificial intelligence, deep learning, lip analysis system.