Catchapp is a mobile app designed for students who do not study in their native language. The purpose of the app is to help them maintain a high level of learning and understanding while not disturbing them in class. The app listens to the lecture in real time and identifies professional terms. Then it translates these terms into the student’s native language and sends it back to him. In addition, the student can save the professional terms for later use. Terms are presented as text bubbles on the screen. Clicking on each text bubble opens a window with an explanation on the relevant concept accompanied by a picture.
How does it work?
With a click on the Catchapp button the app starts recording the lecture’s audio, and using a Google’s service converts the speech-to-text (speech-recognition). Then, an HTTP request with the detected text is sent to a server that sits in a cloud. The server identifies key concepts in the resulting text using a TextRazor service. Then, for each concept it pulls out relevant information from Wikipedia and returns it to the client.