Live Translator

Real-Time Voice Translation with Japanese Kana Romanization

June 14, 2023 - Tommy Dräger

As part of a side project to enhance my Japanese language skills and improve communication with my colleagues, I developed a live messenger. Unlike Google or Slack, which don't offer a real-time English to Japanese solution, my tool converts spoken language into text, translates it, and even romanizes Japanese kana. Supporting German, English, and Japanese, it uses a simple Node.js server to handle multiple user sessions, making it an effective and interactive language-learning platform.

How It Works

1. Session Connect

   • Using Socket.IO, the server listens and forks new connections for the user to join a global session.

2. Voice-to-Text Conversion:

   • The browser captures voice input and converts it to text using Chrome's Speech Recognition.

3. Translation:

   • After transcription, the text is translated between supported languages (Japanese, English, and German) using a Google WebApp that utilizes Google Translate.
   • Users can set the source and target languages with simple commands, enhancing usability.

4. Romanization of Japanese Kana:

   • The tool includes a feature that romanizes Japanese kana using Kuroshiro, helping learners familiarize themselves with kanji and making learning more accessible and engaging.

Supported Commands

Users can easily adjust the settings with the following commands:

You have to speak clearly into the microphone and say:

• Set Source and Target Languages:

  • "Japanese English"
  • "English Japanese"
  • "English German"

• Control Microphone:

  • "turn off" (to mute the mic)
  • "turn on" (to unmute the mic)

Live Demo

To see the messenger in action, visit Live Translator. This demo showcases the real-time translation and transcription capabilities of the tool.

Future Enhancements

There are several planned features to improve user experience:

• Profile Pictures: To help identify users in chats.
• Clickable Interface: For easier interaction.
• Invitation Links: Generate links to join chat rooms.
• Improved Kana to Romaji Conversion: Enhancing language learning.
• AI-Powered Translation Refinement: To provide more natural translations.
• Customizable Design: Options for larger fonts, suitable for meetings.
• Learning Mode: Additional features focused on language acquisition.
• Copy-to-Clipboard: Quick copying of translated text.

Conclusion

This live messenger becomes quite handy, especially in situations where you need to follow along in a Japanese meeting. You can also forward your audio output with a virtual cable into your microphone, making it easy to work with Google Meet or any other voice call app.