Yimu 譯幕

Live translation demo

This recording shows English narration appearing as source text above live Traditional Chinese subtitles.

Turn on sound to follow the English narration alongside the translation. The subtitles were generated during the recording.

Following English audio across applications

Long lectures and meetings take more effort to follow in English. When I miss a phrase, replaying it interrupts the rest of the session.

The tools I tried did not give me the combination I needed: desktop audio capture, streaming translation, and Traditional Chinese with wording familiar in Taiwan. That last detail mattered in everyday reading, including terms such as 人工智慧.

Yimu began as a tool for my own listening habits. Capturing system audio meant it could work with videos, calls, and other audio playing through the computer's default output device.

Building and checking the app in stages

The first working version took one day. A written plan divided the work into four stages so I could check each part before connecting the full application.

1. 01

   Audio capture

   Capture Windows playback through WASAPI loopback and prepare it for the translation API. I checked recorded audio by ear for changes in pitch or speed.

2. 02

   Translation session

   Stream audio to Gemini and display the translated text as it arrives. Checks covered live speech and reconnection after a dropped session.

3. 03

   Subtitle window

   Build a draggable, semi-transparent window that stays above other applications. Automated UI checks and screen review covered its appearance and behavior.

4. 04

   Integration and iteration

   Connect capture, translation, and display. Subsequent work added the second engine, settings, transcript saving, and summaries.

Acceptance checks used recorded audio, live API sessions, and the running interface. Offline regression tests covered behavior that could be checked without sending API requests.

From desktop audio to translated text

Yimu sends captured audio to the selected translation engine and displays its text output as subtitles.

The pipeline

After the session

With transcript saving enabled, Finish closes the session and offers a summary in the selected target language. The confirmation shows the provider, transcript character count, and estimated number of requests.

The summary uses source-language text when available and falls back to translated text. Long transcripts are processed in sections. Each summary is saved as a separate Markdown file beside the transcript, preserving earlier summaries.

In use

Decisions behind the interface

Pricing information appears beside the engine selector. Summary requests are separate from live translation, and their estimated request count appears before confirmation.

Problems found through use and review

Daily use and code review revealed several failures that needed changes beyond the initial working flow.

• Repeated Chinese text
  Chinese speech could reappear in the transcript output even when no translated audio was produced. A local filter now compares output with recent source text, accounting for Simplified and Traditional Chinese.
• An unfinished final sentence
  When I paused a video, OpenAI sometimes left the last sentence incomplete. Windows loopback stopped sending audio, so the session received no trailing silence. A short, capped burst of synthetic silence lets the final text complete.
• A microphone failure that kept reconnecting
  Python's TaskGroup wrapped an exception that the error handler did not recognize. The app repeatedly reconnected instead of reporting the microphone failure. Error handling was corrected, with a regression test for the wrapped exception.
• Missing source subtitles
  OpenAI source text was absent because input transcription had not been enabled in the session configuration. The same omission also affected echo filtering. Enabling that setting restored both.
• A rejected summary request
  The Gemini SDK serialized a schema flag that the API did not accept. The fix separated the schema sent to the provider from the stricter validation applied to the returned summary before saving.

These fixes covered playback pauses, missing configuration, device failures, and rejected requests alongside the normal translation flow.

How I used AI in this project

This project was built in collaboration with AI; the design decisions, content, and direction are mine.

AI tools wrote the implementation code. Claude Code helped build the initial application, and later work with Codex added the summary workflow.

My work covered the product definition, written build plan, name, and logo direction. I decided how the subtitle window should behave, which controls belonged in Settings, and when a saved transcript could be sent for summarization.

Reviewing the running app and reporting problems from daily use remained part of my role throughout development.

Using Yimu for lectures and meetings

Yimu now runs on my desktop for lectures and meetings. Depending on the session, I adjust the subtitle size and line count, save a transcript, or request a summary afterward.

Continued use shaped the work beyond the first version. Pausing a video exposed the unfinished-sentence problem; reading both languages exposed the missing source-text setting. Those observations led to specific fixes in the tool I use.