Local Transcription and Meeting Minutes
Personal Project
Overview
Built a Claude Code skill that transcribes recordings locally on a Mac and turns the meeting in a given time range into minutes with an LLM. Transcription runs locally on the Mac, and the transcript goes to an LLM when the minutes are written.
Transcription Pipeline
- Picking a recording: the skill scans the macOS Voice Memos store, lists the recordings, and the user picks one. Recordings are copied, never moved, so the Voice Memos database and iCloud sync stay intact.
- Preprocessing: two-pass loudness normalization with ffmpeg (EBU R128, -16 LUFS), conversion to 16 kHz mono, and silence trimming with Silero VAD (silences of 500 ms or more, keeping 400 ms of padding).
- Transcription: mlx-whisper (whisper-large-v3) on the Mac's GPU, with the trimmed silences added back so timestamps match the original clock.
- Hallucination control: no carry-over of previous text (so loops cannot propagate), a silence threshold, and dropping segments that look like silence and have low confidence.
- Time-range extraction: from long recordings, the 30-minute pieces that overlap the requested range are cut out and merged, and any missing part is reported. Speech separated by gaps under 10 minutes is grouped into one meeting candidate.
- Speaker diarization (optional, off by default): Nemotron-3-Diarization runs on DGX Spark over SSH and Docker in parallel with local transcription. Audio goes up as FLAC, speaker turns come back, and each word is assigned a speaker. On any failure it falls back to a transcript without speakers.
Writing the Minutes
- By default a Claude worker writes the minutes in Markdown, with an option to switch to Gemini. The user confirms the speaker-to-name mapping before generation.
- Past minutes as context: only the meeting-name and participant sections of the 3 newest minutes are passed (up to 4,000 characters) so names, honorifics, and meeting types stay consistent, without passing the content of past discussions.
- Choosing the effort level: in my own blind scoring, high reasoning effort averaged 88.8 against 86.5 for medium, and medium sometimes fell below the bar, so high is the default (a recorded evaluation).
Problems Found, and Fixes
- Diarization can run out of memory on the DGX Spark (shared with vLLM) → it waits 30 seconds and retries once.
- Nemotron was not trained on Japanese, so it can get the number of speakers wrong → the speaker-to-name mapping is confirmed by a person.
- NeMo from PyPI could not load the model → built a Docker image with NeMo pinned to a commit that can.
- SSH runs non-interactively with a short connect timeout, and the remote job directory and container are always cleaned up.
Development
Managed with Python 3.12 and uv, with settings loaded through Pydantic Settings. Transcription and the LLM sit behind Protocols, which tests replace with mocks. pytest collects 357 tests (a collection count, not a run result) (51 for speaker alignment, 45 for long-recording handling, and more).
Technologies
Python 3.12uvClaude Codemlx-whisperSilero VADffmpegGemini APINemotron-3-DiarizationNeMoDockerDGX SparkPydanticpytest