Whisper and Faster-Whisper: A Revolution in Local Transcription

Transcribing audio to text is an essential task for developers, whether it's for creating voice recognition applications, analyzing meeting recordings, or adding subtitles to videos. Performing this transcription locally on your own machine offers significant advantages in terms of privacy protection and reducing recurring costs associated with cloud services.

In this article, we will explore how to set up a fast local audio transcription system using Whisper and its optimized version, Faster-Whisper. We will cover audio preprocessing steps, such as converting MP3 files to WAV, writing a Python script, and discuss running on CPU and GPU.

What is Whisper? And why use a local variant?

Whisper, developed by OpenAI, is an automatic speech recognition (ASR) model that has been trained on a vast amount of multilingual audio. This model can operate effectively even in the presence of background noise or with different accents. However, using Whisper in its original form can be slow on a CPU and consume a significant amount of memory.

To overcome these limitations, optimized variants have been developed. whisper.cpp is a C++ version that does not require heavy dependencies. It is very fast on CPU but requires compilation and is less user-friendly for Python users. In contrast, Faster-Whisper is a reimplementation using CTranslate2, which runs up to four times faster than the original Whisper model, uses less RAM, and integrates seamlessly with Python. In this tutorial, we will use Faster-Whisper.

Both variants operate 100% locally, meaning no data leaves your computer, thus ensuring the confidentiality of the processed information.

Setting up your environment (multi-platform)

The setup described here is compatible with Windows, macOS, and Linux, provided you are using Python 3.8 or later. It is advisable to create and activate a virtual environment to isolate the necessary dependencies:

python -m venv whisper_env

To activate the virtual environment on macOS and Linux, use the following command:

source whisper_env/bin/activate

On Windows, the command is slightly different:

whisper_env\Scripts\activate

Once the environment is activated, install Faster-Whisper with the following command:

pip install faster-whisper

Installing audio preprocessing tools

Whisper requires audio to be in mono WAV format at 16 kHz. To convert common audio formats such as MP3, M4A, or OGG, you need to use FFmpeg and the Python library pydub.

• On Windows, download FFmpeg from FFmpeg.org and add it to your PATH, or use winget install ffmpeg.
• On macOS, the command is: brew install ffmpeg
• On Linux (Ubuntu/Debian), use: sudo apt install ffmpeg

Next, install pydub with the following command:

pip install pydub

Optional GPU support

For those with an NVIDIA GPU who wish to further accelerate transcription, it is possible to install cuBLAS and cuDNN by following the GPU guide for Faster-Whisper. In the absence of these installations, the code will run on the CPU by default.

Audio preprocessing: converting non-WAV files

Most audio files you encounter are not raw WAV files. They often use compression (like MP3) or container formats (such as M4A). Therefore, it is necessary to convert them to mono PCM WAV at 16 kHz before providing them to Whisper.

Here is a Python function that uses pydub (which calls FFmpeg in the background) to perform this conversion:

from pydub import AudioSegment

def convert_to_wav(input_path, output_path=None):
    """Convert any audio file (MP3, M4A, OGG, etc.) to WAV (16 kHz, mono)."""
    if output_path is None:
        base, _ = os.path.splitext(input_path)
        output_path = base + ".wav"
    
    # Load the audio (pydub uses ffmpeg)
    audio = AudioSegment.from_file(input_path)
    
    # Convert to mono and set the sample rate to 16000 Hz
    audio = audio.set_channels(1).set_frame_rate(16000)
    audio.export(output_path, format="wav")
    
    return output_path

wav_file = convert_to_wav("meeting.mp3")
print(f"Converted to: {wav_file}")

Basic transcription script with Faster-Whisper

Now let's write a complete Python script that loads a Whisper model, transcribes a WAV file, and prints the result:

from faster_whisper import WhisperModel

def transcribe_audio(wav_path, model_size="base", device="cpu"):
    """Transcribe a WAV file (16 kHz mono) using Faster-Whisper."""
    model = WhisperModel(model_size, device=device, compute_type="int8")
    
    # Run the transcription
    segments, info = model.transcribe(wav_path, beam_size=5, language="en")
    
    print(f"Detected language: {info.language} (probability: {info.language_probability:.2f})")
    print("\nTranscription:")
    for segment in segments:
        print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")
    
    # Return the full text if needed
    full_text = " ".join([seg.text for seg in segments])
    return full_text

if __name__ == "__main__":
    text = transcribe_audio("my_recording.wav", model_size="small", device="cpu")

What happens in the code above?

WhisperModel downloads the chosen model (e.g., small) to ~/.cache/huggingface/hub on the first run.

• beam_size=5 balances accuracy and speed. Higher values (e.g., 10) are slower but more accurate.
• compute_type="int8" uses 8-bit integer math for faster inference. For GPU, you can try "float16".

Configuration complexity

• Slow (but acceptable for files under 10 minutes)
• None (just install)
• Beginners, laptops, small projects
• Requires NVIDIA drivers, cuBLAS, cuDNN
• Long files, batch transcription

To use a GPU, change device="cuda" in the code. Faster-Whisper automatically detects CUDA if installed correctly.

Tip: Even on CPU, Faster-Whisper is much faster than the original Whisper. For a 10-minute MP3, the base model on a modern CPU takes about 2 minutes.

Converting MP3 to transcription: a complete example

Here is a complete script that converts any audio file to WAV, then transcribes it:

from pydub import AudioSegment
from faster_whisper import WhisperModel

def convert_to_wav(input_path):
    """Convert any audio to mono WAV at 16 kHz."""
    audio = AudioSegment.from_file(input_path)
    audio = audio.set_channels(1).set_frame_rate(16000)
    wav_path = os.path.splitext(input_path)[0] + ".wav"
    audio.export(wav_path, format="wav")

def transcribe_file(audio_path, model_size="base", device="cpu"):
    # Step 1: Convert if not already a WAV
    if not audio_path.lower().endswith(".wav"):
        print(f"Converting {audio_path} to WAV...")
        audio_path = convert_to_wav(audio_path)

    # Step 2: Transcribe
    print(f"Loading model '{model_size}' on {device.upper()}...")
    model = WhisperModel(model_size, device=device, compute_type="int8")
    segments, info = model.transcribe(audio_path, beam_size=5)
    
    print(f"\nLanguage: {info.language} (probability: {info.language_probability:.2f})")
    print("\nTranscription:")
    for seg in segments:
        print(seg.text, end=" ", flush=True)
    print()  # final newline

if __name__ == "__main__":
    # Example: transcribe an MP3 file
    transcribe_file("interview.mp3", model_size="small", device="cpu")

Save this as transcribe.py and run:

python transcribe.py

The script will download the model once, convert the file, and display the transcription.

You now have a fast, local audio transcription system that respects privacy. A few key points to remember:

• Faster-Whisper provides near real-time transcription on a CPU and excellent speed on a GPU.
• Always preprocess audio to 16 kHz mono WAV using pydub and FFmpeg.
• The model_size parameter trades accuracy for speed—start with "base" or "small".
• Running locally means no API keys, no data sharing, and no monthly fees.
• Try different Whisper model sizes for better accuracy. Add speaker diarization (identifying who spoke when) using libraries like pyannote.audio. Create a simple web interface with Gradio or Streamlit.