Whisper Self-Hosted Transcription: A Complete Docker Compose Deployment Guide

Introduction

Voice data is one of the most sensitive types of information you can generate. Every meeting recording, voice memo, or dictated note contains personal and professional details you likely don't want third-party cloud services analyzing. Self-hosting a speech-to-text service using OpenAI's Whisper model puts transcription entirely under your control, keeping audio files and transcripts on your own hardware.

This guide walks you through deploying a production-ready Whisper transcription service using Docker Compose. You'll set up a REST API that accepts audio files and returns text transcripts, with optional GPU acceleration for faster inference. The deployment uses faster-whisper, a reimplementation of Whisper that is significantly more efficient than the original model while maintaining accuracy.

By the end of this guide, you'll have a fully functional transcription service accessible via HTTP, ready to integrate with your existing homelab tools like Nextcloud, Paperless-ngx, or custom automation scripts. You'll also learn how to secure it with a reverse proxy, configure automatic backups, and diagnose common deployment issues.

Prerequisites

Before starting, ensure your system meets the following requirements. These are typical estimates — actual resource usage varies based on audio length, concurrent requests, and model size.

Step-by-Step Installation

Step 1: Create Project Directory and Environment File

Create a dedicated directory for your Whisper service and set up a .env file to store configuration variables. This keeps secrets out of your Compose file and makes upgrades easier.

mkdir -p ~/whisper-transcriber && cd ~/whisper-transcriber

Create the .env file with your preferred editor:

nano .env

Paste the following content, adjusting values as needed:

# Model size: tiny, base, small, medium, large-v3, or distil-large-v3
WHISPER_MODEL=large-v3
# Compute type: int8, float16, or int8_float16
WHISPER_COMPUTE_TYPE=int8
# Device: cpu or cuda
WHISPER_DEVICE=cpu
# API authentication token (change this to a strong random string)
API_TOKEN=change-me-to-a-random-64-char-string

Warning: Never commit .env to version control. Add it to your .gitignore file immediately:

echo ".env" >> .gitignore

Step 2: Create Docker Compose File

Create a docker-compose.yml file in the same directory:

nano docker-compose.yml

Paste the complete configuration below:

services:
  whisper:
    image: fedirz/faster-whisper-server:latest
    container_name: whisper-server
    restart: unless-stopped
    ports:
      - "8000:8000"
    environment:
      - WHISPER_MODEL=${WHISPER_MODEL:-large-v3}
      - WHISPER_COMPUTE_TYPE=${WHISPER_COMPUTE_TYPE:-int8}
      - WHISPER_DEVICE=${WHISPER_DEVICE:-cpu}
      - API_TOKEN=${API_TOKEN}
    volumes:
      - ./models:/root/.cache/huggingface
      - ./audio:/audio
    command: --model ${WHISPER_MODEL:-large-v3} --device ${WHISPER_DEVICE:-cpu} --compute-type ${WHISPER_COMPUTE_TYPE:-int8} --host 0.0.0.0 --port 8000
    # Uncomment for GPU support (requires nvidia-container-toolkit installed on host)
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: 1
    #           capabilities: [gpu]

Note on versions: image: fedirz/faster-whisper-server:latest is intentionally unpinned. Check the official GitHub releases page before pinning a version — the version above may be outdated by now. The faster-whisper-server project by fedirz is actively maintained and provides pre-built Docker images for both CPU and CUDA.

Step 3: Start the Service

Launch the container and verify it boots correctly:

docker compose up -d && docker compose logs -f whisper

Wait until you see a log line similar to Uvicorn running on http://0.0.0.0:8000. This indicates the API is ready.

Step 4: Test the Transcription API

Open a new terminal and test with a sample audio file. First, download a test clip:

curl -L -o test.mp3 "https://github.com/fedirz/faster-whisper-server/raw/main/tests/data/test.mp3" && curl -X POST "http://localhost:8000/v1/audio/transcriptions" -H "Authorization: Bearer ${API_TOKEN}" -F file="@test.mp3" -F model="${WHISPER_MODEL}" -F language="en"

You should receive a JSON response containing the transcript text.

Step 5: Verify Model Persistence

Check that the model weights downloaded to your local ./models directory:

ls -la ~/whisper-transcriber/models && du -sh ~/whisper-transcriber/models

This confirms models persist across container restarts, avoiding re-downloads on every start.

Step 6: Configure Automatic Restarts

Ensure the service survives system reboots. The restart: unless-stopped policy in the Compose file handles this. Verify your Docker daemon starts on boot:

sudo systemctl enable docker && sudo systemctl start docker

Step 7: Set Up a Systemd Service for Compose (Optional)

For more robust process management, create a systemd service:

sudo tee /etc/systemd/system/whisper-transcriber.service > /dev/null <<EOF
[Unit]
Description=Whisper Transcription Service
Requires=docker.service
After=docker.service

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/home/your-user/whisper-transcriber
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
StandardOutput=journal

[Install]
WantedBy=multi-user.target
EOF

Then enable and start it:

sudo systemctl daemon-reload && sudo systemctl enable whisper-transcriber && sudo systemctl start whisper-transcriber

Step 8: Enable GPU Acceleration (Optional)

If you have an NVIDIA GPU, install the container toolkit and uncomment the GPU section in your Compose file:

sudo apt-get install -y nvidia-container-toolkit && sudo systemctl restart docker

Then edit docker-compose.yml to uncomment the deploy block and change WHISPER_DEVICE=cuda in your .env file. Restart the service:

docker compose down && docker compose up -d

Step 9: Process Local Audio Files

You can transcribe files stored in the ./audio directory by mounting them into the container. Copy an audio file there:

cp /path/to/your/recording.mp3 ~/whisper-transcriber/audio/ && docker compose exec whisper python -c "from faster_whisper import WhisperModel; model = WhisperModel('large-v3', device='cpu', compute_type='int8'); segments, info = model.transcribe('/audio/recording.mp3'); print('Detected language:', info.language); [print(s.text) for s in segments]"

Advanced Configuration and Optimization

Reverse Proxy with SSL

Expose the service securely using Caddy or Nginx. Here's a Caddy example:

mkdir -p ~/caddy && cd ~/caddy && nano Caddyfile

transcribe.yourdomain.com {
    reverse_proxy localhost:8000
}

Run Caddy:

docker run -d --name caddy -p 80:80 -p 443:443 -v $PWD/Caddyfile:/etc/caddy/Caddyfile -v caddy_data:/data caddy:latest

Backup Strategy

The critical data to back up are the model weights in ./models (re-downloadable but large) and any configuration changes. Use a simple cron job with tar:

crontab -e

Add this line to back up nightly:

0 2 * * * tar -czf /backup/whisper-$(date +\%Y\%m\%d).tar.gz -C ~/whisper-transcriber models .env docker-compose.yml

Optional Hardening

The following security measures are application-specific and may break container functionality if copied verbatim. Test each carefully against your deployment.

Troubleshooting Common Issues

Conclusion

You now have a fully self-hosted Whisper transcription service running behind a Docker Compose setup. This deployment gives you complete ownership of your audio data and transcripts, eliminating reliance on third-party speech-to-text APIs. The service integrates easily with other homelab applications via its REST API, and you can extend it with custom post-processing, batch job queues, or webhook notifications.

Start by transcribing a few test files to calibrate model accuracy against your typical audio sources. From there, explore model quantization options and GPU acceleration to optimize for your hardware. The privacy benefits alone make this deployment worthwhile — your voice data stays on your hardware, period.

FAQ

Q: Can I use this with other Whisper implementations like WhisperX or whisper.cpp?

Yes. The fedirz/faster-whisper-server image is built on faster-whisper, but you can swap it for onerahmet/openai-whisper-asr-webservice (original Whisper) or build a custom image from whisper.cpp. Each has different API contracts — check their documentation for endpoint compatibility.

Q: How do I transcribe files larger than 25MB?

The default API has no strict file size limit, but memory usage scales with audio length. For very long recordings (over 2 hours), split the audio into chunks using ffmpeg before sending to the API. Alternatively, mount the file into the container and process it directly as shown in Step 9.

Q: What's the best model for non-English languages?

The large-v3 model offers the best multilingual accuracy, but medium is a good compromise for resource-constrained hosts. Set the language parameter in your API requests to the target language code (e.g., fr, de, ar) to improve accuracy and speed.

Q: Can I run multiple model sizes simultaneously?

The container downloads models on-demand, so you can switch between sizes by restarting with different environment variables. For concurrent use of multiple models, run separate containers on different ports with different WHISPER_MODEL values.

Q: How do I update the service when a new version is released?

Check the project's GitHub releases page for the latest image tag, update the image: line in your Compose file, then run docker compose pull && docker compose up -d. Your models and config persist in the mounted volumes.