Text Generation WebUI Docker Setup: Secure Self-Hosted LLM Interface

Introduction

Running a local LLM interface is no longer a novelty; it's a practical privacy solution for teams and individuals who want full control over their prompts and data. Text Generation WebUI (often called oobabooga) is one of the most feature-rich frontends for this purpose, supporting multiple model backends, LoRA fine-tuning, and an OpenAI-compatible API. However, its Python-based setup can be fragile when you mix system dependencies, CUDA versions, and model weights.

This guide provides a production-oriented Docker Compose deployment that isolates the application, simplifies upgrades, and keeps your host clean. You will learn how to set up persistent volumes for models and logs, configure GPU passthrough, and secure the service behind a reverse proxy with SSL. We will also cover the most frequent errors people encounter—like CUDA mismatches and tokenizer issues—and give you copy-paste solutions.

By the end, you'll have a working instance on http://your-server:7860 with a clear path to add authentication and backup routines. All commands are written for Bash on a Linux host (Ubuntu 22.04/24.04 or Debian 12), but they should work on any system with Docker Engine 24+ and Docker Compose v2.

Version note: The image tag used below is latest. Check the official GitHub releases page before pinning a version — the version above may be outdated by now.

Prerequisites

Before you begin, verify your hardware and software against the following table. The numbers are typical ranges, not guarantees—actual performance depends on model size, context length, and GPU memory bandwidth.

NVIDIA Container Toolkit is mandatory for GPU access. Install it from the official repository, not from pip. After installation, test with docker run --rm --gpus all nvidia/cuda:11.8-base-ubuntu22.04 nvidia-smi.

Step-by-Step Installation

Step 1: Create Project Directory and .env File

Create a dedicated folder and a .env file to hold your secrets and version pins. Never commit this file to Git.

mkdir -p ~/textgen-webui && cd ~/textgen-webui && touch .env

Now edit .env with your editor. At minimum, set a strong admin password and a download token if you plan to use Hugging Face gated models.

# Inside .env
ADMIN_PASSWORD=change_this_to_a_long_random_string
HF_TOKEN=your_huggingface_token_optional
TEXTGEN_VERSION=latest

Warning: Keep .env outside version control. Add .env to your .gitignore immediately if you initialize a repo.

Step 2: Create docker-compose.yml

Create the main Compose file. We use the official image ghcr.io/oobabooga/text-generation-webui:latest. The service listens on port 7860 internally; we map it to host port 7860.

# docker-compose.yml
services:
  textgen:
    image: ghcr.io/oobabooga/text-generation-webui:${TEXTGEN_VERSION:-latest}
    container_name: textgen-webui
    restart: unless-stopped
    ports:
      - "7860:7860"
    volumes:
      - ./models:/app/models
      - ./loras:/app/loras
      - ./prompts:/app/prompts
      - ./extensions:/app/extensions
      - ./logs:/app/logs
    environment:
      - ADMIN_PASSWORD=${ADMIN_PASSWORD}
      - HUGGINGFACE_TOKEN=${HF_TOKEN}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

Step 3: Launch the Container

Start the service in detached mode. The first run will pull the image and initialize the model cache.

cd ~/textgen-webui && docker compose up -d

Check the logs to ensure CUDA is detected:

cd ~/textgen-webui && docker compose logs -f textgen

Look for lines like CUDA: True or Using device: cuda. If you see CPU-only, your GPU passthrough is misconfigured.

Step 4: Download a Model

The WebUI has a built-in model downloader. Navigate to http://localhost:7860, go to the Model tab, and paste a model name like TheBloke/Llama-2-7B-Chat-GGUF. For direct CLI download, exec into the container:

cd ~/textgen-webui && docker compose exec textgen python download-model.py TheBloke/Llama-2-7B-Chat-GGUF

Step 5: Configure Default Model Parameters

Create a config.yaml in the project root to set default generation settings. This file is read by the container on startup.

# config.yaml
model: TheBloke/Llama-2-7B-Chat-GGUF
max_seq_len: 4096
loader: llama.cpp
n_gpu_layers: 99

Restart the service to apply:

cd ~/textgen-webui && docker compose restart textgen

Step 6: Enable API Mode (Optional)

To use the OpenAI-compatible endpoint, add the --api flag. Modify the Compose file to override the default command:

# In docker-compose.yml under textgen service
command: ["--listen", "--api", "--api-port", "5000"]

Then restart. The API will be available at http://localhost:5000/v1.

Step 7: Set Up Persistent Logging

Logs are stored in ./logs on the host. To rotate them, add a logrotate config on your host:

sudo tee /etc/logrotate.d/textgen << 'EOF'
/home/USERNAME/textgen-webui/logs/*.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
}
EOF

Replace USERNAME with your actual home directory name.

Step 8: Update the Container

When a new version is released, pull and recreate:

cd ~/textgen-webui && docker compose pull && docker compose up -d

Advanced Configuration / Optimization

Reverse Proxy with SSL (Caddy)

Expose the service securely using Caddy as a reverse proxy. Create a separate docker-compose.proxy.yml or add a second service to your existing file:

# docker-compose.proxy.yml
services:
  caddy:
    image: caddy:2-alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
    restart: unless-stopped

volumes:
  caddy_data:

Create a Caddyfile with your domain:

textgen.example.com {
    reverse_proxy textgen:7860
}

Backups

Back up the models and prompts directories. Use rsync to a remote location or a separate disk:

rsync -avz --progress ~/textgen-webui/models /mnt/backup/

Optional Hardening

Warning: These settings restrict container capabilities and may break the app if copied blindly. Test in a staging environment first.

Add these to the textgen service in your Compose file only if you understand the implications:

security_opt:
  - no-new-privileges:true
read_only: true
tmpfs:
  - /tmp
cap_drop:
  - ALL
cap_add:
  - NET_BIND_SERVICE

read_only will likely break model downloads unless you mount a writable volume to /app/models. The tmpfs mount helps with runtime temp files.

Troubleshooting Common Errors

Conclusion

You now have a functional Text Generation WebUI instance running in Docker with persistent storage and optional API access. The setup isolates your host from Python dependency hell and makes upgrades as simple as a docker compose pull. For production, always put the service behind a reverse proxy with SSL and enforce strong authentication—the WebUI itself does not provide rate limiting.

Remember to pin your image version before relying on this in a critical workflow. The latest tag is convenient but unpredictable. Check the project's GitHub releases monthly and update deliberately.

FAQ

Q1: Can I use this setup with AMD GPUs?

Yes, but you must replace the image with a ROCm-compatible variant. The official project publishes rocm tags. Modify the image: line to ghcr.io/oobabooga/text-generation-webui:rocm-latest and remove the NVIDIA deploy section. Performance will vary; check the project's docs for supported cards.

Q2: How do I expose the API to other applications?

Add --api to the command as shown in Step 6. Then use http://localhost:5000/v1 as your OpenAI base URL. For external access, place the reverse proxy in front of port 5000 and add authentication via Caddy's basic_auth directive.

Q3: What is the expected RAM usage for a 7B model?

With 4-bit quantization, expect 6-8 GB of RAM for the model itself, plus overhead for context. With 16-bit, it's closer to 14 GB. These are typical estimates; actual values depend on your GPU VRAM offloading settings.

Q4: How do I add multiple models without filling my disk?

Use the models volume to store them. Delete unused models via the UI or directly in the models folder. You can also mount a network share (NFS) instead of a local directory if you have a NAS.

Q5: Is it safe to expose the WebUI directly to the internet?

No. The default server has no rate limiting or brute-force protection. Always use a reverse proxy with SSL and HTTP authentication. For extra security, add fail2ban on the proxy layer.