Self-Hosted AI on Homelab Hardware: A Realistic 2026 Guide to Requirements, Sizing, and Deployment

Introduction

The promise of self-hosted AI is seductive: full data ownership, no per-token fees, and the ability to fine-tune models on your private corpus. But the gap between a marketing blog and a working inference server is often paved with crashed OOM killers and thermal throttling. In 2026, the landscape has shifted: quantization techniques like GGUF and AWQ have made 7B-parameter models runnable on a 16GB RAM laptop, while new NPUs in consumer CPUs blur the line between CPU and GPU inference. Yet the most common failure I see in homelab forums is not a lack of compute—it's a mismatch between the model you want to run and the hardware you actually have.

This guide is not a benchmark farm. Instead, it gives you a decision framework: how to translate a model's parameter count and quantization level into concrete RAM and VRAM requirements, how to choose between CPU, GPU, and hybrid setups, and how to deploy a production-grade stack with Docker Compose that survives reboots and power outages. You'll learn to size a system for three popular workloads: local LLM chat (using Ollama), image generation (Stable Diffusion WebUI), and speech-to-text (Whisper).

By the end, you will be able to answer three questions: "Can my existing NUC run a 13B model?", "What is the minimum RAM for a 30B model with 4-bit quantization?", and "How do I expose these services safely through a reverse proxy?" We'll also cover the hard lessons: why you should never trust a random docker-compose.yml without checking version pins, and why your backup strategy must include model weights, not just container volumes.

A note on honesty: all performance figures in this article are typical ranges or estimated values based on community reports and vendor documentation. Real throughput depends on your exact CPU generation, memory bandwidth, GPU drivers, and model quantization. Always test with your own workload before committing to a hardware purchase.

Prerequisites / Requirements Table

The table below summarizes the hardware and software baseline for a single-user homelab AI server. This assumes you are running Linux (Ubuntu 22.04 LTS or newer) with Docker Engine 24+ and Docker Compose v2. If you are on Windows, use WSL2 with GPU passthrough—but expect additional latency.

Important: The above RAM figures are typical for single-stream inference. If you plan to run multiple models simultaneously or fine-tune (which requires more memory for gradients), add 50% to RAM and VRAM.

Installation and Setup Steps (10 Steps)

This section walks you through deploying an AI stack with three services: Ollama (LLM), Stable Diffusion WebUI (via AUTOMATIC1111 or Forge), and faster-whisper (speech-to-text). We'll use a single Docker Compose project, but you can split them. All commands are Bash; assume you are in a terminal on your homelab server.

Step 1: Create Project Directory and .env File

First, create a dedicated directory and a .env file for secrets and version pins. Never hardcode passwords in Compose files.

mkdir -p ~/homelab-ai && cd ~/homelab-ai && touch .env && chmod 600 .env

Now edit .env with your preferred editor (nano or vim). Fill in strong passwords. This file will be read by Docker Compose. Never commit .env to Git—it contains secrets.

cat > .env << 'EOF'
# Secrets - change these!
POSTGRES_PASSWORD=change_me_strong_password
OLLAMA_BASE_URL=http://ollama:11434
# Version pins - check official releases before updating
OLLAMA_VERSION=${OLLAMA_VERSION:-latest}
WEBUI_VERSION=${WEBUI_VERSION:-latest}
WHISPER_VERSION=${WHISPER_VERSION:-latest}
EOF

Warning: The version variables above are set to latest for demonstration. Before pinning a specific version, check the official GitHub releases page for each project (ollama/ollama, open-webui/open-webui, fedirz/faster-whisper-server) — the version above may be outdated by now.

Step 2: Prepare GPU Support (If You Have an NVIDIA GPU)

For GPU acceleration, install the NVIDIA Container Toolkit. If you have no GPU, skip to Step 3.

# On Ubuntu/Debian, after installing NVIDIA drivers (check nvidia-smi)
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list && sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit && sudo systemctl restart docker

Verify that Docker sees your GPU:

sudo docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi && echo "GPU OK"

If you get errors, check the official NVIDIA Container Toolkit documentation.

Step 3: Create the Base Docker Compose File

We'll create a docker-compose.yml that defines three services. We'll use named volumes for persistence. The Compose file is complete and copy-paste ready, but note the image: tags use ${VAR} from .env.

version: "3.8"

services:
  ollama:
    image: ollama/ollama:${OLLAMA_VERSION}
    container_name: ollama
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    restart: unless-stopped
    # Optional: enable GPU if you have NVIDIA
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: 1
    #           capabilities: [gpu]

  open-webui:
    image: ghcr.io/open-webui/open-webui:${WEBUI_VERSION}
    container_name: open-webui
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY:-change_me}
    volumes:
      - webui_data:/app/backend/data
    ports:
      - "3000:8080"
    restart: unless-stopped

  whisper:
    image: fedirz/faster-whisper-server:${WHISPER_VERSION}
    container_name: whisper
    command: --model small --device auto
    volumes:
      - whisper_data:/root/.cache
    ports:
      - "9000:8000"
    restart: unless-stopped

volumes:
  ollama_data:
  webui_data:
  whisper_data:

Note: The WEBUI_SECRET_KEY is used to sign session cookies. Add it to your .env file (the example uses a default fallback, but you should set a random long string). We did not put it in the Compose file directly; it's referenced from environment.

Step 4: Pull and Start the Services

Before starting, ensure your .env is loaded by Compose automatically (it reads .env in the same directory). Then pull images and start.

cd ~/homelab-ai && docker compose pull && docker compose up -d

Check that all containers are running:

docker compose ps && docker compose logs --tail=20 ollama

If you see errors, proceed to the Troubleshooting section.

Step 5: Download and Run Your First LLM with Ollama

Now pull a small model to test. The llama3.2:1b is a 1.3B parameter model that runs on CPU with 8GB RAM. For GPU, try llama3.2:3b.

# Execute inside the ollama container
docker exec -it ollama ollama pull llama3.2:1b && docker exec -it ollama ollama run llama3.2:1b "Hello, why is self-hosting AI important?"

Expected: a short answer. If you get a timeout, increase OLLAMA_NUM_PARALLEL or adjust environment variables.

Step 6: Integrate Open WebUI with Ollama

Open WebUI is automatically connected to Ollama via the OLLAMA_BASE_URL environment variable. Access http://your-server-ip:3000, create an admin account, and you should see the model you pulled. You can also upload documents for RAG later.

Step 7: Set Up Stable Diffusion WebUI (Optional but Recommended)

For image generation, we'll add a second Compose service. This requires a GPU with at least 6GB VRAM. Append the following to your docker-compose.yml under services:

  sd-webui:
    image: ghcr.io/automatic1111/stable-diffusion-webui:latest
    container_name: sd-webui
    ports:
      - "7860:7860"
    volumes:
      - sd_data:/app/data
    environment:
      - CLI_ARGS=--medvram --xformers
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped

volumes:
  sd_data:

Note: The image tag latest is used; check the AUTOMATIC1111 docker repo for a pinned version. Also, --medvram is a typical flag for 6-8GB cards; adjust based on your VRAM.

Then start the new service:

cd ~/homelab-ai && docker compose up -d sd-webui

Access http://your-server-ip:7860. Download a checkpoint (e.g., SD 1.5 or SDXL) from Hugging Face and place it in the sd_data volume—you'll need to copy it into the container or mount a host folder. This is beyond our scope, but the official wiki explains it.

Step 8: Add Whisper for Speech-to-Text

Our whisper service is already running. Test it by sending an audio file. You'll need to install curl and have an audio file ready.

curl -X POST http://localhost:9000/transcribe -F "file=@your-audio.mp3" -F "model=small"

Returns JSON with transcribed text. You can also use the OpenAI-compatible endpoint for integration with other tools.

Step 9: Configure a Reverse Proxy (Caddy) for Secure Access

For secure remote access, add a Caddy service that provides HTTPS and route to your services. We'll use Caddy's automatic TLS. Add this to your docker-compose.yml:

  caddy:
    image: caddy:2.8.4
    container_name: caddy
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - caddy_data:/data
      - caddy_config:/config
      - ./Caddyfile:/etc/caddy/Caddyfile
    restart: unless-stopped

volumes:
  caddy_data:
  caddy_config:

Create a Caddyfile in the same directory:

ai.example.com {
    reverse_proxy open-webui:8080
}

whisper.example.com {
    reverse_proxy whisper:8000
}

Replace example.com with your domain and point DNS to your server. Then start Caddy:

cd ~/homelab-ai && docker compose up -d caddy

Caddy will automatically obtain SSL certificates. For LAN-only use, you can skip this and just use IP:port.

Step 10: Set Up Automated Backups

Backup the volumes that contain critical data: ollama_data (model weights), webui_data (chat history, user data), and sd_data (generated images). Use a cron job to tar the volumes.

# Create a backup script
cat > ~/homelab-ai/backup.sh << 'EOF'
#!/bin/bash
BACKUP_DIR=~/homelab-ai/backups
DATE=$(date +%Y%m%d)
mkdir -p $BACKUP_DIR
docker run --rm --volumes-from ollama -v $BACKUP_DIR:/backup alpine tar czf /backup/ollama_$DATE.tar.gz /root/.ollama
docker run --rm --volumes-from open-webui -v $BACKUP_DIR:/backup alpine tar czf /backup/webui_$DATE.tar.gz /app/backend/data
# Add similar for sd-webui if you have it
EOF
chmod +x ~/homelab-ai/backup.sh && echo "0 3 * * * ~/homelab-ai/backup.sh" | crontab -

Advanced Configuration & Optimization

Reverse Proxy and SSL (Caddy)

We already covered Caddy. For more control, use Traefik or Nginx Proxy Manager. Ensure you set WEBUI_SECRET_KEY properly and enable CORS if you call APIs from external apps.

Performance Tuning for Ollama

Ollama's default settings are conservative. To increase throughput, set environment variables in docker-compose.yml for the ollama service:

    environment:
      - OLLAMA_NUM_PARALLEL=1  # for concurrency
      - OLLAMA_MAX_LOADED_MODELS=1
      - OLLAMA_KEEP_ALIVE=5m

For CPU inference, consider using OLLAMA_INTEL_GPU if you have an iGPU, but this is experimental. For GPU, install the NVIDIA toolkit as shown.

Backups with Restore Script

Create a restore script to extract the tarballs back into volumes. Test it once.

Security Hardening: Optional

Warning: The following settings are advanced. They may break container functionality if copied blindly. Adjust them per service and test thoroughly.

For services that don't need outbound network (like ollama if you pull models manually), you can add network_mode: none to isolate them. But you'll need to pull models first. Alternatively, use read_only: true for the container filesystem, but then you need to mount /tmp as writable. Example for whisper:

  whisper:
    ...
    security_opt:
      - no-new-privileges:true
    read_only: true
    tmpfs:
      - /tmp
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETUID
      - SETGID
      - DAC_OVERRIDE

This is not a one-size-fits-all. Test each container after applying. Also, never run containers as root; use user: "1000:1000" but verify with id -u and id -g on your host and check the image's documentation for default user (e.g., Ollama image runs as root by default, so you may need to create a non-root user inside).

Troubleshooting Common Issues

Conclusion

Self-hosting AI is not about having the biggest GPU—it's about matching your workload to your hardware, then building a reliable stack around it. We've covered how to estimate memory requirements from model size and quantization, how to set up a multi-service Docker Compose environment for LLM, image generation, and transcription, and how to expose it securely. The key takeaway: start small (7B models), measure your actual resource usage with docker stats, and scale up only when you hit bottlenecks. Your homelab is a learning environment—document your configs, back up your volumes, and always check for updated images.

FAQ

Q1: Can I run a 13B model on a 16GB RAM machine without a GPU?

Yes, but only with 4-bit quantization and even then it will be slow—expect a few tokens per second. The model weights take ~8GB, leaving 8GB for OS and overhead. You'll need to close other applications. For a smoother experience, use a 7B model or add a GPU.

Q2: What is the difference between GGUF and AWQ quantization?

GGUF is a format that runs on CPU and GPU (via llama.cpp) and supports many quantization levels (q4_0, q5_1). AWQ is a GPU-only quantization that preserves more accuracy at low bit-widths but requires a GPU. Choose GGUF for CPU-only, AWQ for high-end GPUs.

Q3: How do I update my models and containers safely?

For containers, pull the new image and docker compose up -d — but first backup your volumes. For models, use ollama pull to get the latest. Always read the changelog for breaking changes.

Q4: Is it safe to expose my AI services to the internet?

Only if you use a reverse proxy with TLS and strong authentication. Open WebUI has built-in auth, but you should also enable rate limiting and consider VPN for sensitive data. Never expose raw Ollama port 11434 without a proxy.

Q5: How do I add a new model to Ollama?

Use ollama pull modelname inside the container. You can find models on the Ollama library. For custom models, you can create a Modelfile and use ollama create. Remember to adjust RAM if the model is larger.