Ollama & Open WebUI: Complete Self-Hosted AI Stack Guide (2026)

Executive Summary & Architecture Overview

Running Large Language Models (LLMs) locally has shifted from an experimental hobby to an essential infrastructure requirement for privacy-conscious engineers, organizations, and homelabbers. Commercial AI APIs introduce continuous recurring subscription costs, rate limiting, and severe data privacy risks as enterprise intellectual property and personal prompts are transmitted to third-party data centers.

By pairing Ollama as the backend model execution daemon with Open WebUI (formerly Ollama WebUI) as the feature-rich frontend interface, you deploy a private, local alternative to ChatGPT that runs completely within your own local area network.

+-----------------------------------------------------------------------+
|                         Client Web Browser                            |
+-----------------------------------^-----------------------------------+
                                    | HTTP / WebSockets (Port 3000)
+-----------------------------------v-----------------------------------+
|               Nginx Proxy Manager / Tailscale Zero Trust              |
+-----------------------------------^-----------------------------------+
                                    | Internal Docker Bridge Network
+-----------------------------------v-----------------------------------+
|      Open WebUI Container (ghcr.io/open-webui/open-webui:main)         |
|      - Authentication (OAuth/Local)                                   |
|      - RAG (Retrieval-Augmented Generation) & ChromaDB                |
|      - Chat history & Prompt engineering templates                    |
+-----------------------------------^-----------------------------------+
                                    | REST API (Port 11434)
+-----------------------------------v-----------------------------------+
|            Ollama Daemon Container (ollama/ollama:{$tag})            |
|            - GGML / GGUF model execution runtime                      |
|            - GPU Acceleration (NVIDIA CUDA / AMD ROCm)                |
|            - Model storage volume (/root/.ollama)                     |
+-----------------------------------------------------------------------+

Hardware, OS & Network Requirements

Running modern quantizations (Q4_K_M or Q8_0) requires adequate VRAM or unified memory. CPU inference is supported but significantly slower.

Step 1: Host Preparation & Directory Layout

Create a dedicated application directory within /opt or your user home directory to ensure isolation and straightforward backups:

sudo mkdir -p /opt/ai-stack/{ollama_data,webui_data} && \
cd /opt/ai-stack && \
sudo chown -R $USER:$USER /opt/ai-stack && \
chmod 755 /opt/ai-stack

Verify that Docker and the Docker Compose plugin are properly installed on your host:

docker compose version && docker version

If utilizing NVIDIA GPUs, ensure the NVIDIA Container Toolkit is installed and tested:

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Create your .env environment file:

cat << 'EOF' > /opt/ai-stack/.env
# Secret key used to encrypt Open WebUI session cookies
WEBUI_SECRET_KEY=176f9d2e482b9a7c64a385f9e21dc0ba89234b65cd71829e01fb34aa98124efb

# Default Ollama base URL internal to docker network
OLLAMA_BASE_URL=http://ollama:11434

# Enable/Disable Open WebUI user signups (set to False after creating your admin account)
ENABLE_SIGNUP=true

# Ollama Keep-Alive time for models in memory (e.g., 30m, 2h, -1 for infinite)
OLLAMA_KEEP_ALIVE=24h
EOF

Step 2: Production-Grade docker compose.yaml

Save the following configuration as /opt/ai-stack/compose.yaml. Notice that under modern Docker Compose Specification V2, no top-level version: key is used.

services:
  ollama:
    image: ollama/ollama:{$tag}
    container_name: ollama
    restart: unless-stopped
    tty: true
    environment:
      - OLLAMA_KEEP_ALIVE=\${OLLAMA_KEEP_ALIVE:-24h}
      - OLLAMA_HOST=0.0.0.0:11434
    volumes:
      - /opt/ai-stack/ollama_data:/root/.ollama
    networks:
      - ai-network
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:8080"
    environment:
      - OLLAMA_BASE_URL=\${OLLAMA_BASE_URL:-http://ollama:11434}
      - WEBUI_SECRET_KEY=\${WEBUI_SECRET_KEY}
      - ENABLE_SIGNUP=\${ENABLE_SIGNUP:-true}
    volumes:
      - /opt/ai-stack/webui_data:/app/backend/data
    depends_on:
      ollama:
        condition: service_started
    networks:
      - ai-network

networks:
  ai-network:
    name: ai-network
    driver: bridge

Security Note: Notice that Open WebUI binds to 127.0.0.1:3000:8080. This prevents raw unencrypted access from your local network and enforces routing through our SSL reverse proxy or Tailscale VPN.

Step 3: Deployment & Health Verification

Launch the stack in detached mode:

cd /opt/ai-stack && docker compose up -d

Monitor the container startup logs:

docker compose logs -f --tail=50

Verify that both containers are running in a healthy state:

docker compose ps

Pull your primary foundational LLM directly through the Ollama container CLI:

# Pull Llama 3.1 8B (balanced general model)
docker exec -it ollama ollama pull llama3.1:8b

# Pull DeepSeek-Coder-V2 or Qwen 2.5 for programming tasks
docker exec -it ollama ollama pull qwen2.5-coder:7b

Access the web interface at http://YOUR_SERVER_IP:3000 (or via localhost) and register your first account. The first registered account automatically receives Administrator privileges. Once registered, immediately edit /opt/ai-stack/.env to set ENABLE_SIGNUP=false and run docker compose up -d to block unauthorized account creations.

Step 4: Reverse Proxy, Domain & SSL Hardening

Never expose your AI interface over raw plaintext HTTP. If using Nginx Proxy Manager:

1. Log into your NPM dashboard (http://npm-host:81).
2. Navigate to Hosts -> Proxy Hosts -> Add Proxy Host.
3. Configure the details:
   • Domain Names: ai.yourhomelab.net
   • Scheme: http
   • Forward Hostname / IP: Your server's local IP or Docker host gateway (e.g. 192.168.1.100 or 172.17.0.1).
   • Forward Port: 3000
   • Check: Block Common Exploits
   • Check: Websockets Support (MANDATORY for token streaming).
4. Under the SSL Tab:
   • Request a new SSL Certificate with Let's Encrypt.
   • Check Force SSL, HTTP/2 Support, and HSTS Enabled.
5. Under Custom Nginx Configuration, add buffer streaming timeouts:

proxy_buffering off;
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;

Step 5: Zero-Trust Remote Access with Tailscale

If you lack a static public IP or wish to avoid opening router inbound ports, use Tailscale:

1. Install Tailscale on your host machine:

curl -fsSL https://tailscale.com/install.sh | sh && sudo tailscale up

2. Retrieve your server's Tailscale IPv4 address:

tailscale ip -4

3. You can now access your Open WebUI dashboard from your laptop, smartphone, or tablet anywhere in the world by navigating to http://100.x.y.z:3000 while connected to your Tailnet, completely shielded from internet scrapers.

Step 6: Automated Backup & Disaster Recovery

Create an automated backup script /opt/ai-stack/backup.sh that archives model configurations, user chats, prompts, and document embeddings:

#!/usr/bin/env bash
set -euo pipefail

BACKUP_DIR="/mnt/backups/ai-stack"
DATE=$(date +"%Y%m%d_%H%M%S")
TARGET_DIR="/opt/ai-stack"

mkdir -p "\${BACKUP_DIR}"

echo "[+] Starting AI Stack Backup: \${DATE}"

# Backup WebUI database and vector stores
tar -czf "\${BACKUP_DIR}/webui_data_\${DATE}.tar.gz" -C "\${TARGET_DIR}" webui_data .env

# Retain only last 7 days of backups
find "\${BACKUP_DIR}" -type f -name "*.tar.gz" -mtime +7 -delete

echo "[✓] Backup completed successfully."

Make the script executable and schedule it via cron:

chmod +x /opt/ai-stack/backup.sh && \
(crontab -l 2>/dev/null; echo "0 3 * * * /opt/ai-stack/backup.sh >> /var/log/ai-backup.log 2>&1") | crontab -

Step 7: Deep Troubleshooting Matrix

Step 8: Frequently Asked Questions (FAQ)

1. Can I run this stack without a dedicated GPU?

Yes. Ollama will automatically fallback to CPU inference if no GPU is detected. Ensure your CPU supports AVX2 instructions. Expect inference speeds around 4-10 tokens/second on modern 8-core CPUs for 7B parameter models.

2. How do I update Ollama and Open WebUI safely?

Pull the latest images and recreate the containers without losing data, since all state is stored in persistent host volumes:

cd /opt/ai-stack && docker compose pull && docker compose up -d --remove-orphans

3. How does Open WebUI handle RAG (Document Question Answering)?

Open WebUI features a built-in embedding engine (all-MiniLM-L6-v2) and ChromaDB vector store. Uploaded PDFs or documents are chunked, embedded locally, and queried through vector similarity before passing context to Ollama.

4. Can I share models across multiple external tools?

Yes. Ollama exposes a standard OpenAI-compatible API endpoint at http://YOUR_SERVER_IP:11434/v1. You can plug this endpoint into third-party tools like Continue.dev in VS Code, Obsidian, or Home Assistant.

5. How can I keep models loaded permanently in memory?

Set the environment variable OLLAMA_KEEP_ALIVE=-1 in your compose file or .env. This tells Ollama never to unload model weights from VRAM between queries, eliminating loading latency.