Self-Hosted AI with AnythingLLM: A 2026 Production Deployment and Administration Guide

Executive Summary & Architecture Overview

AnythingLLM is a full-stack, open-source application that transforms any LLM (local or remote) into a private, RAG-powered AI assistant. It provides a unified chat interface, document management, and workspace isolation, all under your control. Self-hosting AnythingLLM eliminates data leakage to commercial SaaS providers, ensures compliance with internal data governance policies, and offers unlimited customization. You own your data, your usage logs, and your infrastructure.

In 2026, the verified stable release is v1.9.0 (released 2026-09-05). This guide walks you through a production-grade deployment using Docker Compose, from bare-metal preparation to zero-trust remote access and automated backups.

Architecture Overview:

• AnythingLLM Core: Serves the web UI and API, orchestrates chat sessions, manages workspaces, and coordinates with vector storage and LLM providers.
• Vector Database (LanceDB): Stores document embeddings for RAG. LanceDB is embedded by default, but we will use a dedicated container for scalability and backup simplicity.
• LLM Provider: Can be a local model via Ollama, or a remote API (OpenAI, Anthropic, etc.). This guide uses Ollama for full local inference.
• Embedder: Converts documents into vectors using a local model (e.g., nomic-embed-text) or API.
• Reverse Proxy: Routes external HTTPS traffic securely.
• Tailscale: Provides encrypted remote access without exposing public ports.

flowchart LR
    User[User] -->|HTTPS| RP[Reverse Proxy]
    RP -->|HTTP| ALM[AnythingLLM Container]
    ALM -->|HTTP| LanceDB[LanceDB Container]
    ALM -->|HTTP| Ollama[Ollama Container]
    Ollama -->|Model Weights| GPU[GPU/NPU]
    ALM -->|Backup| Backup[Backup Script]
    Backup -->|tarball| Offsite[Offsite Storage]
    User -->|Tailscale| Tailscale[Tailscale Node]
    Tailscale --> ALM

Hardware, OS & Network Requirements

Ports Used:

• 3001 (TCP): AnythingLLM web UI/API (internal, not exposed publicly).
• 11434 (TCP): Ollama API (internal).
• 80/443 (TCP): Reverse proxy (if exposed).
• 3478/41641 (UDP): Tailscale (outbound only).

Step 1: Host Preparation & Directory Layout

We'll use a standard layout under /opt/anythingllm. Create directories and set permissions:

sudo mkdir -p /opt/anythingllm/{data,hotdir,storage,backups,ollama}
sudo chown -R $USER:$USER /opt/anythingllm

Create the .env file with secure secrets:

cd /opt/anythingllm
openssl rand -hex 32  # generate a strong JWT secret
openssl rand -base64 32  # generate a strong password for LanceDB (if used)

Create /opt/anythingllm/.env with the following content (adjust paths as needed):

# AnythingLLM Environment Variables
# Directory paths inside container
STORAGE_DIR=/app/server/storage

# JWT Secret (use output of openssl rand -hex 32)
JWT_SECRET=replace_with_your_hex_secret

# Database connection (LanceDB)
LANCE_DB_PATH=/app/server/storage/lancedb

# Ollama settings (if using local LLM)
OLLAMA_BASE_URL=http://ollama:11434

# User timezone
timezone=America/New_York

# Container user/group (match your host UID/GID)
PUID=1000
PGID=1000

Step 2: Production-Grade docker compose.yaml

Create /opt/anythingllm/docker-compose.yaml with the following content. This uses the modern Compose Spec (no version key) and includes healthchecks, custom networks, and volume mounts.

services:
  anythingllm:
    image: mintplexlabs/anythingllm:latest
    container_name: anythingllm
    restart: unless-stopped
    ports:
      - "3001:3001"
    env_file:
      - .env
    environment:
      - STORAGE_DIR=${STORAGE_DIR}
      - JWT_SECRET=${JWT_SECRET}
      - LANCE_DB_PATH=${LANCE_DB_PATH}
      - OLLAMA_BASE_URL=${OLLAMA_BASE_URL}
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${timezone}
    volumes:
      - ./storage:/app/server/storage
      - ./hotdir:/app/server/hotdir
      - ./data:/app/server/data
    networks:
      - anythingllm_net
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3001/api/ping"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    security_opt:
      - no-new-privileges:true

  lance-db:
    image: lancedb/lancedb:latest
    container_name: lance-db
    restart: unless-stopped
    environment:
      - LANCE_DB_PATH=/data
    volumes:
      - ./lance-data:/data
    networks:
      - anythingllm_net
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ./ollama:/root/.ollama
    networks:
      - anythingllm_net
    # Uncomment to use GPU (NVIDIA)
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: all
    #           capabilities: [gpu]
    healthcheck:
      test: ["CMD", "ollama", "list"]
      interval: 30s
      timeout: 10s
      retries: 3

networks:
  anythingllm_net:
    driver: bridge

Important: The lance-db and ollama containers are optional if you use embedded LanceDB or remote LLMs. But for production, we isolate them for better backup and scaling.

Step 3: Deployment & Health Verification

Pull images and start in background:

cd /opt/anythingllm
docker compose pull
docker compose up -d

Check container status and logs:

docker compose ps
docker compose logs -f anythingllm

Wait for the healthcheck to pass (up to 40 seconds). Then access http://localhost:3001 in your browser to begin the onboarding wizard.

Onboarding Steps:

1. Create an administrator account (email + strong password).
2. Choose your LLM provider. For local, select Ollama and set the base URL to http://ollama:11434.
3. Pull a model (e.g., llama3.1:8b) via the Ollama container:

   docker exec ollama ollama pull llama3.1:8b
   

4. Configure the embedder (e.g., Ollama with nomic-embed-text).
5. Create a workspace and start chatting.

Step 4: Reverse Proxy, Domain & SSL Hardening

For secure external access, use a reverse proxy. We'll use Nginx Proxy Manager as an example, but Caddy or Traefik work similarly.

Option A: Nginx Proxy Manager

1. Add a proxy host for anythingllm.yourdomain.com.
2. Forward to http://anythingllm:3001 (if proxy on same Docker network) or http://localhost:3001.
3. Enable Websockets support.
4. Add custom headers:

   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   proxy_set_header X-Forwarded-Proto $scheme;
   proxy_set_header X-Forwarded-Host $host;
   

5. Request a Let's Encrypt SSL certificate and force HTTPS.

Option B: Caddy

Create a Caddyfile:

anythingllm.yourdomain.com {
    reverse_proxy anythingllm:3001 {
        header_up X-Forwarded-For {remote_host}
        header_up X-Forwarded-Proto {scheme}
        header_up Upgrade {http.request.header.Upgrade}
        header_up Connection {http.request.header.Connection}
    }
}

Caddy automatically provisions SSL certificates.

Hardening:

• Set HSTS header via proxy (e.g., add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;).
• Restrict access to admin endpoints by IP or use Tailscale (recommended).

Step 5: Zero-Trust Remote Access with Tailscale

Tailscale provides a secure VPN mesh, eliminating the need to open public ports.

1. Install Tailscale on your host:

   curl -fsSL https://tailscale.com/install.sh | sh
   sudo tailscale up --ssh
   

2. On your client devices (laptop, phone), install Tailscale and sign in to the same account.
3. Access AnythingLLM via its Tailscale IP (e.g., http://100.x.x.x:3001) or set up a MagicDNS name:

   sudo tailscale up --hostname=anythingllm
   

   Then browse to http://anythingllm:3001.

Best Practices:

• Do not expose the AnythingLLM port to the internet; rely solely on Tailscale.
• Use Tailscale ACLs to restrict which users can reach the service.
• For web access, use tailscale serve or tailscale funnel to expose the UI via HTTPS on your Tailscale network.

Step 6: Automated Backup & Disaster Recovery

Create a backup script that stops the containers, archives volumes, and compresses them. We'll use tar for simplicity, but you can adapt to Restic.

Create /opt/anythingllm/backup.sh:

#!/bin/bash
# AnythingLLM Backup Script

set -euo pipefail

BACKUP_DIR="/opt/anythingllm/backups"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
BACKUP_FILE="$BACKUP_DIR/anythingllm_backup_$TIMESTAMP.tar.gz"

# Stop containers to ensure consistent state
docker compose -f /opt/anythingllm/docker-compose.yaml stop

# Backup volumes (storage, data, lance-data, ollama) using tar
tar -czf "$BACKUP_FILE" \
    -C /opt/anythingllm \
    storage data lance-data ollama

# Restart containers
docker compose -f /opt/anythingllm/docker-compose.yaml start

# Optional: Encrypt with age or gpg
# gpg --encrypt --recipient your@email.com "$BACKUP_FILE"

# Cleanup old backups (keep last 7)
find "$BACKUP_DIR" -name "*.tar.gz" -mtime +7 -delete

echo "Backup completed: $BACKUP_FILE"

Make it executable and add to crontab:

chmod +x /opt/anythingllm/backup.sh
crontab -e

Add a daily schedule at 2 AM:

0 2 * * * /opt/anythingllm/backup.sh >> /var/log/anythingllm_backup.log 2>&1

Disaster Recovery:

# Restore from backup
cd /opt/anythingllm
tar -xzf /path/to/backup.tar.gz

docker compose up -d

Step 7: Deep Troubleshooting Matrix

Step 8: Frequently Asked Questions (FAQ)

1. How do I choose between local LLM and API providers?

For privacy, use local models like Llama 3.1 or Mistral via Ollama. For quality, APIs like GPT-4 may be better. Consider hybrid: local for sensitive data, API for generic tasks.

2. Should I use Watchtower for automatic updates?

Avoid automatic updates for AnythingLLM; new versions may introduce breaking changes. Instead, pin the image tag (e.g., mintplexlabs/anythingllm:1.9.0) and update manually after reading release notes.

3. Can I migrate from an existing AnythingLLM installation?

Yes. Backup the storage and data directories, then restore them in the new installation. Ensure the vector database schema is compatible; if not, re-index documents.

4. How do I optimize memory usage for large document collections?

Use external LanceDB and set LANCE_DB_PATH to a fast NVMe. Increase OLLAMA_NUM_PARALLEL if using multiple GPUs. Consider chunking documents and using a smaller embedding model.

5. Is it safe to expose AnythingLLM to the internet via reverse proxy?

It can be safe if you enforce strong authentication, SSL, and rate limiting. However, for a homelab, Tailscale is strongly recommended to minimize attack surface.

6. How do I back up only the vector database?

Stop the LanceDB container and copy the lance-data volume. The backup script above includes it.

Conclusion

You now have a production-ready AnythingLLM deployment with proper security, remote access, and backup strategies. Enjoy your private AI assistant in 2026.