Ollama API Mastery: Production-Ready Self-Hosted LLM Deployment Guide

Introduction

Serving large language models (LLMs) locally has shifted from a hobbyist experiment to a practical infrastructure decision. With growing concerns over data privacy, API costs, and vendor lock-in, the Ollama API has emerged as a leading solution for self-hosted inference. Ollama provides a streamlined way to run models like Llama 3, Mistral, and Qwen on your own hardware, exposing a clean REST API that integrates seamlessly with tools like Open WebUI, LangChain, or custom scripts.

This guide is not a basic "hello world" tutorial. It is a production-focused walkthrough for deploying Ollama v0.33.2 (the latest official release as of 2026-08-27) inside a Docker Compose stack on your homelab. You will learn how to move beyond a simple test container to a resilient setup that survives reboots, handles concurrent requests, and secures your API from unauthorized access.

The core of this guide is a complete, copy-paste-ready docker-compose.yml file, followed by meticulous configuration steps. We will cover environment variable management, persistent storage, GPU passthrough (NVIDIA), and health checks. Beyond the basic install, we will dive into advanced topics such as reverse proxy integration with Traefik or Nginx, SSL termination, automated backup strategies, and a dedicated section on security hardening.

By the end of this 1400+ word guide, you will have a running Ollama instance that is not only functional but also maintainable and secure. You will understand the "why" behind each configuration choice, not just the "how," enabling you to adapt this setup to evolving Ollama releases. Let's begin by ensuring your hardware and software are ready for the task.

Prerequisites / Requirements

Before you start executing commands, you must verify that your homelab environment meets the minimum requirements. Ollama is resource-intensive, and underestimating requirements is the most common cause of failure. The table below outlines the expected specifications. Remember, these are estimated ranges; actual consumption varies wildly based on the model size, context window length, concurrent request count, and hardware acceleration.

Critical Pre-flight Checks:

1. Docker Installed: Run docker --version and docker compose version in your terminal. If not installed, follow the official Docker installation docs for your OS.
2. User Permissions: Your user must be in the docker group to run commands without sudo. Run sudo usermod -aG docker $USER and log out/in.
3. NVIDIA GPU Support: If you have an NVIDIA GPU, you must install the nvidia-container-toolkit first. Do not proceed to the Docker Compose step until this is done. Verify with nvidia-smi (should show your GPU) and docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi (should show your GPU inside the container).

Step-by-Step Installation and Configuration

This section provides a precise, numbered workflow. Execute these commands in your terminal. We will create a dedicated directory for Ollama to keep your homelab organized.

Step 1: Create Project Directory and Environment File

First, we create a directory structure and a .env file. This file will store all our secrets and configurable variables, keeping them out of the docker-compose.yml file.

mkdir -p ~/ollama-stack && cd ~/ollama-stack && touch .env

Now, open the .env file with your text editor (e.g., nano .env). Paste the following content. CRITICAL: Never commit this file to Git. It contains your API password. Add .env to your .gitignore file if you are using version control.

# .env file content
# Ollama API Password - CHANGE THIS TO A STRONG, UNIQUE PASSWORD
OLLAMA_API_PASSWORD=YourSuperSecretPasswordHere123!

# Host port to expose the Ollama API on
OLLAMA_PORT=11434

# Model storage path on the host (persistent)
OLLAMA_MODELS_HOST_PATH=./ollama_models

# Timezone (optional, for logging)
TZ=UTC

Step 2: Create the Docker Compose File

Create a file named docker-compose.yml in the same directory. This file defines the Ollama service. Notice how we reference the variables from .env using ${VAR} syntax. We are pinning the image to a specific version v0.33.2 for reproducibility. Check the official GitHub releases page before pinning a version — the version above may be outdated by now. If you prefer to always pull the latest, change the image to ollama/ollama:latest.

cat > docker-compose.yml <<'EOF'
services:
  ollama:
    image: ollama/ollama:v0.33.2
    container_name: ollama
    restart: unless-stopped
    ports:
      - "${OLLAMA_PORT:-11434}:11434"
    volumes:
      - ${OLLAMA_MODELS_HOST_PATH:-./ollama_models}:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0:11434
      - OLLAMA_KEEP_ALIVE=5m
      - OLLAMA_MAX_LOADED_MODELS=1
      - OLLAMA_NUM_PARALLEL=1
      - OLLAMA_FLASH_ATTENTION=1
      - OLLAMA_API_PASSWORD=${OLLAMA_API_PASSWORD}
      - TZ=${TZ:-UTC}
    healthcheck:
      test: ["CMD", "ollama", "list"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    # Uncomment the next lines only if you have an NVIDIA GPU and installed the toolkit.
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: all
    #           capabilities: [gpu]
EOF

Step 3: Start the Ollama Container

Now, we pull the image and start the container in detached mode. The --env-file flag ensures Docker reads our .env file.

docker compose --env-file .env up -d

Step 4: Verify the Container is Running and Healthy

Check the status of the container. The healthcheck we defined will report healthy once the API is ready.

docker compose ps

You should see STATUS: Up X seconds (healthy). If it shows unhealthy, wait a few seconds and check the logs.

docker compose logs ollama

Step 5: Pull a Model

Ollama does not come with models pre-installed. You need to pull a model from the Ollama library. For this guide, we will pull a small model like llama3.2:1b to test the setup quickly. For production, you would likely choose a larger one like llama3.1:8b or qwen2.5:7b.

docker exec -it ollama ollama pull llama3.2:1b

Step 6: Test the API Endpoint

First, test the API locally from within the container to confirm it is listening.

docker exec -it ollama curl http://localhost:11434/api/tags

You should see a JSON response listing the model you just pulled. Now, test it from your host machine to verify port mapping is correct.

curl http://localhost:11434/api/tags

Step 7: Run a Generation Request

Let's send a prompt to the model to ensure the inference pipeline works. We will use the ollama CLI inside the container.

docker exec -it ollama ollama run llama3.2:1b "Why is the sky blue?"

Step 8: Configure API Password Authentication

Important: The Ollama API does not natively support password authentication. The environment variable OLLAMA_API_PASSWORD we set in Step 1 is not a standard Ollama variable. It is a placeholder for you to use with a reverse proxy (like Authelia or Basic Auth in Nginx) or a custom middleware. The container will start successfully with this variable set, but it will not be used by Ollama itself. You must implement authentication at the reverse proxy level. This is covered in the Advanced Setup section. Do not expose port 11434 directly to the internet without this proxy layer.

Step 9: Persistent Storage Verification

Ensure your models are persisted to the host volume. If you restart the container, the models should still be there.

docker compose restart && docker exec -it ollama ollama list

Step 10: Pull a Larger Model (Optional)

If you have the hardware, pull a production-scale model to test performance.

docker exec -it ollama ollama pull llama3.1:8b

Advanced Configuration and Optimization

Now that the basic container is running, let's harden it for production. This involves putting it behind a reverse proxy, enabling HTTPS, and setting up backups.

Reverse Proxy and SSL with Nginx

Exposing Ollama directly is a security risk. We will use Nginx as a reverse proxy to terminate SSL and add Basic Authentication. This assumes you have a domain name pointed to your homelab IP.

First, create an Nginx config file /etc/nginx/sites-available/ollama.

sudo nano /etc/nginx/sites-available/ollama

Paste the following configuration. Replace your-domain.com with your actual domain. The proxy_pass points to the Ollama container on the Docker network.

server {
    listen 80;
    server_name your-domain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name your-domain.com;

    ssl_certificate /etc/letsencrypt/live/your-domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/your-domain.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:11434;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Basic Authentication
        auth_basic "Restricted Access";
        auth_basic_user_file /etc/nginx/.htpasswd;
    }
}

Generate the password file using htpasswd (install apache2-utils if needed).

sudo htpasswd -c /etc/nginx/.htpasswd ollama_user

Enable the site and test the configuration.

sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/ && sudo nginx -t && sudo systemctl reload nginx

Now, access https://your-domain.com/api/tags with your browser. You should be prompted for a username and password.

Backups

Backing up Ollama is straightforward since all state (models) is in the volume you mapped. The simplest backup is to stop the container and copy the directory.

docker compose stop && tar -czvf ollama_backup_$(date +%Y%m%d).tar.gz ./ollama_models && docker compose start

For a zero-downtime backup, you can use rsync to copy the files while the container is running, but there is a small risk of corruption if a model is being written at that exact moment. The tar method is safer.

Security Hardening (Optional)

The following docker-compose.yml additions are for advanced users who want to enforce least-privilege principles. Warning: These settings are not universal. They may break the container if the image or application requires specific permissions. Test thoroughly before applying to production.

Add these to your docker-compose.yml under the ollama service:

    read_only: true
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE

• read_only: true makes the container filesystem read-only. Ollama needs to write to /root/.ollama, but since we mounted that as a volume, it can still write there. However, it might need to write to other temporary locations (e.g., /tmp). If it fails, you may need to add a tmpfs mount.
• cap_drop: ALL removes all Linux capabilities. NET_BIND_SERVICE is often needed to bind to port 80, but since we are mapping to port 11434, it might not be necessary. If you see permission errors, adjust the cap_add list.

A more compatible hardening approach is to use Docker's built-in --cap-drop and --read-only flags with a tmpfs for temporary files. Here is an alternative snippet:

    tmpfs:
      - /tmp

This allows Ollama to write temporary files while keeping the rest of the filesystem read-only.

Troubleshooting Common Issues

Even with a perfect guide, issues arise. The table below addresses the most frequently encountered problems in homelab deployments.

Conclusion and FAQ

Deploying Ollama v0.33.2 via Docker Compose is a robust way to bring state-of-the-art LLM capabilities into your homelab. You have moved past the basics by implementing persistent storage, health checks, and preparing for reverse proxy authentication. The key to a successful production deployment lies not in the initial docker compose up, but in the ongoing management of security, backups, and resource tuning. By using environment variables and pinning versions, you have made your setup reproducible and maintainable.

The troubleshooting table addresses the most common pitfalls, empowering you to solve issues without scouring forums. Remember that running LLMs is resource-intensive; monitor your system's RAM and disk usage regularly. The optional hardening steps show you how to enforce security best practices, but they require careful testing to ensure they do not break functionality. This guide provides a solid foundation—now it is up to you to adapt it to your specific hardware and use case, whether that is a personal chatbot, a code assistant, or an experimental agent.

Frequently Asked Questions

Q1: How do I update Ollama to a newer version?

To update, pull the new image and recreate the container. First, check the official GitHub releases for the latest version number. Then, edit your docker-compose.yml to change the image tag (e.g., from v0.33.2 to v0.34.0). Finally, run docker compose pull && docker compose up -d. Your models will persist because they are stored in the mounted volume. Always backup your models directory before a major update.

Q2: Can I use Ollama with an AMD GPU?

Yes, Ollama supports AMD GPUs via the ROCm stack. In your docker-compose.yml, you would need to use the ollama/ollama:rocm image variant and add a device mapping for your GPU. The configuration is more complex than NVIDIA's nvidia-container-toolkit. Consult the official Ollama documentation on AMD GPU support for the latest instructions, as the Docker integration is less seamless than NVIDIA's.

Q3: Is it safe to expose the Ollama API to the public internet?

No, it is not safe without a reverse proxy that implements authentication. The default Ollama API has no built-in authentication or rate limiting. Exposing it directly allows anyone to pull models, consume your GPU resources, or potentially access your network. Always place it behind Nginx, Traefik, or a similar tool with Basic Auth, OAuth2, or API keys. The OLLAMA_API_PASSWORD variable is not a native feature; it requires external middleware to enforce.

Q4: How can I increase the context window for a model?

The context window is set at the API request level, not the server level. When you make a request to the /api/generate or /api/chat endpoint, you can pass an options object with a num_ctx parameter. For example, using curl, you would add "options": {"num_ctx": 8192} to your JSON payload. The maximum value depends on the model's architecture and your available GPU memory. Setting it too high will cause out-of-memory errors.

Q5: What is the best way to monitor Ollama's resource usage?

You can use standard Docker monitoring tools. docker stats ollama gives you a real-time view of CPU and memory usage. For more detailed metrics and historical data, consider integrating cAdvisor with Prometheus and Grafana. Ollama also exposes basic metrics at /api/ps which shows loaded models and their memory usage. For homelab-level monitoring, docker stats is often sufficient for immediate troubleshooting.