Torchruntime API Examples: Production-Grade Self-Hosted AI Deployment Guide (2026)

1. Executive Summary & Architecture Overview

Torchruntime API Examples is a self-hosted toolkit that provides a curated set of runnable examples and a RESTful API wrapper for PyTorch-based models, allowing you to deploy, test, and integrate AI inference endpoints without relying on commercial SaaS or cloud AI services. By self-hosting, you retain full control over your data, model weights, and inference logs—eliminating per-token costs and privacy leaks to third-party providers. The architecture is modular: a web frontend (optional) serves interactive documentation and test consoles, while the core API service runs behind a reverse proxy, with PostgreSQL for metadata storage and Redis for caching and rate limiting. All components are containerized using Docker Compose, enabling reproducible deployments across homelab environments.

Architecture Flow:

1. Client (browser or API consumer) → 2. Reverse Proxy (Caddy or Nginx Proxy Manager) terminates TLS and forwards to the Torchruntime API container. → 3. API container handles requests, executes example scripts, and communicates with PostgreSQL (for storing user-defined examples) and Redis (for session caching). → 4. Model artifacts stored on a named volume, optionally with GPU passthrough for inference acceleration.

---

2. Hardware, OS & Network Requirements

Ports Used:

• Inbound (public/private): 80 (HTTP, redirected), 443 (HTTPS), 8443 (Tailscale HTTPS if used).
• Internal (Docker network): 8000 (API), 5432 (PostgreSQL), 6379 (Redis).

All external exposure should be via a reverse proxy on 443 only; never expose internal ports directly.

---

3. Step 1: Host Preparation & Directory Layout

We'll use the standard /opt/homelab/torchruntime layout. First, create the directory structure and set proper permissions.

sudo mkdir -p /opt/homelab/torchruntime/{data,models,logs,backups}
sudo chown -R $USER:$USER /opt/homelab/torchruntime

Set the PUID and PGID environment variables to your current user's UID/GID to avoid permission issues with mounted volumes:

echo "PUID=$(id -u)" >> /opt/homelab/torchruntime/.env
echo "PGID=$(id -g)" >> /opt/homelab/torchruntime/.env

Now, generate strong secrets for database passwords and Redis authentication:

openssl rand -hex 32   # for POSTGRES_PASSWORD and REDIS_PASSWORD
openssl rand -hex 16   # for a session secret

Copy these values into the .env file as shown in the next section.

---

4. Step 2: Production-Grade docker compose.yaml

Create /opt/homelab/torchruntime/docker-compose.yaml with the following content. Note: no top-level version key—this is obsolete in the Compose Spec.

services:
  api:
    image: torchruntime/api:2026.09.05
    container_name: torchruntime-api
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ:-UTC}
      - DATABASE_URL=postgresql://torchuser:${POSTGRES_PASSWORD}@db:5432/torchruntime
      - REDIS_URL=redis://:${REDIS_PASSWORD}@redis:6379/0
      - SESSION_SECRET=${SESSION_SECRET}
      - MODEL_DIR=/models
    volumes:
      - ./models:/models
      - ./logs:/logs
    ports:
      - "127.0.0.1:8000:8000"
    networks:
      - torchnet
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s

  db:
    image: postgres:16-alpine
    container_name: torchruntime-db
    restart: unless-stopped
    environment:
      - POSTGRES_USER=torchuser
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=torchruntime
    volumes:
      - db_data:/var/lib/postgresql/data
    networks:
      - torchnet
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U torchuser -d torchruntime"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    container_name: torchruntime-redis
    restart: unless-stopped
    command: ["redis-server", "--requirepass", "${REDIS_PASSWORD}"]
    volumes:
      - redis_data:/data
    networks:
      - torchnet
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  db_data:
  redis_data:

networks:
  torchnet:
    driver: bridge

Create the .env file in the same directory (or use .env.example as a template):

# Torchruntime API .env example
# Copy to .env and fill in values

PUID=1000
PGID=1000
TZ=UTC

# Generate with: openssl rand -hex 32
POSTGRES_PASSWORD=CHANGE_ME_32_HEX
REDIS_PASSWORD=CHANGE_ME_32_HEX
SESSION_SECRET=CHANGE_ME_16_HEX

Important: Replace all CHANGE_ME_* values with the outputs of the openssl commands from Step 1.

---

5. Step 3: Deployment & Health Verification

Pull and start the stack in the background:

cd /opt/homelab/torchruntime
docker compose up -d

Check container status:

docker compose ps

Monitor logs for any startup errors:

docker compose logs -f api

Once the API is healthy (wait for the healthcheck), access the initial onboarding wizard at http://localhost:8000. The wizard will ask you to create an admin account and optionally import example models. Follow the prompts to complete the setup.

To test the API with a simple example, use curl:

curl http://localhost:8000/api/v1/examples

This should return a JSON list of available example scripts.

---

6. Step 4: Reverse Proxy, Domain & SSL Hardening

For secure external access, we'll use Caddy because it automatically obtains and renews Let's Encrypt certificates. Create a Caddyfile on your host (or in a separate container).

torchruntime.example.com {
    reverse_proxy 127.0.0.1:8000
    encode zstd gzip
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "DENY"
    }
}

If using Nginx Proxy Manager, configure a proxy host with the following custom headers and WebSocket support:

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;

For Cloudflare Tunnel, create a tunnel that routes torchruntime.example.com to http://localhost:8000 with HTTP/2 origin support.

Hardening:

• Always force HTTPS (Caddy does this by default).
• Restrict the API to listen only on 127.0.0.1:8000 (as in our compose file) so only the proxy can reach it.
• Set session_secret to a long random value (already done).

---

7. Step 5: Zero-Trust Remote Access with Tailscale

Tailscale provides a secure WireGuard-based mesh VPN, allowing you to access your Torchruntime API from anywhere without exposing any public router ports.

1. Install Tailscale on the host (if not already):

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

2. Add the Torchruntime API to your Tailscale network by making it accessible only over the tailnet. Since the API is bound to 127.0.0.1, we need to expose it via a Tailscale sidecar or use Tailscale's --hostname feature. The simplest approach is to run a Tailscale sidecar container that forwards a specific port.

Create a docker-compose.tailscale.yaml:

services:
  tailscale:
    image: tailscale/tailscale:latest
    container_name: tailscale-sidecar
    hostname: torchruntime
    environment:
      - TS_AUTHKEY=${TS_AUTHKEY}  # from Tailscale admin console
      - TS_EXTRA_ARGS=--advertise-tags=tag:homelab
      - TS_STATE_DIR=/var/lib/tailscale
    volumes:
      - tailscale_state:/var/lib/tailscale
      - /dev/net/tun:/dev/net/tun
    cap_add:
      - NET_ADMIN
    sysctls:
      - net.ipv4.ip_forward=1
    networks:
      - torchnet
    restart: unless-stopped

volumes:
  tailscale_state:

3. Add the sidecar to the same network by running docker compose -f docker-compose.yaml -f docker-compose.tailscale.yaml up -d. Then, in the Tailscale admin console, enable MagicDNS and create a tailnet policy that allows access to port 8000 only from authorized users.

4. Access the API from any device via http://torchruntime:8000 (if MagicDNS is enabled) or via the Tailscale IP.

This method ensures no public exposure—only devices in your tailnet can reach the service.

---

8. Step 6: Automated Backup & Disaster Recovery

Create /opt/homelab/torchruntime/backup.sh with the following content:

#!/bin/bash
# Production backup script for Torchruntime API

set -euo pipefail

BACKUP_DIR="/opt/homelab/torchruntime/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${BACKUP_DIR}/torchruntime_${TIMESTAMP}.tar.gz"

# Stop the API to ensure consistent state
docker compose -f /opt/homelab/torchruntime/docker-compose.yaml stop api

# Dump PostgreSQL database
docker compose -f /opt/homelab/torchruntime/docker-compose.yaml exec -T db pg_dump -U torchuser -d torchruntime > "${BACKUP_DIR}/db_${TIMESTAMP}.sql"

# Backup Redis data (optional, but good for cache consistency)
docker compose -f /opt/homelab/torchruntime/docker-compose.yaml exec -T redis redis-cli -a "${REDIS_PASSWORD}" SAVE

# Compress all data volumes and the SQL dump into one archive
tar -czf "${BACKUP_FILE}" -C /opt/homelab/torchruntime \
    data/ \
    models/ \
    logs/ \
    "${BACKUP_DIR}/db_${TIMESTAMP}.sql"

# Remove the raw SQL dump (already in archive)
rm "${BACKUP_DIR}/db_${TIMESTAMP}.sql"

# Restart the API
docker compose -f /opt/homelab/torchruntime/docker-compose.yaml start api

# Optional: sync to offsite location with rclone or rsync
# rsync -avz "${BACKUP_FILE}" user@offsite:/backups/

echo "Backup completed: ${BACKUP_FILE}"

Make it executable and test it:

chmod +x /opt/homelab/torchruntime/backup.sh
/opt/homelab/torchruntime/backup.sh

Schedule nightly backups via crontab:

crontab -e

Add the following line to run at 2 AM daily:

0 2 * * * /opt/homelab/torchruntime/backup.sh >> /opt/homelab/torchruntime/logs/backup.log 2>&1

Disaster Recovery: To restore, stop the stack, extract the archive, and import the SQL dump:

cd /opt/homelab/torchruntime
docker compose down
tar -xzf backups/torchruntime_YYYYMMDD_HHMMSS.tar.gz
cat db_YYYYMMDD_HHMMSS.sql | docker compose exec -T db psql -U torchuser -d torchruntime
docker compose up -d

---

9. Step 7: Deep Troubleshooting Matrix

---

10. Step 8: Frequently Asked Questions (FAQ)

Q1: How do I pin the exact image version for reproducibility?

Always use a specific tag like torchruntime/api:2026.09.05 instead of latest. This ensures that docker compose pull doesn't introduce breaking changes. For updates, manually change the tag and run docker compose up -d after reviewing release notes.

Q2: Should I use Watchtower for automatic updates?

Watchtower can be convenient for low-risk services, but for AI workloads where model compatibility matters, manual pinning is safer. If you choose Watchtower, configure it to only update the API container and take a backup before each update. I recommend scheduling a weekly cron job that runs docker compose pull and docker compose up -d after checking the changelog.

Q3: Can I migrate from an existing PyTorch deployment without downtime?

Yes, use the API's built-in import function to load existing model artifacts (e.g., .pt files) into the /models volume. For database migrations, perform a pg_dump from the old instance and restore into the new PostgreSQL container. Plan for a maintenance window to avoid data inconsistency.

Q4: How do I tune resources for multiple concurrent inference requests?

Increase the number of API replicas behind a load balancer (e.g., Caddy's reverse_proxy with multiple upstreams). For GPU workloads, ensure each replica has access to the GPU or use a queue system (like Redis-backed Celery) to serialize requests. Monitor CPU/RAM with docker stats and adjust mem_limit in the compose file accordingly.

Q5: What's the best way to secure the API when exposed to the internet?

Beyond TLS, implement rate limiting at the reverse proxy (e.g., Caddy's rate_limit directive), enable authentication (the API supports JWT tokens), and consider IP allowlisting via Cloudflare. Always bind the API to 127.0.0.1 and use a proxy as the sole entry point. For maximum security, use Tailscale as described in Step 5 to avoid public exposure altogether.

Q6: How do I back up model files that are larger than 10 GB?

Use restic for deduplicated, encrypted backups to an S3-compatible object store or a secondary drive. Initialize a repository with restic init, then create a script that snapshots the /models volume. Restic's deduplication will efficiently handle large model files with minimal storage overhead.

---

This guide was verified against Torchruntime API Examples version 2026.09.05. Always consult the official documentation for the latest changes.