Open WebUI on NAS: Setup Guide

Open WebUI is a self-hosted, ChatGPT-style interface for running large language models locally. It connects to Ollama (a local LLM runtime) and provides a polished chat interface for models like Llama 3, Mistral, Phi-3, and others. Running it on a NAS means your AI assistant is always available on your local network, requires no API key or subscription for an Ollama-only setup; prompts and chats remain local only when cloud providers and external networked features are not enabled. This guide covers deploying the Open WebUI + Ollama Docker stack on a NAS, model selection for NAS hardware, and the GPU passthrough configuration that significantly improves inference speed on capable hardware.

Hardware Reality: What to Expect from NAS LLMs

NAS hardware is not designed for LLM inference. Setting realistic expectations:

• Intel Celeron N5095 (TS-464) and Intel Celeron J4125 (DS423+): CPU-only inference speed for a 3B model varies with the exact processor, model build, quantisation, context length, memory configuration, runtime version and cooling. Usable for non-time-critical tasks. CPU-only responsiveness for 7B-and-larger models varies with the model build, prompt and context lengths, memory configuration and the user's latency tolerance.
• AMD Ryzen Embedded V1500B (DS925+ and TS-473A): Performance varies with the exact model, quantisation, context length, memory configuration, runtime version and NAS thermal limits; use measured results from the target configuration.
• QNAP with PCIe GPU (TS-473A + NVIDIA GPU): Adding a QNAP-validated low-profile GPU that fits the NAS's dimensional and power limits may enable GPU inference; verify the exact card against QNAP's compatibility list and driver support before purchase. GPU inference speed varies with the exact GPU, model, quantisation, context length, software version, PCIe configuration and whether the model fits fully in VRAM. This is the correct hardware approach for real-time LLM use on NAS hardware

For casual, non-real-time use (asking questions and waiting 30-60 seconds for a full response), CPU-only inference on a Celeron NAS is functional. For interactive use, a GPU or dedicated inference hardware is needed.

Step 1: Deploy Ollama and Open WebUI

Create a Docker Compose file at /volume1/docker/openwebui/docker-compose.yml (Synology) or /share/docker/openwebui/docker-compose.yml (QNAP):

version: '3.8'
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ./ollama:/root/.ollama
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    ports:
      - 3000:8080
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - ./open-webui-data:/app/backend/data
    restart: unless-stopped

Deploy with docker compose up -d. First startup downloads the Open WebUI image (~1.5GB) and starts both services. Access Open WebUI at http://[NAS-IP]:3000. Create an admin account on first access.

Step 2: Pull a Model

After Open WebUI loads, pull a model from Ollama's library. Model selection depends on your NAS RAM:

• 4GB RAM available for Ollama: Select a current small model whose weights, context allocation and runtime overhead fit the available memory; Phi-3 Mini has been superseded in the Phi family by Phi-4 Mini. These are the smallest capable models
• 8GB RAM available: Choose a current small model whose weights, context allocation and runtime overhead fit the available memory; Mistral 7B is no longer Mistral AI's current edge-model generation. Mistral 7B is significantly more capable than 3B models
• 16GB RAM available: Use Llama 3.1 8B (~4.7GB) or Mistral 7B. More comfortable headroom

To pull a model in Open WebUI: Admin Settings → Connections → Ollama → Manage (wrench icon), or use the download prompt in the model selector. Enter the model name (e.g. llama3.2:3b) and click Pull. Model downloads from Ollama's registry. Sizes range from 2GB to 70GB+. First pull may take 20-60 minutes depending on model size and internet speed.

Alternatively, pull from the Ollama container CLI: docker exec -it ollama ollama pull llama3.2:3b

Step 3: GPU Passthrough (QNAP PCIe Models)

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ./ollama:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped