Run your own AI chat with Ollama and Open WebUI

Ollama runs open AI language models, such as Llama, Gemma, Qwen and Mistral, on your own server. Open WebUI gives them a ChatGPT-style chat interface in your browser, with multiple users, chat history and document uploads. Your prompts and data stay on your VPS and are never sent to an outside AI company.

This guide runs both in Docker, with Caddy in front to provide a free HTTPS certificate automatically. It works on every operating system we offer: Ubuntu 22.04, 24.04 and 26.04, Debian 12 and 13, AlmaLinux 8, 9 and 10, and Rocky Linux 9 and 10.

What to expect on a VPS

Our VPS plans run AI models on the CPU, without a graphics card. Smaller models work well for everyday chat, summarising and drafting, but replies are slower than big cloud services, and very large models won't fit. As a guide:

• Small models (1 to 4 billion parameters): at least 4 vCPU and 8 GB of RAM
• Medium models (7 to 8 billion parameters): at least 8 vCPU and 16 GB of RAM

Each model also takes a few GB of disk, so allow at least 30 GB free. More CPU cores make replies noticeably faster.

Before you start

• Docker must be installed. If it isn't, follow our guide: Install Docker Engine and Docker Compose on your VPS.
• A domain or subdomain with an A record pointing to your VPS IP address, for example ai.example.co.nz.
• Nothing else using ports 80 or 443 on the server.
• SSH access as root or as a user with sudo rights. If you're logged in as root, you can leave sudo off the commands.

Step 1: Open the firewall

Ubuntu/Debian with UFW enabled:

sudo ufw allow 80/tcp
sudo ufw allow 443

AlmaLinux/Rocky with firewalld:

sudo firewall-cmd --permanent --add-service=http --add-service=https --add-port=443/udp
sudo firewall-cmd --reload

Step 2: Create the configuration files

sudo mkdir -p /opt/ai-chat
cd /opt/ai-chat
sudo nano .env

Paste in the following, replacing the domain with your own:

DOMAIN=ai.example.co.nz

Press Ctrl+O and Enter to save, then Ctrl+X to exit. Next, create the Docker Compose file:

sudo nano compose.yaml

Paste in the following. You don't need to change anything in this file.

services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ollama:/root/.ollama

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama

  caddy:
    image: caddy:2
    container_name: caddy
    restart: unless-stopped
    environment:
      - DOMAIN=${DOMAIN}
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config

volumes:
  ollama:
  open-webui:
  caddy_data:
  caddy_config:

Save and exit. Ollama isn't published to the internet in this setup, so only Open WebUI can talk to it. Finally, create the Caddy configuration:

sudo nano Caddyfile

Paste in the following exactly as shown:

{$DOMAIN} {
    reverse_proxy open-webui:8080
}

Save and exit.

Step 3: Start everything

sudo docker compose up -d

The first start downloads a few GB of images, so give it a few minutes. Caddy gets an SSL certificate for your domain automatically.

Step 4: Download a model

Pull a small model to start with. For example:

sudo docker exec -it ollama ollama pull llama3.2:3b

You can browse other models, and see their sizes, at ollama.com/library. Pick one whose size comfortably fits in your VPS's RAM. To see what you've downloaded, run sudo docker exec -it ollama ollama list.

Step 5: Create your admin account

Open https://ai.example.co.nz straight away and click Get started. The first account created becomes the administrator, so create yours before anyone else finds the page.

Then choose your model from the list at the top of the chat window and start chatting.

Step 6: Control who can sign up

By default, new sign-ups have to be approved by an administrator. To check or change this, go to your profile → Admin Panel → Settings → General. You can turn off Enable New Sign Ups entirely, and add users yourself under Admin Panel → Users.

Keeping it up to date

cd /opt/ai-chat
sudo docker compose pull
sudo docker compose up -d

Your chats, users and settings are kept in the open-webui volume, and downloaded models in the ollama volume.

Troubleshooting

• No models show in the list: download one first (Step 4), then refresh the page.
• Replies are very slow or the model crashes: the model is too big for your VPS. Try a smaller one, or upgrade to a plan with more CPU and RAM. Check memory use with free -h.
• The site won't load over HTTPS: check the Caddy logs with sudo docker logs caddy. The usual cause is DNS that doesn't point to this VPS yet, or ports 80 and 443 being blocked.
• Open WebUI can't reach Ollama: check both containers are running with sudo docker ps, and look for errors with sudo docker logs open-webui.

Need help?

Our Cloud VPS plans are self-managed, so you're responsible for installing, securing and maintaining the software on your server. If something on our side isn't working, such as the network, or your VPS won't boot, open a support ticket. New to your VPS? Start with our Cloud VPS Getting Started Guide.