How to Self-Host Llama 3 on a Cheap VPS with Ollama and Docker
Complete guide to running Llama 3 8B on your own server: RAM requirements, Docker Compose, Nginx with TLS, and how to measure real performance.
Nelson Barbosa
1. Why self-host instead of calling an API
There are three concrete reasons to run Llama 3 on your own machine. The first is predictable cost: a VPS charges a flat monthly fee regardless of how many requests you make, while commercial APIs bill per token and the bill grows with usage. The second is privacy — if you process customer data, contracts or internal records, keeping everything inside a server you control avoids sending that content to a third party and simplifies GDPR compliance. The third is the absence of rate limits: no quotas, no throttling halfway through a batch job. In exchange you take on server maintenance, security patching, and answer quality below that of frontier models. It pays off for text classification, data extraction, summarisation and repetitive work; it does not pay off for complex reasoning.
2. How much RAM you need, and why
Llama 3 8B quantized to 4 bits (Q4_K_M) takes roughly 4.7 GB on disk, and the whole thing has to be loaded into memory. On top of that sits the context cache (KV cache), which grows with prompt size, plus the memory used by the operating system and the containers. In practice: **8 GB of RAM is the functional minimum** and gets tight with long contexts; **16 GB is comfortable** and allows larger contexts or a second model loaded. Below 8 GB the system falls back to swap and latency becomes unusable. As for CPU, core count matters less than generation. A recent processor with AVX-512 runs inference considerably faster than an older core at the same vCPU count. Prefer NVMe over SATA: the initial model load reads several gigabytes. **On speed:** on pure CPU, with no GPU, expect reading pace — tens of tokens per second is not realistic on most budget VPS instances. Measure on your own server before trusting any figure; section 8 shows how.
AlphaVPS
AMD Ryzen instances with NVMe. Check the plan’s RAM and CPU generation before you commit.
Affiliate link: we earn a commission if you subscribe, at no extra cost to you.
3. Prepare the server before installing anything
A freshly provisioned server is exposed. Before installing Ollama, close the ports and create an unprivileged user. These steps take five minutes and stop the machine being compromised on its first night. Disable password authentication only after you have confirmed your SSH key works — not before, or you will lock yourself out of your own server. Note that the firewall opens only ports 22, 80 and 443. Ollama’s port, 11434, stays deliberately closed to the outside; section 6 explains why.
# Utilizador sem privilégios
adduser deploy
usermod -aG sudo deploy
# Chave SSH (a partir da sua máquina local)
ssh-copy-id deploy@SEU_IP
# Firewall: apenas SSH e HTTP/HTTPS
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable
# Só depois de confirmar que a chave funciona:
sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
systemctl restart ssh
# Actualizações de segurança automáticas
apt update && apt install -y unattended-upgrades fail2ban
systemctl enable --now fail2ban4. Install Docker
Use Docker’s official repository, not the distribution `docker.io` package, which tends to lag several versions behind and does not ship the modern `compose` plugin. After adding your user to the `docker` group, log out and back in for the change to take effect.
curl -fsSL https://get.docker.com | sh
usermod -aG docker deploy
# Confirmar
docker --version
docker compose version5. Ollama and Open WebUI with Docker Compose
Note the most important detail in this file: Ollama publishes its port on `127.0.0.1:11434`, not `0.0.0.0`. That means it is reachable only from the server itself and from the other containers — never directly from the internet. Open WebUI connects to Ollama over Docker’s internal network and is the only piece that will eventually be exposed, through Nginx and always behind authentication. The volumes ensure downloaded models and user accounts survive container restarts and upgrades.
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
restart: unless-stopped
# Apenas loopback: nunca exposto à internet
ports:
- "127.0.0.1:11434:11434"
volumes:
- ollama_models:/root/.ollama
webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
depends_on:
- ollama
environment:
- OLLAMA_BASE_URL=http://ollama:11434
- WEBUI_AUTH=true
ports:
- "127.0.0.1:8080:8080"
volumes:
- webui_data:/app/backend/data
volumes:
ollama_models:
webui_data:6. Pull the model and choose the quantization
Quantization sets the trade-off between memory and quality. `Q4_K_M` is the usual balance point and where I would start. `Q5_K_M` improves coherence slightly for roughly 1 GB more. `Q8_0` is near-lossless but takes around 8.5 GB, which only fits comfortably in 16 GB of RAM. If the server has 8 GB, stay on `Q4_K_M`. The quality difference is small; the difference between fitting in memory and not fitting is not.
docker compose up -d
# Descarregar o modelo (alguns GB, demora)
docker exec -it ollama ollama pull llama3:8b
# Variantes de quantização
docker exec -it ollama ollama pull llama3:8b-instruct-q5_K_M
# Listar o que está instalado e o espaço ocupado
docker exec -it ollama ollama list7. Nginx reverse proxy and TLS certificate
This is the step that turns a local service into one that is safely reachable. Nginx takes traffic on 443, terminates TLS and forwards to Open WebUI on loopback. The extended timeout is not decorative: generating long responses on CPU can exceed Nginx’s default 60-second limit and return a 504 partway through the answer. Point your domain’s `A` record at the server IP first, and only then run `certbot`, or validation will fail.
server {
server_name ia.seudominio.pt;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Streaming de tokens
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Inferência em CPU excede os 60s por omissão
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
}8. Measure real performance on your server
Do not trust performance figures published by third parties, including the ones in this article. Generation pace depends on the CPU model, its generation, the quantization, the context size, and the load from whoever shares the same physical machine — something you do not control on a shared VPS. Ollama’s `--verbose` returns the real metrics for your instance. The number that matters is `eval rate`, in tokens per second. If it comes out too low for your use case, the options in order of effectiveness are: reduce the context, drop the quantization, move to a smaller model such as Llama 3.2 3B, or move to a GPU instance.
# Métricas reais desta máquina
docker exec -it ollama ollama run llama3:8b --verbose "Resume em três frases o que é a quantização de modelos."
# Interessa a linha:
# eval rate: X tokens/s
# Memória e CPU em tempo real durante a geração
docker stats ollama9. Calling the API from your own code
Ollama exposes an API compatible with OpenAI’s format, which means most existing libraries work by changing nothing but the `base_url`. There is no need to rewrite integrations. Because port 11434 is closed to the outside, your code has to run on the same server, or reach it over an SSH tunnel or a private network. **Never open 11434 to the public:** the Ollama API has no authentication whatsoever. Anyone who finds your IP can use your server freely, and automated scanners look for exactly this.
from openai import OpenAI
# Aponta para o Ollama local em vez da OpenAI
client = OpenAI(
base_url="http://127.0.0.1:11434/v1",
api_key="ollama", # ignorado, mas exigido pela biblioteca
)
resposta = client.chat.completions.create(
model="llama3:8b",
messages=[
{"role": "system", "content": "És um assistente técnico conciso."},
{"role": "user", "content": "Classifica este email como urgente ou normal."},
],
temperature=0.2,
)
print(resposta.choices[0].message.content)10. When this is not worth it
Be honest about volume. If you make a few hundred calls a month, a commercial API costs less than any VPS and saves you the maintenance. Self-hosting starts to pay off under steady, heavy use, or when data privacy is a requirement rather than a preference. It also does not pay off if you need frontier quality. A quantized 8B is capable at bounded tasks and clearly weaker at reasoning, complex code or long-form writing. Comparing it against the large commercial models and expecting parity leads to disappointment. And count your time. Security patching, certificate renewal, monitoring and failure recovery are your hours. If your time has a market value, put it in the calculation before deciding.
Frequently Asked Questions
Can I run Llama 3 70B on a VPS without a GPU?
It technically starts with aggressive quantization and enough RAM, but CPU speed makes it impractical for interactive use. On a budget VPS, 8B is the sensible ceiling. If you need 70B, you need a GPU.
Is it legal to use Llama 3 commercially?
Meta’s license permits commercial use with conditions — among them a monthly active user threshold above which a specific license is required, and attribution requirements. Read the license for the version you download before shipping it inside a paid product.
What happens if I reboot the server?
With `restart: unless-stopped` in the compose file, the containers come back on their own and the models stay in the volumes. The first response after boot is slower because the model has to be read from disk into memory again.
How do I back this up?
The models need no backup: a `ollama pull` fetches them again. What matters is the `webui_data` volume, which holds accounts, conversations and settings. A `docker run --rm -v webui_data:/data -v $(pwd):/backup alpine tar czf /backup/webui.tar.gz /data` covers it.
Do I really need Nginx, or can I expose port 8080 directly?
You can, but you lose TLS — credentials and conversations travel in the clear. Nginx with certbot gives you free HTTPS and automatic renewal for a few minutes of setup. That step is not worth skipping.