Setting up Ollama on Arch Linux

I’ve been experimenting more with AI and hosting it locally ever since I ran out of Claude tokens and some of the free AI agents I was running was also unavailable. There is also the threat of some models being restricted under export controls. We saw this recently with Claude Fable being restricted.

I’ve decided to set up on my desktop machine some of the self-hosted solutions as a backup. One of the easiest ways to get started is to use Ollama.

Hardware

My desktop machine has the following specifications:

• MOTHERBOARD: ROG STRIX X570-F GAMING
• CPU: AMD Ryzen 9 5950X
• GPU: AMD Radeon RX 6800 XT
• RAM: 64 GB
• DISKS: 2x1TB NVME (Sabrent branded)

This machine was built around 2019 and originally had an AMD Ryzen 3700X, an ASUS 1080Ti and 32GB of RAM installed. I upgraded to an AMD 6800XT, AMD Ryzen 5950X and 64GB back in 2022. The RAM and CPU upgrade was due to a heavy-weight JetBrains IDE e.g. Rider, PyCharm, WebStorm, Large .NET stack would get to 30GB of RAM usage.

This is still a relatively fast machine for normal development, gaming and video editing and many other tasks.

Installing Ollama

Installing Ollama on Arch Linux is rather straightforward, and you can follow the wiki

$ sudo pacman -S ollama ollama-rocm

Partition Layout

I have two 1TB NVMe drives. I bought them when I first built this machine which was back sometime in 2019/2020.

$ lsblk -d -o NAME,MODEL,SIZE,TYPE
NAME    MODEL     SIZE TYPE
zram0               4G disk
nvme0n1 Sabrent 953.9G disk
nvme1n1 Sabrent 953.9G disk

You can see that both drives are identical.

NOTE: The Linux kernel naming convention for NVME drives works as follows:

Originally I had the first drive (nvme0n1) for Windows and second drive (nvme1n1) for Linux. I removed Windows after I had realised that I hadn’t booted into it in over a year, so that Windows partitions no longer exist on nvme0n1.

The Linux drive is partitioned as follows:

nvme1n1     259:1    0 953.9G  0 disk  
├─nvme1n1p1 259:3    0     1G  0 part  /boot
├─nvme1n1p2 259:4    0  93.1G  0 part  
│ └─root    253:0    0  93.1G  0 crypt /
└─nvme1n1p3 259:5    0 859.7G  0 part  
  └─home    253:1    0 859.7G  0 crypt /home

Having a 100G partition is normally enough to install all of my programs in Arch Linux. Directories other than /home or /boot are in the 100G drive. However, I kept on running out of on /(root) once I had installed Ollama and ROCm.

ROCm software stack is installed under /opt/rocm.

$ df -h /opt/rocm/ 
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p3   53G   23G   28G  45% /opt/rocm

You can see that it comes in at about 23 gigabytes.

The Ollama models live under /var/lib/ollama/.ollama/models. If I list the models it also shows SIZE which is the size on the disk.

$ ollama list
NAME                         ID              SIZE      MODIFIED     
qwen3-coder:latest           06c1097efce0    18 GB     47 hours ago    
gemma4:26b                   5571076f3d70    17 GB     47 hours ago    
qwen3.5:9b                   6488c96fa5fa    6.6 GB    3 days ago      
qwen2.5-coder:7b-instruct    dae161e27b0e    4.7 GB    3 days ago      
deepseek-r1:14b              c333b7232bdb    9.0 GB    3 days ago      
qwen2.5-coder:7b             dae161e27b0e    4.7 GB    3 days ago      
qwen2.5-coder:14b            9ec8897f747e    9.0 GB    3 days ago  

As you can see from the sizes of the downloaded Ollama models, they range between 4GB and 18GB.

It should be quite obvious that a 100 gigabyte partition is going to run out of space quickly.

I did just consider upgrading the drives as these Sabrent drives at almost 7 years old. However, prices for NVME drives are quite expensive, it is approximately £200 and £350 for a 2TB and 4TB model respectively.

Since I had a 1TB drive sitting there doing nothing. It makes sense to use the spare drive for the software libraries and the Ollama models.

I also like to use Virtual Machines for demos whenever possible, mainly because this protects me from accidentally exposing anything potentially sensitive. A modern Linux distro will need at minimum 5–10 gigabytes of space. I normally provision 25-50 gigabyte per virtual machine, so I have plenty of headroom.

I asked Google Gemini to suggest me some sizes for the partitions.

• A 30-50 gigabyte partition for the ROCm SDK.
• A 600 gigabyte partition for Ollama and any other AI software and models I am likely to serve from this machine.
• A 300 gigabyte partition for my VMs.

Linux has a Filesystem Hierarchy Standard. There is no specific place to put things like Ollama, however this machine will be serving Ollama on the network and serving VMs potentially on the network. So I opted .to put these under /srv.

I won’t bore you with going over partitioning and formatting, there are thousands of tutorials online of how to do this and isn’t worth covering here.

I ended up with the following partitions. Note that at the time the partitions hadn’t been mounted anywhere, so the mount points wouldn’t have been shown in the output of lsblk

$ lsblk
nvme1n1     259:1    0 953.9G  0 disk  
├─nvme1n1p1 259:3    0     1G  0 part  /boot
├─nvme1n1p2 259:4    0  93.1G  0 part  
│ └─root    253:0    0  93.1G  0 crypt /
└─nvme1n1p3 259:5    0 859.7G  0 part  
  └─home    253:1    0 859.7G  0 crypt /home

The /srv/ai partition holds all my ai stuff. I am looking up at setting up Hermes. The /srv/vm partition will hold the vm.

I updated the /etc/fstab to have the following entries.

# --- snip the other mounted drives ---

# ai & vm
UUID=<partition id> 	/srv/vm   	ext4  		defaults,noatime  0  2
UUID=<partition id>	/srv/ai   	ext4  		defaults,noatime  0  2

# rocm
UUID=<partition id>   	/opt/rocm   	ext4   		defaults,noatime   0   2

Moving the ROCm Installation

As previously mentioned ROCm was already installed to /opt/rocm.

To avoid having to uninstall everything, I opted to just move the files into our new partition. I did this by simply making a temporary directory under /mnt moving the files across:

sudo mkdir -p /mnt/rocm_stage
sudo mount /dev/nvme0n1p3 /mnt/rocm_stage
sudo cp -ax /opt/rocm/. /mnt/rocm_stage/
sudo rm -rf /opt/rocm/*
sudo umount /mnt/rocm_stage
sudo mount -a

Ollama Configuration

Arch Linux provides a systemd unit file where we can run Ollama as a service. However, we have to override the configuration.

We need to have the following overridden:

• Models directory needs to sit somewhere in our new /srv/ai directory. This can be set using the OLLAMA_MODELS environment variable (See: How do I set them to a different location?)
• By default, Ollama binds to localhost. So we need to change the bind address. This can be set using the OLLAMA_HOST environment variable. (See: How can I expose Ollama on my network?)

Before we make changes to the service, however, we should create the directories required and set the ownership properly.

sudo mkdir -p /srv/ai/ollama/models
sudo chown -R ollama:ollama /srv/ai/ollama

You can create a systemd override file:

$ sudo touch /etc/systemd/system/ollama.service.d/ollama_all_overrides.conf

Then set the environment variables.

[Service]
Environment="OLLAMA_MODELS=/srv/ai/ollama/models"
Environment="OLLAMA_HOST=0.0.0.0"

Then reload systemd services and enable Ollama:

sudo systemctl enable --now ollama

Verifying the Environment

A quick sanity check confirms the overrides took effect:

$ sudo cat /proc/$(pgrep -o ollama)/environ | tr '\0' '\n' | grep -E 'OLLAMA'
OLLAMA_MODELS=/srv/ai/ollama/models
OLLAMA_HOST=0.0.0.0

Testing that this is working

$ ollama run deepseek-r1:14b

Firstly let’s check that it is downloading into the correct directory.

Ollama put the models in ${OLLAMA_MODELS}/manifests/registry.ollama.ai/library

If I list the contents of this directory:

$ ls -l /srv/ai/ollama/models/manifests/registry.ollama.ai/library/
total 24
drwxr-xr-x 2 ollama ollama 4096 Jul  2 02:06 deepseek-r1
drwxr-xr-x 2 ollama ollama 4096 Jul  3 14:45 gemma4
drwxr-xr-x 2 ollama ollama 4096 Jul  2 09:43 qwen2.5-coder
drwxr-xr-x 2 ollama ollama 4096 Jul  3 14:57 qwen3-coder
drwxr-xr-x 2 ollama ollama 4096 Jul  3 15:20 qwen3-coder-32k
drwxr-xr-x 2 ollama ollama 4096 Jul  2 10:36 qwen3.5

Next thing we want to check is if the GPU is actually being used properly by Ollama. There are several ways to do this, however the nicest way I’ve found to use on Arch Linux is to install mission-center

This can be done using pacman:

$ sudo pacman -S mission-center

If we launch it, we get something very similar to task manager on Windows. We can simply select the GPU in the devices list.

Then, we run a prompt to ask our model to do something.

You can see the GPU usage is spiking while the model is working. This appears to be working correctly.