Ollama server

Feb 5, 2025 llm linux

Overview

Setting up an old school Self-Hosted Server Stack: Mattermost, OpenWebUI, and Monitoring

This is how Claude and I installed an Ollama server from scratch.

Part 1: Installing Ollama and OpenWebUI

Installing Ollama

First, we installed Ollama, which is the backend for our AI operations:

1curl https://ollama.ai/install.sh | sh

After installation, we created a systemd service to manage Ollama:

1sudo vim /etc/systemd/system/ollama.service

Added this configuration:

 1[Unit]
 2Description=Ollama Service
 3After=network-online.target
 4
 5[Service]
 6ExecStart=/usr/local/bin/ollama serve
 7User=root
 8Restart=always
 9
10[Install]
11WantedBy=multi-user.target

Started and enabled the service:

1sudo systemctl daemon-reload
2sudo systemctl enable ollama
3sudo systemctl start ollama

To verify the installation, we pulled a model:

1ollama pull phi3

Setting up OpenWebUI with Ollama

OpenWebUI serves as our frontend interface for Ollama. We installed it using Python's virtual environment:

 1# Install required packages
 2sudo apt install python3-full python3-pip python3-venv
 3
 4# Create and activate virtual environment
 5mkdir open-webui && cd open-webui
 6python3 -m venv openwebui-env
 7source openwebui-env/bin/activate
 8
 9# Install OpenWebUI
10pip install open-webui

Created a systemd service for OpenWebUI:

1sudo vim /etc/systemd/system/openwebui.service

Added the configuration (change 'username' to your username):

 1[Unit]
 2Description=OpenWebUI
 3After=network.target
 4
 5[Service]
 6Type=simple
 7User=username
 8WorkingDirectory=/home/username
 9Environment=PATH=/home/username/open-webui/openwebui-env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
10ExecStart=/home/username/open-webui/openwebui-env/bin/open-webui serve
11Restart=always
12
13[Install]
14WantedBy=multi-user.target

Then start it.

1sudo systemctl daemon-reload
2sudo systemctl enable openwebui
3sudo systemctl start openwebui

Part 2: Setting up Mattermost

Initial Installation

We started by installing Mattermost on our Ubuntu server. Mattermost runs on port 8065 by default.

Troubleshooting WebSocket Issues

Initially, we encountered the error: "Please check connection, Mattermost unreachable. If issue persists, ask administrator to check WebSocket port." This is a common issue when setting up Mattermost, especially before configuring Nginx properly.

The solution involved:

  1. Checking if port 8065 was accessible:
1sudo netstat -tulpn | grep 8065
  1. Ensuring the WebSocket configuration in config.json matched our setup:
1{
2    "ServiceSettings": {
3        "SiteURL": "http://your-server-ip:8065",
4        "WebsocketURL": "",  // Empty to use SiteURL
5        "TLSStrictTransport": false
6    }
7}
  1. Most importantly, properly configuring Nginx to handle WebSocket connections. The critical part in the Nginx configuration is:
1location ~ /api/v[0-9]+/(users/)?websocket$ {
2    proxy_set_header Upgrade $http_upgrade;
3    proxy_set_header Connection "upgrade";
4    proxy_set_header Host $http_host;
5    # ... other proxy settings ...
6}

Adding SSL with Nginx

To secure our Mattermost instance, we set up Nginx as a reverse proxy and added SSL certificates:

  1. Created Nginx configuration:
1sudo vim /etc/nginx/sites-available/mattermost

Added:

 1upstream backend {
 2    server 127.0.0.1:8065;
 3    keepalive 32;
 4}
 5
 6proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=mattermost_cache:10m max_size=3g inactive=120m use_temp_path=off;
 7
 8server {
 9    server_name mattermost.yourdomain.com;
10
11    location ~ /api/v[0-9]+/(users/)?websocket$ {
12        proxy_set_header Upgrade $http_upgrade;
13        proxy_set_header Connection "upgrade";
14        client_max_body_size 50M;
15        proxy_set_header Host $http_host;
16        proxy_set_header X-Real-IP $remote_addr;
17        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
18        proxy_set_header X-Forwarded-Proto $scheme;
19        proxy_set_header X-Frame-Options SAMEORIGIN;
20        proxy_buffers 256 16k;
21        proxy_buffer_size 16k;
22        proxy_read_timeout 600s;
23        proxy_pass http://backend;
24    }
25
26    location / {
27        client_max_body_size 50M;
28        proxy_set_header Connection "";
29        proxy_set_header Host $http_host;
30        proxy_set_header X-Real-IP $remote_addr;
31        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
32        proxy_set_header X-Forwarded-Proto $scheme;
33        proxy_set_header X-Frame-Options SAMEORIGIN;
34        proxy_buffers 256 16k;
35        proxy_buffer_size 16k;
36        proxy_read_timeout 600s;
37        proxy_cache mattermost_cache;
38        proxy_cache_use_stale error timeout invalid_header http_500;
39        proxy_cache_revalidate on;
40        proxy_cache_min_uses 2;
41        proxy_cache_lock on;
42        proxy_cache_valid 200 7d;
43        proxy_pass http://backend;
44    }
45}

Securing Mattermost

We implemented several security measures:

  1. Disabled open registration by modifying config.json:
1sudo vim /opt/mattermost/config/config.json

Added:

1{
2    "ServiceSettings": {
3        "EnableOpenServer": false
4    },
5    "TeamSettings": {
6        "EnableUserCreation": false,
7        "EnableTeamCreation": false
8    }
9}
  1. Enabled Multi-Factor Authentication:
1{
2    "ServiceSettings": {
3        "EnforceMultifactorAuthentication": true,
4        "EnableMultifactorAuthentication": true
5    }
6}
  1. Set up email notifications using SendGrid:
 1{
 2    "EmailSettings": {
 3        "EnableSignUpWithEmail": true,
 4        "EnableSignInWithEmail": true,
 5        "EnableSMTPAuth": true,
 6        "SMTPUsername": "apikey",
 7        "SMTPPassword": "your-sendgrid-api-key-here",
 8        "SMTPServer": "smtp.sendgrid.net",
 9        "SMTPPort": 587,
10        "ConnectionSecurity": "STARTTLS",
11        "SendEmailNotifications": true,
12        "FeedbackName": "Mattermost",
13        "FeedbackEmail": "your-verified-sender@yourdomain.com",
14        "ReplyToAddress": "your-verified-sender@yourdomain.com"
15    }
16}

Part 3: Setting up Monitoring

Installing Prometheus

  1. Created necessary users and directories:
1sudo useradd --no-create-home --shell /bin/false prometheus
2sudo mkdir /etc/prometheus
3sudo mkdir /var/lib/prometheus
  1. Downloaded and installed Prometheus:
1wget https://github.com/prometheus/prometheus/releases/download/v2.49.1/prometheus-2.49.1.linux-amd64.tar.gz
2tar xvf prometheus-*.tar.gz
3cd prometheus-*/
4sudo cp prometheus /usr/local/bin/
5sudo cp promtool /usr/local/bin/
6sudo cp -r consoles/ /etc/prometheus
7sudo cp -r console_libraries/ /etc/prometheus
  1. Created Prometheus configuration:
1sudo vim /etc/prometheus/prometheus.yml

Added basic configuration:

 1global:
 2  scrape_interval: 15s
 3
 4scrape_configs:
 5  - job_name: 'prometheus'
 6    static_configs:
 7      - targets: ['localhost:9090']
 8
 9  - job_name: 'node'
10    static_configs:
11      - targets: ['localhost:9100']

Installing Node Exporter

  1. Set up Node Exporter for system metrics:
1wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
2tar xvf node_exporter-*.tar.gz
3sudo cp node_exporter-*/node_exporter /usr/local/bin/
4sudo useradd --no-create-home --shell /bin/false node_exporter
5sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
  1. Created systemd service for Node Exporter:
1sudo vim /etc/systemd/system/node_exporter.service

Added:

 1[Unit]
 2Description=Node Exporter
 3Wants=network-online.target
 4After=network-online.target
 5
 6[Service]
 7User=node_exporter
 8Group=node_exporter
 9Type=simple
10ExecStart=/usr/local/bin/node_exporter
11
12[Install]
13WantedBy=multi-user.target

Installing Grafana

  1. Added Grafana repository and installed:
1sudo apt-get install -y apt-transport-https software-properties-common
2wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
3echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/sources.list.d/grafana.list
4sudo apt-get update
5sudo apt-get install grafana
  1. Set up Nginx as reverse proxy for Grafana:
1sudo vim /etc/nginx/sites-available/grafana

Added:

 1server {
 2    server_name grafana.yourdomain.com;
 3
 4    location / {
 5        proxy_pass http://localhost:3000;
 6        proxy_http_version 1.1;
 7        proxy_set_header Upgrade $http_upgrade;
 8        proxy_set_header Connection 'upgrade';
 9        proxy_set_header Host $host;
10        proxy_cache_bypass $http_upgrade;
11    }
12}

Useful Shell Aliases

If you're using bash/zsh, you might want to add these helpful aliases to your .bashrc or .zshrc:

 1# Service management
 2alias mm-restart='sudo systemctl restart mattermost'
 3alias mm-status='sudo systemctl status mattermost'
 4alias mm-logs='sudo journalctl -u mattermost -f'
 5
 6# Quick edits
 7alias mm-config='sudo vim /opt/mattermost/config/config.json'
 8alias ng-edit='cd /etc/nginx/sites-available'
 9
10# Monitoring
11alias prom-status='sudo systemctl status prometheus'
12alias graf-status='sudo systemctl status grafana-server'

Conclusion

We now have a fully functional, secure server stack with:

  • Mattermost for team communication
  • OpenWebUI for AI interactions
  • Comprehensive system monitoring with Prometheus and Grafana
  • Everything secured with SSL certificates
  • Basic monitoring dashboards set up

Next steps could include:

  • Setting up specific Mattermost monitoring
  • Adding Ollama monitoring
  • Creating custom Grafana dashboards
  • Setting up alerting rules
  • Implementing backup solutions
  • Monitor token per second

Remember to regularly update all components and monitor system resources to ensure everything runs smoothly. Setting up a Self-Hosted Server Stack: Mattermost, OpenWebUI, and Monitoring