How to install ollama LLM (large language model) loader in Debian Bookworm (12.2) using a bash shell script

Posted: 2023/11/23 in A.I., Debian
Tags: , , , , , ,
#!/usr/bin/env bash
# Modification date: 2023/11/23
# Author: Mark Rijckenberg
# How to install ollama LLM loader in debian-latest container
# See https://github.com/jmorganca/ollama
# first install Nvidia container toolkit:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list \
&& \
sudo apt-get update
sudo apt install -y nvidia-container-toolkit
# Install lshw to automatically detect and install NVIDIA CUDA drivers while using ollama script:
sudo apt install -y lshw
# install btop process monitor to kill open ollama processes when needed:
sudo apt install -y btop
python -m venv .venv
source .venv/bin/activate
curl https://ollama.ai/install.sh | sh
ollama serve &
ollama run mistral
echo "Get list of models here:  https://ollama.ai/library?sort=newest"

# Model	Parameters	        Size	Download
# Mistral	7B	            4.1GB	ollama run mistral
# Llama 2	7B	            3.8GB	ollama run llama2
# Code Llama	7B	        3.8GB	ollama run codellama
# Llama 2 Uncensored	7B	3.8GB	ollama run llama2-uncensored
# Llama 2 13B	13B	        7.3GB	ollama run llama2:13b
# Llama 2 70B	70B	         39GB	ollama run llama2:70b
# Orca Mini	3B	            1.9GB	ollama run orca-mini
# Vicuna	7B	            3.8GB	ollama run vicuna

Leave a comment