Ollama - Local AI Chatbots
>> INTRODUCTION
🦙 WHAT IS OLLAMA?
Ollama lets you run Large Language Models (LLMs) like
LLaMA, Mistral, and Gemma directly on your PC!
Imagine ChatGPT – but completely local, free, and private.
Your chats never leave your computer!
✅ BENEFITS OF OLLAMA:
- • Works offline – no internet required
- • 100% Private – all data stays local
- • Free – no API costs, no subscriptions
- • Many models – LLaMA 3, Mistral, Gemma, CodeLLaMA...
- • Simple – one command to start!
>> PREREQUISITES
NVIDIA GPU with 8GB+ VRAM
For 7B models. Larger models need more VRAM.
16GB+ RAM
More RAM = larger models possible (even without GPU)
20GB+ Free Storage
Models are 4-40GB in size
💡 GOOD TO KNOW:
Ollama also runs on CPU only – just slower. With enough RAM (32GB+) you can use larger models without a GPU!
>> INSTALLATION
Download & Install Ollama
- Go to ollama.com/download
- Download the version for your OS (Windows/Mac/Linux)
- Run the installer
Download Your First Model
Open a terminal/PowerShell and run:
ollama run llama3.2
🦙 RECOMMENDED MODELS:
llama3.2 |
3B, ~2GB | Fast, good for chat |
llama3.1:8b |
8B, ~5GB | Best balance |
mistral |
7B, ~4GB | Very smart |
codellama |
7B, ~4GB | For programming |
llama3.1:70b |
70B, ~40GB | GPT-4 level (needs 48GB+ VRAM) |
Chat with the Model
After download, a chat starts automatically:
Imagine regular computers are like coins...
You now have a local AI chatbot!
>> GRAPHICAL INTERFACE
Terminal not your thing? There are nice Web UIs!
🌐 OPEN WEBUI (Recommended)
The best UI for Ollama – looks like ChatGPT!
Then open http://localhost:3000
🖥️ JAN.AI
Native desktop app with beautiful UI. jan.ai
🔌 VS CODE EXTENSION
Continue – AI coding assistant directly in VS Code!
>> USING THE API
Ollama provides a REST API – perfect for your own projects!
curl http://localhost:11434/api/generate \
-d '{"model": "llama3.2", "prompt": "Hello!"}'
import ollama
response = ollama.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Hello!'}]
)
print(response['message']['content'])
>> RESOURCES
>> CONCLUSION
With Ollama you have ChatGPT-like AI right on your PC! Completely free, private, and works offline.
Try different models and find the best one for your use case. Have fun experimenting! 🦙
>> FAQ / TROUBLESHOOTING
❓ Which model should I start with?
For a balance of speed and quality: llama3.1:8b (8GB RAM minimum) or
mistral (7B, very fast). For higher quality and 16GB+ RAM: llama3.1:70b or
qwen2.5:32b. For coding: codellama:13b or
deepseek-coder-v2. Run ollama list to see installed models.
❓ Responses are very slow — what's wrong?
Three common causes: (1) model is too big for your RAM — switch to a smaller quant (e.g.
q4_0); (2) running on CPU instead of GPU — check ollama ps to see GPU
layer offloading; (3) many concurrent requests — Ollama serializes them by default. Apple
Silicon users: ensure you have ≥ the model's memory size in unified RAM.
❓ Can Ollama use my NVIDIA GPU?
Yes, automatically on Linux and Windows. Just install the latest NVIDIA drivers. Ollama detects
CUDA and offloads as many layers as VRAM allows. To force full GPU usage, set
OLLAMA_NUM_GPU=99 in the systemd service or environment.
❓ How is Ollama different from LM Studio?
Ollama is CLI-first, runs as a background service, great for scripting and APIs. LM Studio is GUI-first, great for chatting and exploring models visually. Both use the same llama.cpp backend. Pick Ollama if you like terminals and APIs, LM Studio if you prefer a desktop app.
❓ Can I run Ollama on a server and access it from other devices?
Yes. The API listens on http://localhost:11434 by default. To expose it, edit
/etc/systemd/system/ollama.service and add
Environment="OLLAMA_HOST=0.0.0.0:11434", then reload. Never expose to the
internet without authentication — at least put it behind a reverse proxy with basic
auth, or use a VPN/Tailscale to access it.
❓ How do I delete a downloaded model?
ollama rm <model-name> (e.g. ollama rm llama3.1:8b). Models are
stored in ~/.ollama/models/ on Linux/Mac and
%USERPROFILE%\.ollama\models\ on Windows. Use ollama list to see all
installed ones first.