Install and Use Ollama for Local AI

Run large language models locally with Ollama — no internet needed, full privacy.

What is Ollama?

Ollama is a free, open-source tool that lets you run large language models (LLMs) directly on your own computer. Instead of sending your data to a cloud service, everything stays local — your conversations, your prompts, and your data are completely private.

  • •Run models like Llama 3, Mistral, Phi, and Gemma on your own hardware
  • •Works entirely offline once models are downloaded
  • •No accounts, no API keys, no usage limits
  • •Available on macOS, Windows, and Linux

Step 1. Check Requirements

Before installing Ollama, make sure your system meets these minimum requirements:

  • •RAM: 8 GB minimum, 16 GB or more recommended for larger models
  • •CPU: Any modern processor works — inference is generally CPU-friendly
  • •GPU (optional): Apple Silicon (M1/M2/M3/M4), NVIDIA CUDA, or AMD ROCm for faster inference
  • •Disk space: 2–40 GB per model depending on size — plan accordingly

Step 2. Install Ollama

Download and install Ollama for your operating system:

  • •Windows & macOS: Download the installer from ollama.com and follow the guided setup.
  • •Linux: Run the one-liner install script:

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation was successful:

ollama --version

Step 3. Pull and Run a Model

Download and start chatting with your first model:

  • •Pull a model — this downloads it to your machine:

ollama pull llama3.2

  • •Run the model interactively:

ollama run llama3.2

  • •Type your prompts and the model will respond. Type /bye to exit the chat session

Step 4. Available Models

Ollama supports a wide range of models. Here are some popular ones to try:

  • •llama3.2 — Meta's latest general-purpose model, great all-around choice
  • •mistral — Fast, efficient model from Mistral AI with strong reasoning
  • •phi3 — Microsoft's compact model, excellent for constrained hardware
  • •gemma2 — Google's lightweight open model family
  • •codegemma — Optimized for code generation and technical tasks
  • •llama3.2-vision — Multimodal model that can understand images

To see all models you've downloaded:

ollama list

Browse the full library at ollama.com/library.

Step 5. Use the API

Ollama exposes an HTTP API on port 11434, making it easy to integrate with other tools and applications:

Generate a completion with a single request:

curl http://localhost:11434/api/generate -d '{

"model": "llama3.2",

"prompt": "Explain quantum computing in simple terms"

}'

For multi-turn conversations, use the chat endpoint:

curl http://localhost:11434/api/chat -d '{

"model": "llama3.2",

"messages": [{"role": "user", "content": "Hello!"}]

}'

Quick Tips

  • •Remove models you no longer need with ollama rm model-name to free up disk space
  • •Downloaded models are stored at ~/.ollama/models — back this up if you want to preserve your model collection
  • •Expose Ollama on your local network by setting OLLAMA_HOST=0.0.0.0 — then other devices can access it at your machine's IP
  • •For a polished chat interface, pair Ollama with Open WebUI — it gives you a ChatGPT-like experience running entirely on your own hardware

This guide covers setting up Ollama as a standalone tool. If you're looking for an integrated, ChatGPT-like experience with a web UI, check out our Local AI guide which walks you through setting up Open WebUI with Ollama for a complete local AI workstation.

Continue Reading

Run a Local AI Model

Python Basics for Non-Programmers

Choosing Hardware for Self-Hosting

Ready to Take Control of Your AI?

Running AI locally gives you privacy, control, and freedom from subscriptions. Explore more guides to level up your skills.

Browse All How-To Guides