Ollama Tutorial: Run LLMs on Your Own Computer for Free

📅 2026-05-01 · AI Quick Start Guide · ~ 28 min read

Imagine for a moment that every time you wanted to use a powerful tool—say, a chainsaw—you had to send it to a remote facility, wait for someone else to operate it, and then pay by the minute. That’s essentially how most people interact with Large Language Models (LLMs) today. You type a prompt, it travels to a server farm somewhere in the cloud, and the results come back with a small delay and a monthly subscription fee.

But what if you could bring that chainsaw to your own workshop? That’s the promise of running LLMs locally. With Ollama, this isn't just a futuristic dream; it's a practical, free, and surprisingly simple reality. This tutorial will walk you through setting up and using Ollama to run powerful AI models on your own computer, giving you privacy, offline capability, and unlimited usage.

Think of Ollama as a streamlined, one-stop app store for AI models. Instead of wrestling with complex Python environments, Docker containers, and CUDA drivers, Ollama abstracts away the technical grunt work. You just tell it which model you want, and it handles the download and execution.

Why Run a Local LLM? The "Home Kitchen" Analogy

Before we dive into the code, let's address the "why." Using a cloud-based LLM like ChatGPT is like ordering takeout. It's convenient, you don't have to cook, and the variety is huge. But running a local LLM with Ollama is like having a home kitchen.

Of course, there's a trade-off. Your "home kitchen" might not be as fast as a professional restaurant's kitchen. Local models are generally smaller and less powerful than the behemoths like GPT-4. But for many tasks—coding help, brainstorming, text summarization, creative writing—they are surprisingly capable.

Step 1: Installing Ollama – Your AI Model Manager

The installation process is the easiest part. Ollama is designed for simplicity. Head over to ollama.com and download the installer for your operating system (macOS, Linux, or Windows).

    curl -fsSL https://ollama.com/install.sh | sh

After installation, open your terminal (Command Prompt on Windows, or Terminal on macOS/Linux) and run the following command to verify everything is working:

ollama --version

If you see a version number, you're ready to go. Ollama is now your AI concierge, waiting for your commands.

Step 2: Pulling Your First Model – The "Download and Go" Experience

This is where the magic happens. Let's grab a model. The most popular starting point is Llama 3.2, a powerful and efficient model from Meta.

In your terminal, type:

ollama pull llama3.2

That's it. Ollama will automatically download the model. The size will be around 2-3 GB, so it might take a few minutes depending on your internet speed. You'll see a progress bar in your terminal.

What just happened? You didn't need to install Python, configure a virtual environment, or figure out PyTorch. ollama pull is the equivalent of docker pull or git clone. It fetches the model and stores it locally, ready for action.

A Quick Tour of Available Models

You aren't limited to just Llama. Ollama has a library of models you can browse on their website. Here are a few popular ones you might want to try:

You can find a complete list at the Ollama Library. To download any of them, just replace the model name in the pull command:

ollama pull mistral
ollama pull codellama

Step 3: Running and Interacting with Your Local LLM

Now for the fun part. Once a model is downloaded, you can start a chat session with it directly from the terminal.

ollama run llama3.2

Your terminal will transform into a chat interface. You can type prompts and get responses instantly.

>>> What is the capital of France?
The capital of France is Paris.

>>> Write a short poem about a cat who loves computers.
A tabby sits with glowing eyes,
Before a screen of vast surmise.
The mouse he bats, a plastic prey,
While data streams the night away.

To exit the chat, type /bye or press Ctrl + D.

A Deeper Look: Using Ollama Programmatically

The interactive chat is great for testing, but the real power of Ollama lies in its ability to be integrated into your own scripts and tools. It provides a simple REST API.

First, ensure the Ollama service is running in the background (it usually is after installation). You can then send requests to http://localhost:11434/api/generate.

Here’s a simple curl command to get a response:

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Explain the concept of recursion in one sentence.",
  "stream": false
}'

The "stream": false parameter tells Ollama to wait for the entire response and return it as a single JSON object. This is perfect for scripting.

Python Integration Example

Let's make it even more practical. Here’s how you can call your local LLM from a Python script. This is a foundational skill for building your own AI-powered tools.

import requests
import json

def ask_ollama(prompt, model="llama3.2"):
    """Send a prompt to a local Ollama model and get a response."""
    url = "http://localhost:11434/api/generate"
    payload = {
        "model": model,
        "prompt": prompt,
        "stream": False
    }
    try:
        response = requests.post(url, json=payload)
        response.raise_for_status()  # Raise an exception for bad status codes
        result = response.json()
        return result['response']
    except requests.exceptions.RequestException as e:
        return f"An error occurred: {e}"

# Example usage
if __name__ == "__main__":
    user_prompt = "What are the three best practices for writing clean Python code?"
    answer = ask_ollama(user_prompt)
    print(f"Prompt: {user_prompt}\n")
    print(f"Response: {answer}")

How to run it:

You'll see a detailed, reasoned answer about clean code practices. This simple script is the foundation for building much more complex applications, from automated report generators to personal coding assistants.

Advanced Tips: Customizing Your AI Experience

Ollama isn't just a launcher; it's a customization platform.

Creating a Custom Modelfile

A Modelfile is like a Dockerfile for your AI. It allows you to set a system prompt, change the temperature (creativity), and even define a custom stop token.

Create a file named Modelfile (no extension) with the following content:

FROM llama3.2

# Set a system prompt to define the AI's personality
SYSTEM "You are a helpful and slightly sarcastic coding tutor. You always provide code examples."

# Set parameters
PARAMETER temperature 0.8
PARAMETER top_p 0.9

Now, "build" this custom model:

ollama create sarcastic-tutor -f ./Modelfile

You can now run your custom personality:

ollama run sarcastic-tutor
>>> What's a for loop?
Oh, a for loop? How original. Fine. It's how you tell your computer to do the same boring thing multiple times. Like this: for i in range(5): print("I'm repeating myself, thanks to you.")

This level of control is what makes local LLMs so powerful for specific use cases.

Summary and Your Action Steps

You've just unlocked a new level of control over AI. You are no longer a passive consumer of cloud services; you are an operator of your own local intelligence. The privacy, the freedom from API costs, and the ability to customize are transformative.

Here is your action plan to solidify this skill:

For a more structured deep dive into the world of local AI, including curated project ideas and a comprehensive glossary of terms like "temperature" and "top_p," visit the Learning Path section at www.aiflowyou.com. It's designed to guide you from your first prompt to building your own AI tools. And if you’re on the go, you can explore the same structured content through the WeChat Mini Program "AI快速入门手册" — perfect for quick learning sessions on your phone.

More AI learning resources at aiflowyou.com →

Mini Program QR

Scan to open Mini Program

WeChat QR

Scan to add on WeChat