Ollama API with Python: Build Local LLM Apps

Ollama API with Python: Build Local LLM Apps

Ollama lets you run open-source large language models such as Llama, Mistral, and DeepSeek locally. Its HTTP API and Python client make it easy to integrate local LLMs into scripts, notebooks, and web apps without sending data to the cloud.

Why use Ollama with Python?

• Privacy: data never leaves your machine
• Cost: no per-token API charges
• Offline: works without an internet connection after the model is pulled
• Flexibility: swap models by changing one string

Install Ollama

Download Ollama from ollama.com or install via command line:

curl -fsSL https://ollama.com/install.sh | sh

Pull a model:

ollama pull llama3.2
ollama pull mistral

The Ollama REST API

Ollama exposes a local server at http://localhost:11434. You can call it with any HTTP client.

Generate a completion

curl http://localhost:11434/api/generate -d '
{
  "model": "llama3.2",
  "prompt": "Explain Ollama in one sentence."
}
'

Chat with history

curl http://localhost:11434/api/chat -d '
{
  "model": "llama3.2",
  "messages": [
    {"role": "user", "content": "What is a local LLM?"}
  ],
  "stream": false
}
'

Use the Python client

Install the official client:

pip install ollama

Generate text:

import ollama

response = ollama.generate(
    model="llama3.2",
    prompt="Write a Python function that reverses a string."
)
print(response["response"])

Chat with messages:

messages = [
    {"role": "user", "content": "What is retrieval-augmented generation?"}
]

for chunk in ollama.chat(model="llama3.2", messages=messages, stream=True):
    print(chunk["message"]["content"], end="")

Streaming responses

Streaming keeps your app responsive. The Python client returns a generator when stream=True.

stream = ollama.chat(
    model="llama3.2",
    messages=[{"role": "user", "content": "Tell me a joke about Python."}],
    stream=True
)

for chunk in stream:
    print(chunk["message"]["content"], end="", flush=True)

Maintain chat history

Store messages in a list and append assistant responses to keep context across turns.

history = []

def chat(user_input: str) -> str:
    history.append({"role": "user", "content": user_input})
    response = ollama.chat(model="llama3.2", messages=history, stream=False)
    reply = response["message"]["content"]
    history.append({"role": "assistant", "content": reply})
    return reply

Build a simple CLI assistant

import ollama

print("Local LLM assistant. Type "exit" to quit.")
history = []

while True:
    user_input = input("\nYou: ")
    if user_input.lower() == "exit":
        break
    history.append({"role": "user", "content": user_input})
    reply = ""
    for chunk in ollama.chat(model="llama3.2", messages=history, stream=True):
        reply += chunk["message"]["content"]
    print(f"AI: {reply}")
    history.append({"role": "assistant", "content": reply})

Next steps

• Create a custom model with a Modelfile
• Combine Ollama with LangChain or LlamaIndex for RAG
• Expose your local model through a FastAPI endpoint

Running LLMs locally with Ollama and Python is one of the fastest ways to prototype AI features while keeping full control of your data.