---
url: https://lettuceai.app/docs/ollama
title: "Ollama Integration — LettuceAI"
description: "Connect LettuceAI to your Ollama server for local AI model inference with full configuration options."
---

Menu 

# Ollama Integration

If you already have **Ollama** running on your computer or local network, LettuceAI can connect to it as a provider. This is useful if you prefer Ollama's model management or want to share one Ollama server across multiple apps.

Ollama vs built-in llama.cpp

LettuceAI also has a built-in llama.cpp engine that runs models directly inside the app with no external software. See the [Model Browser & Local Inference](/docs/model-browser) page for that approach.

## Connecting to Ollama

1.  Make sure Ollama is running (default port is `11434`)
2.  In LettuceAI, go to **Settings → Providers → Ollama**
3.  Enter your Ollama base URL (e.g. `http://localhost:11434`)
4.  The app will fetch your installed models from Ollama's `/api/tags` endpoint

You do not need to download models through LettuceAI when using Ollama. Ollama manages its own model storage. Just pull models through Ollama's CLI and they will appear in LettuceAI.

### Remote access

Ollama does not have to run on the same device. If you have it running on another machine on your local network, use that machine's IP address as the base URL (e.g. `http://192.168.1.100:11434`).

Make sure Ollama is configured to accept connections from other devices. By default it only listens on localhost.

## Model management

Once connected, LettuceAI automatically discovers all models installed in Ollama. Each model appears with its display name and parameter size extracted from Ollama's metadata.

To add new models, use the Ollama CLI on the machine running Ollama:

```
ollama pull llama3.1
ollama pull mistral
ollama pull gemma2
```

After pulling, refresh the model list in LettuceAI and the new models will appear.

## How it works

LettuceAI communicates with Ollama using its native `/api/chat` endpoint. Conversations are streamed in real-time, and tool/function calling is supported for models that support it.

The app automatically handles URL normalization. It will strip a trailing `/v1`, `/api/chat`, or `/api/tags` from the base URL so you can paste in whatever form is handy.

## Configuration

Ollama exposes a rich set of parameters that you can configure per-model. Go to **Settings → Models**, open one of your Ollama models, and use the **Runtime** panel (labelled "Ollama"). It is grouped into **Memory & Tokens**, **Performance**, and **Sampling & Penalties** sections. Every field starts blank or on Auto, which leaves Ollama's own default in place.

Connection vs per-model settings

The base URL and credentials live in **Settings → Providers → Ollama**. The generation parameters below live on each model under **Settings → Models**.

### Context and generation

| Setting | Range | What it does |
| --- | --- | --- |
| Context Length | 0 – 262,144 | How many tokens of conversation history the model can see. |
| Max Predict | 0 – 131,072 | Maximum tokens to generate per response. |
| Num Keep | 0 – 32,768 | Number of tokens to keep from the beginning of the prompt. Useful for preserving system instructions. |

### Hardware

| Setting | Range | What it does |
| --- | --- | --- |
| Num GPU | 0 – 512 | Layers to offload to GPU. More = faster, but uses more VRAM. |
| Num Thread | 1 – 256 | CPU thread count for generation. |
| Num Batch | 1 – 16,384 | Tokens processed at once during prompt evaluation. |

### Sampling

| Setting | Range | What it does |
| --- | --- | --- |
| Seed | 0 – 2,147,483,647 | Fixed seed for reproducible output. 0 = random. |
| Repeat Penalty | 0 – 2 | Penalizes repeated tokens. Higher values reduce repetition. |
| Min P | 0 – 1 | Minimum probability threshold for token selection. |
| Typical P | 0 – 1 | Typical sampling. Keeps tokens whose surprise is close to the average. |
| TFS Z | 0 – 1 | Tail free sampling threshold. 1 = disabled. |

### Mirostat

Mirostat is an adaptive sampling algorithm that tries to keep the output at a consistent level of surprise. It can produce more natural and coherent text than fixed sampling parameters.

| Setting | Range | What it does |
| --- | --- | --- |
| Mirostat | 0 – 2 | Algorithm version. 0 = disabled, 1 = Mirostat v1, 2 = Mirostat v2. |
| Tau | 0 – 10 | Target entropy. Lower values produce more focused, predictable output. Higher values allow more variety. |
| Eta | 0 – 1 | Learning rate. Controls how quickly the algorithm adapts. |

When Mirostat is enabled (1 or 2), temperature and top-p sampling are bypassed. Mirostat controls the randomness on its own.

### Stop sequences

You can define a list of strings that cause the model to stop generating when encountered. This is useful for preventing the model from running past a natural stopping point.

## Reasoning, images, and model management

-   **Reasoning**: if you set a reasoning effort on a model, LettuceAI passes it through to Ollama's thinking support, so models that can reason will do so.
-   **Image input**: images you attach in chat are sent to Ollama vision models, so multimodal Ollama models can see them.
-   **Manage models from the app**: beyond pulling, you can view and delete models on the Ollama server from within LettuceAI, without switching to the command line.

### Self-signed or HTTPS Ollama servers

If your Ollama server sits behind a reverse proxy with HTTPS and a self-signed or private certificate, you have two options:

-   **Recommended**: import the certificate under **Settings → Security → Trusted Certificates → Custom Root CAs**. This makes LettuceAI trust the certificate properly.
-   **Quick but less secure**: turn on **Allow Invalid TLS** when editing the Ollama provider under **Settings → Providers**. This skips certificate checks for that one endpoint only.

## Ollama vs built-in llama.cpp

|     | Ollama | llama.cpp (built-in) |
| --- | --- | --- |
| Setup | Requires Ollama installed and running | Nothing extra, runs inside the app |
| Model source | Pull models through Ollama CLI | Download GGUF files from HuggingFace browser |
| Configuration | Simpler, Ollama handles most engine details | Full control over GPU layers, threads, KV cache, etc. |
| Network | Connects over HTTP (localhost or LAN) | Fully offline, no network needed |
| Best for | Sharing one server across apps, easier model management | Maximum control and fully offline use |

## Tips

-   If models are not appearing, make sure Ollama is running and the base URL is correct.
-   For remote Ollama servers, check that the firewall allows connections on port 11434.
-   Ollama's context window defaults can be small. Set a higher context window in LettuceAI's advanced settings if your conversations are getting truncated.
-   Try Mirostat v2 with tau around 5 for a good balance of coherence and creativity in roleplay.
-   If you want the same model accessible from multiple apps on your machine, Ollama is the better choice since it runs as a shared service.

[

PreviousMulti-GPU Local Models

](/docs/multi-gpu)[

NextSprout Hardware Probe

](/docs/sprout)