BuildBotsBetter
MA

Setup Ollama with Multiple Agents

Multi-LLM Laboratory, VRAM Allocation, Stable Aliases & Dynamic Swapping

Progress
0%

Select Your Workstation Shell

Terminal commands update automatically for PowerShell, macOS zsh, and Linux bash.

Step 1 of 7 Est: 2 minutes

Audit GPU VRAM & System Memory Budget

Inspect physical VRAM and calculate concurrency limits before installing multiple model families.

Check GPU Capabilities

Query available video memory and dedicated hardware controller information:

PowerShell
PS > nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv
zsh
$ system_profiler SPDisplaysDataType | grep -E "Chipset Model|VRAM"
bash
$ nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv
Hardware Rule: On an 8 GB VRAM GPU (like NVIDIA RTX 5070), each ~7B-8B Q4 quantized model requires 4.5-6.5 GB VRAM depending on context window. Attempting to hold two 8B models concurrently will exhaust VRAM and spill into system RAM, causing a 10x-20x drop in token generation speed. The correct pattern is to install all desired models, but let Ollama dynamically swap the active agent into GPU memory.
Checkpoint 1
Required

Paste the output from nvidia-smi or display profiler showing GPU name and total VRAM:

Step 2 of 7 Est: 3 minutes

Establish Model Registry & Availability Verification

Verify local model availability versus cloud-only models before pulling weights.

Multi-Agent Target Registry

Here is the verified target registry designed specifically for an 8 GB VRAM host:

Agent Name Actual Model Provider Type Params Context Size GPU Status
ollama_qwen qwen3:8b-8k Alibaba LOCAL 8B 8K 5.2 GB 100% GPU (~6.2 GB)
ollama_llama llama3.2:3b Meta LOCAL 3B 4K/8K 2.0 GB 100% GPU (~2.6 GB)
ollama_deepseek deepseek-r1:8b DeepSeek LOCAL 8B 4K/8K 4.9 GB 100% GPU (~5.8 GB)
ollama_gemma gemma2:2b Google LOCAL 2.6B 8K 1.6 GB 100% GPU (~2.2 GB)
ollama_mistral mistral:7b Mistral AI LOCAL 7.2B 4K/8K 4.1 GB 100% GPU (~5.1 GB)
ollama_glm glm4:9b Zhipu AI LOCAL 9.4B 4K/8K 5.5 GB 100% GPU (~6.6 GB)
ollama_kimi kimi-k2 / kimi-k3 Moonshot CLOUD ONLY 1T+ MoE 128K N/A NOT AVAILABLE AS A PRACTICAL LOCAL MODEL
Checkpoint 2
Required

Paste the status of Kimi confirming why it cannot run locally on an 8 GB VRAM workstation:

Step 3 of 7 Est: 5 minutes

Pull Verified Model Families

Download quantized local weights for Llama, DeepSeek, Gemma, Mistral, and GLM.

Execute Sequential Model Downloads

Pull each local model individually to monitor disk and network usage:

Terminal
# Pull LLaMA (Meta 3B parameter model, ~2.0 GB) > ollama pull llama3.2:3b # Pull DeepSeek (Reasoning distilled model, ~4.9 GB) > ollama pull deepseek-r1:8b # List all pulled model weights > ollama list
Checkpoint 3
Required

Paste the output of ollama list showing your downloaded model families:

Step 4 of 7 Est: 3 minutes

Create Stable Agent Aliases via Modelfiles

Decouple the agent identity (ollama_*) from the underlying model tags using layer reuse.

Generate Agent Modelfiles

Create Modelfiles that reference the underlying weights and define custom agent identities:

PowerShell
# Create ollama_qwen pointing to qwen3:8b-8k PS > "FROM qwen3:8b-8k" | Set-Content Modelfile_qwen PS > ollama create ollama_qwen -f Modelfile_qwen # Create ollama_llama pointing to llama3.2:3b PS > "FROM llama3.2:3b" | Set-Content Modelfile_llama PS > ollama create ollama_llama -f Modelfile_llama # Create ollama_deepseek pointing to deepseek-r1:8b PS > "FROM deepseek-r1:8b" | Set-Content Modelfile_deepseek PS > ollama create ollama_deepseek -f Modelfile_deepseek
zsh
$ echo "FROM qwen3:8b-8k" > Modelfile_qwen && ollama create ollama_qwen -f Modelfile_qwen $ echo "FROM llama3.2:3b" > Modelfile_llama && ollama create ollama_llama -f Modelfile_llama $ echo "FROM deepseek-r1:8b" > Modelfile_deepseek && ollama create ollama_deepseek -f Modelfile_deepseek
bash
$ echo "FROM qwen3:8b-8k" > Modelfile_qwen && ollama create ollama_qwen -f Modelfile_qwen $ echo "FROM llama3.2:3b" > Modelfile_llama && ollama create ollama_llama -f Modelfile_llama $ echo "FROM deepseek-r1:8b" > Modelfile_deepseek && ollama create ollama_deepseek -f Modelfile_deepseek
Checkpoint 4
Required

Paste the output from ollama create showing manifest creation for a stable alias:

Step 5 of 7 Est: 3 minutes

Model Switching, Lifecycle & Memory Management

Inspect active GPU allocation and understand how Ollama unloads idle models.

Check Loaded Model Status

Monitor which model currently occupies GPU VRAM using ollama ps:

Terminal
> ollama ps
Switching Mechanics: When you invoke ollama run ollama_llama while ollama_qwen is loaded, Ollama automatically frees the previous model from VRAM and loads the new weights in ~2 seconds. To force immediate unloading without waiting for the 5-minute timeout, send an empty request with keep_alive: 0.
Checkpoint 5
Required

Paste the output of ollama ps showing active model allocation on 100% GPU:

Step 6 of 7 Est: 4 minutes

Standardized Evaluation & Ground Truth Testing

Execute standardized prompts across your agents and record ground truth versus model responses.

Run Standardized Identity Prompt

Send the benchmark prompt to your stable alias without interactive terminal locks:

PowerShell
PS > "Identify yourself by giving your model family, actual model/tag if known, whether you are running locally through Ollama, and your context length. Do not guess." | ollama run ollama_llama
zsh
$ echo "Identify yourself by giving your model family, actual model/tag if known, whether you are running locally through Ollama, and your context length. Do not guess." | ollama run ollama_llama
bash
$ echo "Identify yourself by giving your model family, actual model/tag if known, whether you are running locally through Ollama, and your context length. Do not guess." | ollama run ollama_llama
Checkpoint 6
Required

Paste a response snippet from your agent evaluation prompt:

Step 7 of 7 Est: 2 minutes

BuildBotsBetter Laboratory Integration & Reference Card

Connect autonomous agent runners to your local models and save the lab reference card.

Autonomous Agent Connection

Point your coding agent harness to the local Ollama OpenAI-compatible server:

Terminal
# Environment for any OpenAI-compatible agent harness export OPENAI_BASE_URL="http://127.0.0.1:11434/v1" export OPENAI_API_KEY="ollama" export OPENAI_MODEL="ollama_llama"
Checkpoint 7
Required

Paste your agent endpoint environment variables or routing command:

⚡ Run It Later — Multi-Agent Laboratory Reference

Run Qwen Agent ollama run ollama_qwen Starts Alibaba Qwen coding agent
Run LLaMA Agent ollama run ollama_llama Starts Meta LLaMA 3.2 agent
Run DeepSeek Agent ollama run ollama_deepseek Starts DeepSeek R1 reasoning agent
Inspect Active VRAM ollama ps Displays currently loaded agent and GPU memory

© 2026 BuildBotsBetter. All rights reserved.

BuildBotsBetter's original characters, character designs, artwork, logos, graphics, scripts, animations, and original productions are owned by BuildBotsBetter or their respective rights holders, to the extent applicable. These materials are protected by copyright, trademark, and other applicable intellectual property laws, to the extent such protection is available under applicable law. No permission is granted to use BuildBotsBetter-owned materials in a manner that infringes applicable rights, except as permitted by law. Third-party footage, trademarks, and other materials remain subject to their respective owners' rights and applicable license terms. Attribution does not imply ownership, endorsement, or permission to reuse third-party content.