Local LLM with Ollama

🛠 Lab from the AI Platform Engineering Bootcamp. Used in Week 1. Bootcamp landing page: https://academy.tekanaid.com/bootcamps/ai-platform-engineering-bootcamp Parent course(s):

• Week 1: AI Foundations for Infrastructure Engineers (slug: ai-foundations-101)

🟡 Beta bootcamp lab. Hands-on instructions, check scripts, and solve scripts are in place. Lab is part of the running Platform Assistant capstone arc that grows across all 8 weeks of the bootcamp.

In this lab, you'll deploy a local LLM environment using Ollama and build a Python client that communicates with it.

You'll learn to:

• Deploy and configure Ollama for local LLM inference
• Pull and manage multiple model versions
• Build a Python client that queries local models
• Implement streaming responses for better UX
• Compare model performance and select appropriate models

Prerequisites:

• Python programming basics
• Understanding of APIs and HTTP requests
• Completion of previous AI Foundations labs

Key Concepts:

• Local model inference vs cloud APIs
• Ollama REST API
• Streaming response patterns
• Model selection criteria