🚀 The Ultimate Guide to Running Local LLMs on Any PC

A Practical, Step-by-Step Guide to Running AI Models Locally

---

📖 Introduction

What Are Large Language Models (LLMs)?

Large Language Models are AI systems trained on vast amounts of text data to understand, generate, and manipulate human language. Think of them as incredibly advanced text prediction engines that can write essays, answer questions, write code, and even engage in creative writing.

Why Run LLMs Locally?

In recent years, cloud-based AI assistants like ChatGPT, Claude, and Gemini have become incredibly popular. However, running models on your own computer offers significant advantages:

Benefit	Description
🔒 Complete Privacy	Your data never leaves your machine – perfect for sensitive documents, medical information, or proprietary code
🌐 Offline Access	Work anywhere without internet connectivity – great for travel, remote locations, or secure environments
💰 No Subscription Fees	One-time setup, forever free – no monthly subscriptions or per-request charges
🎨 Full Customization	You control everything – model choice, system prompts, parameters, and fine-tuning
🚫 No Content Filters	Unrestricted access – useful for research, creative writing, and uncensored exploration
📈 Learning Opportunity	Understand how AI works under the hood – invaluable for developers and AI enthusiasts

Who Is This Guide For?

This comprehensive guide is designed for:

• 👨‍💻 Developers wanting to integrate AI into their applications without cloud costs

• 🔬 Researchers who need complete control over their AI environment

• ✍️ Writers and Creators seeking privacy for their work

• 🎓 Students learning about machine learning and AI

• 🏢 Businesses with data privacy requirements

• 🤖 AI Enthusiasts wanting to explore state-of-the-art models

---

💻 System Requirements

Hardware Requirements

Before diving in, let's check if your hardware is ready. Here's what you need:

Minimum Requirements (Will Run Small Models)

Component	Specification
RAM	8 GB
Storage	20 GB free (SSD recommended)
CPU	4+ cores, 2.0 GHz+
GPU	Optional (CPU-only is fine for small models)
OS	Windows 10+, macOS 12+, or Linux (Ubuntu 20.04+)

Recommended Requirements (Good Performance)

Component	Specification
RAM	16 GB
Storage	50 GB free (NVMe SSD recommended)
CPU	8+ cores, 3.0 GHz+
GPU	NVIDIA 6GB+ VRAM (RTX 3060/4060 minimum) or AMD with ROCm support
OS	Windows 11, macOS 14+, or Linux (Ubuntu 22.04)

Optimal Requirements (Best Experience)

Component	Specification
RAM	32-64 GB
Storage	200+ GB free (NVMe SSD)
CPU	12+ cores, 3.5 GHz+ (e.g., Ryzen 9, Intel i9)
GPU	NVIDIA 12GB+ VRAM (RTX 3090/4090) or equivalent
OS	Windows 11 Pro, macOS 15+, or Linux (Ubuntu 22.04+)

How to Check Your System

Windows:

powershell

# Check RAM and CPU
wmic memorychip get capacity
wmic cpu get name,numberofcores

# Check GPU
wmic path win32_VideoController get name,adapterram

# Check Storage
wmic logicaldisk get size,freespace,caption

macOS:

bash

# System info
system_profiler SPHardwareDataType

# GPU info
system_profiler SPDisplaysDataType

# Storage info
df -h

Linux:

bash

# CPU and RAM
lscpu
free -h

# GPU (NVIDIA)
nvidia-smi

# GPU (AMD)
rocm-smi

# Storage
df -h

🛠️ Preparation

Installing Python and Package Managers

Python is essential for most LLM tools. Here's how to set it up:

Windows

powershell

# Download Python from python.org
# Or use winget (Package Manager)
winget install Python.Python.3.11

# Verify installation
python --version

macOS

bash

# Using Homebrew (recommended)
brew install python@3.11

# Or download from python.org

# Verify
python3 --version

Linux (Ubuntu/Debian)

bash

# Install Python
sudo apt update
sudo apt install python3.11 python3-pip python3-venv

# Verify
python3 --version

Setting Up Virtual Environments

Virtual environments isolate your Python packages – crucial for avoiding conflicts.

Create Virtual Environment

bash

# Create venv
python -m venv llm_env

# Activate on Windows
llm_env\Scripts\activate

# Activate on macOS/Linux
source llm_env/bin/activate

# Upgrade pip
pip install --upgrade pip

GPU Drivers and Dependencies

OBS Browser Source Not Working? How to Interact With Web Pages Inside OBS Studio

If you're using OBS Studio to display a website, YouTube page, dashboard, live poll, chat widget, or web application, you may notice something confusing:

The webpage appears perfectly inside OBS — but you can't click anything.

This is usually not a broken Browser Source.

The key is understanding how OBS handles browser interaction.

The Problem

A typical OBS Browser Source looks like this:

OBS Studio
┌──────────────────────────────────────┐
│                                      │
│        WEB PAGE / YOUTUBE            │
│                                      │
│       [ Button ]                     │
│                                      │
└──────────────────────────────────────┘

The page is rendered by OBS, but clicking directly on the OBS preview isn't how you interact with the webpage.

OBS's official documentation describes Browser Source as an embedded web browser that can display web pages, local files, widgets, alerts and other browser-based content.

The solution is Interact.

The Correct Way to Interact With a Browser Source

First select your Browser Source in the Sources panel.

Then either:

Method 1 — Interact button

Select the Browser Source and click:

Build Your Own Local AI Coding Agent: A Complete Multi-Platform Development System

 Imagine telling your computer:

“Build me a Python application, test it, fix the errors and package it.”

Or:

“Create an Android application and generate the APK.”

Or even:

“Build this .NET application for Windows.”

Instead of sending your source code to a cloud AI service, you could have your own local AI coding agent running on your computer.

The agent can understand requirements, write code, execute commands, run tests, debug errors, build applications and manage Git repositories.

Even better, the system can support multiple programming languages and platforms.

The practical way to build such a system is to use Windows as the host, an Ubuntu virtual machine as the development environment, local AI through Ollama, and specialized build machines for Windows and Apple platforms.

The Architecture

The overall architecture looks like this:

                         WINDOWS 11 PRO
                              │
              ┌───────────────┴────────────────┐
              │                                │
         NVIDIA GPU                       Windows Tools
              │                                │
           Ollama                    Visual Studio / MSBuild
              │                                │
              └──────────────┬─────────────────┘
                             │
                       Hyper-V Network
                             │
                    ┌────────▼────────┐
                    │   UBUNTU VM    │
                    │                │
                    │ OpenHands      │
                    │ Cline          │
                    │ Aider          │
                    │ Docker         │
                    │ Git            │
                    │ Python         │
                    │ Java           │
                    │ .NET           │
                    │ C/C++          │
                    │ Rust           │
                    │ Go             │
                    │ Node.js        │
                    │ Android        │
                    │ Flutter        │
                    └────────┬───────┘
                             │
                             │ SSH
                             ▼
                       OPTIONAL MAC
                             │
                       Xcode / Swift
                             │
                         iOS / macOS

This architecture separates AI, development and platform-specific compilation.

That separation is important.

Linux does not need to do everything.

Why Use an Ubuntu VM?

You could install everything directly on Windows, but a dedicated Linux development environment provides several advantages.

Ubuntu gives you:

• A clean development environment
• Native Linux tooling
• Docker
• Python
• Java
• C/C++
• Rust
• Go
• Node.js
• .NET
• Android development
• Flutter
• Linux builds
• Easy automation
• Easier agent sandboxing

Most importantly, the AI agent can operate inside a controlled environment without modifying your main Windows installation.

Your Windows machine remains your everyday desktop.

Ubuntu becomes your AI software factory.

Windows Remains Your Main Desktop

You don't have to sit inside the Ubuntu desktop all day.

Install VS Code on Windows and connect to Ubuntu using Remote SSH.

The experience looks like this:

Windows
   │
   ▼
VS Code
   │
   ▼
Ubuntu VM
   │
   ├── Source code
   ├── Git
   ├── Docker
   ├── Compilers
   └── AI agents

You see the familiar Windows VS Code interface.

But the code executes inside Ubuntu.

This gives you the best of both worlds.

Step 1: Enable Hyper-V

Windows 11 Pro includes Hyper-V.

Open PowerShell as Administrator and run:

Enable-WindowsOptionalFeature `
    -Online `
    -FeatureName Microsoft-Hyper-V `
    -All

Restart Windows after installation.

Then open:

Hyper-V Manager

Hyper-V will be the virtualization layer for your Ubuntu development machine.

Step 2: Create a Virtual Network

Open:

Hyper-V Manager
→ Virtual Switch Manager

Create an:

External Virtual Switch

Name it:

DevAgentSwitch

Connect it to your physical Ethernet or Wi-Fi adapter.

This allows your Ubuntu VM to communicate with:

• Windows
• Ollama
• Other computers
• Build servers
• Future Mac machines

Step 3: Create the Ubuntu VM

Download the current Ubuntu LTS desktop ISO.

Create a new Hyper-V VM with approximately:

Resource	Recommended
Generation	Generation 2
CPU	12 virtual CPUs
RAM	32 GB
Disk	500 GB
Network	DevAgentSwitch
OS	Ubuntu LTS

If your system has 64 GB RAM, 32 GB allocated to the development VM is a good starting point.

You can change the allocation later.

Step 4: Install Ubuntu

Install Ubuntu normally.

A simple hostname is:

localdev

After installation:

sudo apt update
sudo apt upgrade -y

Reboot:

sudo reboot

Verify the resources:

nproc
free -h
df -h

Step 5: Enable SSH

SSH allows Windows to access Ubuntu directly.

Inside Ubuntu:

sudo apt install -y openssh-server

Enable the service:

sudo systemctl enable --now ssh

Find the Ubuntu IP:

hostname -I

For example:

192.168.1.50

From Windows PowerShell:

ssh dev@192.168.1.50

Now you can control Ubuntu directly from Windows.

Step 6: Use VS Code From Windows

Install VS Code on Windows and add the:

Remote - SSH

extension.

Connect to:

dev@192.168.1.50

Now your workflow becomes:

Windows Desktop
      ↓
VS Code
      ↓
Ubuntu VM
      ↓
Development Environment

You can open, edit, compile and test Linux projects without leaving Windows.

Step 7: Install Docker

Docker is critical because it allows the AI agent to work in isolated environments.

Install Docker Engine and Compose in Ubuntu.

After installation:

docker run hello-world

The objective is to eventually have separate development environments such as:

Python container
Java container
Node container
.NET container
Rust container
C++ container
Android container

This prevents dependencies from different projects from interfering with each other.

Step 8: Keep Ollama on Windows

This is an important architectural decision.

Your NVIDIA GPU is physically installed in the Windows machine.

Instead of making GPU passthrough work inside Hyper-V, initially keep Ollama on Windows.

The architecture becomes:

                NVIDIA GPU
                    │
                    ▼
               Windows
                 Ollama
                    │
                    │ HTTP
                    ▼
              Ubuntu VM
                    │
                OpenHands

This is considerably simpler than configuring GPU passthrough.

Install Ollama on Windows and download a suitable coding model.

For example:

ollama run qwen3-coder:30b

The exact model/quantization you choose should depend on your available VRAM and system RAM.

Step 9: Connect Ubuntu to Ollama

The Ubuntu VM needs to communicate with Ollama running on Windows.

Conceptually:

Ubuntu
  │
  │ HTTP
  ▼
Windows:11434
  │
  ▼
Ollama
  │
  ▼
NVIDIA GPU

From Ubuntu, test the connection:

curl http://WINDOWS_IP:11434/api/tags

If the connection works, Ubuntu can use the Windows-hosted local model.

Do not expose Ollama's port to the public internet.

Keep it restricted to your private network.

Step 10: Install OpenHands

OpenHands will become the primary autonomous coding agent.

Install the required Python environment and OpenHands inside Ubuntu.

The resulting workflow is:

User
  ↓
OpenHands
  ↓
Local LLM
  ↓
Planning
  ↓
Code generation
  ↓
Terminal commands
  ↓
Testing
  ↓
Debugging
  ↓
Build

Unlike a simple autocomplete tool, an agent can perform multiple actions to accomplish a task.

Step 11: Add Cline

Cline can be used from VS Code for interactive development.

This gives you two different working modes:

OpenHands

Best for:

Autonomous tasks
Large projects
Long-running workflows
Testing
Debugging
Automation

Cline

Best for:

Interactive coding
Working directly inside VS Code
Reviewing changes
Making targeted modifications

You can connect both to your local Ollama instance.

Step 12: Add Aider

Aider gives you a powerful terminal-based interface.

For example:

aider --model ollama_chat/qwen3-coder:30b

You now have three interfaces:

              Local LLM
                  │
        ┌─────────┼─────────┐
        │         │         │
     OpenHands  Cline     Aider
        │         │         │
        └─────────┼─────────┘
                  │
              Git Project

Step 13: Install Programming Languages

The Ubuntu VM can become your universal development environment.

Install:

Python

sudo apt install python3 python3-venv python3-pip

Java

sudo apt install openjdk-21-jdk maven gradle

C/C++

sudo apt install build-essential gcc g++ clang cmake ninja-build

Go

sudo apt install golang-go

Rust

sudo apt install rustc cargo

Node.js

Install a current Node.js LTS environment.

.NET

Install the appropriate .NET SDK for the Ubuntu release.

The result is a single environment capable of working with:

Python
Java
Kotlin
C
C++
C#
.NET
Go
Rust
JavaScript
TypeScript
PHP
Ruby
and many others

Step 14: Android Development

Android is one of the major advantages of using Linux.

Install:

Android Studio
Android SDK
Android SDK Platform Tools
Android Emulator
Gradle

Then verify:

adb devices

The AI agent can eventually perform:

Generate project
      ↓
Write code
      ↓
Compile APK
      ↓
Launch emulator
      ↓
Install APK
      ↓
Run tests
      ↓
Read logcat
      ↓
Fix bugs
      ↓
Build final APK

This is where the system starts becoming genuinely autonomous.

Step 15: Flutter

Flutter is another important component.

A single Flutter project can target multiple platforms:

Android
iOS
Windows
Linux
macOS
Web

Linux can handle Android and Linux builds.

Windows can handle Windows builds.

A Mac can handle iOS and macOS builds.

This makes Flutter an excellent framework for the agent to use when cross-platform applications are required.

Step 16: Windows Application Builds

Your Windows host can act as the Windows build machine.

Install:

Visual Studio
MSBuild
Windows SDK
.NET SDK
CMake
Flutter

The architecture becomes:

Ubuntu Agent
      │
      │ Build request
      ▼
Windows Host
      │
      ▼
MSBuild / Visual Studio
      │
      ▼
EXE / MSIX

This means the AI can write code in Ubuntu while the native Windows toolchain performs the final build.

Step 17: macOS and iPhone

Apple platforms are different.

For genuine iOS/macOS builds, you eventually need a Mac running macOS and Xcode.

The architecture becomes:

Ubuntu
   │
   │ SSH
   ▼
Mac
   │
   ├── Xcode
   ├── Swift
   ├── CocoaPods
   └── iOS Simulator
          │
          ▼
       iOS build

The Mac doesn't need to run the AI.

It simply becomes your Apple build worker.

The Complete Build Pipeline

Once everything is implemented, a request such as:

Build an expense management application for Android and Windows.

could become:

USER REQUEST
     │
     ▼
AI PLANNER
     │
     ▼
Architecture
     │
     ▼
Technology Selection
     │
     ▼
CODE GENERATOR
     │
     ▼
Git Repository
     │
     ▼
Implementation
     │
     ▼
UNIT TESTS
     │
     ▼
BUILD
     │
 ┌───┴────┐
 │        │
Android  Windows
 │        │
 ▼        ▼
APK      EXE
 │        │
 └───┬────┘
     ▼
INTEGRATION TESTS
     │
     ▼
SECURITY REVIEW
     │
     ▼
DOCUMENTATION
     │
     ▼
RELEASE ARTIFACTS

The AI becomes more than a code generator.

It becomes a software engineering system.

The Project Structure

A useful directory structure is:

~/local-agent/

├── core/
│   ├── planner/
│   ├── coder/
│   ├── reviewer/
│   ├── debugger/
│   └── tester/
│
├── orchestrator/
│
├── builders/
│   ├── linux/
│   ├── windows/
│   ├── android/
│   └── macos/
│
├── runtimes/
│   ├── python/
│   ├── java/
│   ├── dotnet/
│   ├── node/
│   ├── rust/
│   └── cpp/
│
├── projects/
├── artifacts/
├── logs/
└── config/

This gives us a foundation for building a custom orchestration layer.

Add Git to Everything

Every project should be managed through Git.

The agent should follow a workflow such as:

Create branch
     ↓
Understand project
     ↓
Modify code
     ↓
Run formatter
     ↓
Run tests
     ↓
Build
     ↓
Review diff
     ↓
Commit

You should never allow an autonomous agent to blindly modify your only copy of a project.

Git becomes the safety net.

Add Agent Rules

Every repository can contain an AGENTS.md file.

For example:

Before modifying code:

1. Read the project documentation.
2. Inspect the existing architecture.
3. Do not delete working functionality unnecessarily.
4. Create a Git branch.
5. Implement the requested change.
6. Run formatting.
7. Run static analysis.
8. Run unit tests.
9. Build the application.
10. Fix failures.
11. Review the Git diff.
12. Commit only when validation succeeds.

This makes agent behavior considerably more predictable.

The Final System

Ultimately your computer becomes a local development platform:

                       LOCAL AI SOFTWARE FACTORY

                              USER
                               │
                               ▼
                        WEB DASHBOARD
                               │
                               ▼
                         AI ORCHESTRATOR
                               │
               ┌───────────────┼────────────────┐
               │               │                │
            Planner          Coder           Reviewer
               │               │                │
               └───────────────┼────────────────┘
                               │
                         Local LLM
                          Ollama
                               │
                               ▼
                         OpenHands
                               │
                     ┌─────────┴─────────┐
                     │                   │
                  Docker               Git
                     │                   │
                     ▼                   ▼
              Development             Repository
              environments
                     │
        ┌────────────┼─────────────┐
        │            │             │
      Linux       Windows         Mac
        │            │             │
      Build        Build          Xcode
        │            │             │
       APK         EXE           IPA

What Makes This Different?

A normal AI coding assistant might give you:

code

A local coding agent can potentially give you:

requirements
→ architecture
→ source code
→ dependencies
→ tests
→ debugging
→ builds
→ artifacts
→ documentation

And because the entire development environment can remain local, your source code and internal projects don't have to be sent to a third-party AI API.