Let me paint you a picture. It's 2026 and everyone's talking about running AI locally. "Get yourself an RTX 4090," they say. "You need at least 32GB of RAM," they insist. Meanwhile, you're sitting there with a laptop that wheezes when you open three browser tabs.

Good news: those people are wrong. Or at least, dramatically overstating the minimum requirements.

I've been running useful LLMs on a ThinkPad with 4GB of RAM, and in this guide, I'll show you exactly how — no magic, no lies, just clever model selection and the right tools.

The Secret Sauce: Quantization

Full-size language models are enormous. Llama 3 8B, for example, normally needs about 16GB of memory. That's four times what our potato laptop has. So how do we fit a 16GB model into 4GB?

The answer is quantization — a technique that compresses model weights from 16-bit or 32-bit floating point numbers down to 4-bit or even 2-bit integers. Think of it like converting a WAV file to MP3. You lose some quality, but for most practical tasks, you genuinely cannot tell the difference.

A Q4_K_M quantized version of a 3B parameter model? That runs in about 2GB of RAM. Suddenly, our laptop doesn't look so pathetic.

Step 1: Install Ollama

Ollama is the easiest way to run local models. One command on Linux:

curl -fsSL https://ollama.com/install.sh | sh

On Windows or Mac, just download the installer from ollama.com. It takes about 30 seconds. No Python environments. No CUDA drivers. No PhD in systems engineering.

Step 2: Pick the Right Model

This is where most people go wrong. They download Llama 3 70B, watch their laptop catch fire, and conclude that local AI is impossible. Don't be that person.

For 4GB of RAM, here are your best options:

  • Phi-3 Mini (3.8B, Q4): Microsoft's tiny powerhouse. ~2.3GB. Surprisingly good at coding and reasoning.
  • Gemma 2 2B (Q4): Google's lightweight model. ~1.5GB. Great for summarization and quick Q&A.
  • TinyLlama 1.1B (Q4): The absolute minimum. ~700MB. It's not going to write your novel, but it can autocomplete code and answer basic questions.
  • Qwen 2.5 3B (Q4): Alibaba's sleeper hit. ~2GB. Excellent multilingual support.
# Pull a model that actually fits
ollama pull phi3:mini

# Or if you're really tight on RAM
ollama pull gemma2:2b

Step 3: Configure for Low Memory

By default, Ollama tries to be helpful and loads things into memory aggressively. On our budget hardware, we need to tell it to chill:

# Set environment variables before running
export OLLAMA_NUM_PARALLEL=1
export OLLAMA_MAX_LOADED_MODELS=1

# Start Ollama
ollama serve

These settings ensure Ollama only loads one model at a time and doesn't try to handle multiple requests simultaneously. Your laptop will thank you.

Step 4: Talk to Your Budget AI

Now the fun part. Open a terminal and type:

ollama run phi3:mini

You'll get an interactive chat prompt. The first response might take 10-15 seconds (the model needs to load into RAM), but subsequent responses should stream at 3-8 tokens per second. That's not blazing fast, but it's perfectly readable.

What Can You Actually Do With This?

Let's be honest — you're not going to run a customer-facing chatbot on this setup. But here's what works surprisingly well:

  • Code completion: Ask it to finish functions, write boilerplate, or explain error messages.
  • Summarization: Paste in a long article and get the key points.
  • Writing assistance: Grammar checking, rephrasing sentences, brainstorming ideas.
  • Learning: Ask it to explain concepts. It's like having a patient tutor who never judges your questions.
  • Private journaling assistant: Ask it reflective questions about your notes. Everything stays on your machine.

The Performance Reality Check

Here's my actual benchmarks on a 2019 ThinkPad T480 (i5-8250U, 4GB RAM, no GPU):

Model RAM Used Tokens/sec Quality
TinyLlama 1.1B Q4 ~900MB 8-12 Basic but usable
Gemma 2 2B Q4 ~1.7GB 5-8 Good for most tasks
Phi-3 Mini Q4 ~2.5GB 3-5 Impressively smart

Is it slower than ChatGPT? Obviously. Is it private, free, offline, and running on hardware you already own? Absolutely. And that trade-off is worth it for a lot of use cases.

Pro Tips for Squeezing Every Drop

  • Close your browser. I know, radical advice. But Chrome alone can eat 2GB of RAM. Use a lightweight text editor instead.
  • Add swap space. On Linux, create a 4GB swap file. It'll slow things down but prevent out-of-memory crashes.
  • Use the API, not the CLI. Build a simple HTML chat interface that hits localhost:11434. It uses less memory than the terminal.
  • Short context windows. Don't paste your entire codebase. Give the model focused, specific prompts.

The Bottom Line

You don't need expensive hardware to start experimenting with local AI. A 4GB laptop and a quantized model is enough to build real, useful tools. The barrier to entry has never been lower.

Stop waiting for better hardware. Start building with what you have. Your crappy laptop is more capable than you think.