Skip the marketing. Here’s what actually works.

Local AI coding isn’t a replacement for Claude or GPT-4o. It’s a different tool with different trade-offs. You get privacy, zero per-token costs, and offline capability. You give up raw capability, frontier performance, and multi-modal features. That’s the deal. Know it before you start.

If that trade-off makes sense for your workflow, keep reading.

Why Bother?

Three reasons to go local:

  1. Cost. API fees add up fast. Running a local model once you’ve paid for the hardware costs nothing extra. At scale, this matters.
  2. Privacy. Your code never leaves your machine. Can’t put a price on that for enterprise or sensitive work.
  3. Offline. Plane mode, remote locations, air-gapped environments — local works where cloud doesn’t.

If none of those apply to you, cloud is probably the better call. Don’t fight the tool for ideology.

The Setup: Ollama vs LM Studio

Two tools dominate. Here’s the honest breakdown:

Ollama — The CLI-first choice. Pull a model, run it, done. Dead simple setup. Best if you’re comfortable in a terminal and want to script around your model.

# Install, then pull and run
ollama pull codellama:13b
ollama run codellama:13b

LM Studio — The GUI option. Better for experimenting with different models, has a built-in chat interface, and you can run a local API server that mimics the OpenAI interface. Best if you want a drop-in Copilot replacement without CLI fiddling.

Both are free. Both work on Mac, Windows, and Linux. LM Studio wins for beginners. Ollama wins for automation.

The Models Worth Running

Not all models are created equal. Here’s what actually produces usable code:

DeepSeek-Coder (6.7B / 33B)

The current sweet spot for local coding. Trained specifically on code, understands context windows up to 128K tokens in the larger variants. The 6.7B runs on a decent laptop. The 33B needs a decent GPU but produces noticeably better output. Start here.

ollama pull deepseek-coder:6.7b

Qwen2.5-Coder (7B / 14B)

Alibaba’s coding model. Strong performance for the size, good at explaining code and helping with debugging. The 14B variant holds its own against larger models in many benchmarks. Worth trying alongside DeepSeek.

CodeLLama (7B / 13B / 34B)

The original coding-focused model. Still relevant but showing its age compared to DeepSeek and Qwen2.5-Coder. Use if you want a proven, well-documented option. Otherwise, the newer models edge it out.

What to Skip

  • Mistral 7B — Good general model, mediocre at code. Fine for explanations, not for generation.
  • Llama 3 (non-coding variant) — General-purpose. Not terrible, not good for code specifically.
  • Anything under 6B parameters — Too weak for serious coding work. You’ll spend more time correcting output than writing code.

What Local Does Well

Be realistic about the use cases:

  • Boilerplate and scaffolding — Generate standard patterns fast. Local models handle this well.
  • Code explanation — “What does this function do?” — works great locally.
  • Debugging assistance — Describe the error, get suggestions. Solid.
  • Refactoring — Local models are fine at this. Give them context, get back clean code.
  • Writing tests — Works well. Models understand test patterns.

When to Reach for the Cloud

Local has hard limits:

  • Complex architectural decisions — DeepSeek can suggest, but Claude or GPT-4o reasons better about system design.
  • Multi-file refactoring — Works locally in theory, but frontier models handle cross-file context better.
  • Novel problems — If no one has solved this on Stack Overflow, local models struggle more than cloud.
  • Multimodal — Can’t see screenshots, can’t read PDFs of documentation.

If local gives you a bad answer twice, switch to cloud. Don’t waste time beating a dead model.

The Honest Hardware Check

  • MacBook Air / 8GB RAM — Run 6-7B models only. DeepSeek 6.7B works. Don’t push it.
  • MacBook Pro / 16-32GB RAM — 13B models comfortably. CodeLLama 13B, DeepSeek 13B.
  • Desktop with dedicated GPU (RTX 3080+ / M-series Pro/Max) — 33B+ models. This is where local gets genuinely useful.

The jump from 7B to 13B is the biggest quality improvement you’ll notice. Worth the extra hardware.

Connecting to Your Editor

This is where it gets practical. Most editors speak OpenAI’s API format. Local servers speak the same format. Point your editor at http://localhost:1234/v1/chat/completions and you’re running local inference in VS Code, Zed, or whatever you use.

LM Studio exposes this by default. Ollama needs a proxy like ollama-lazy or running ollama serve and using the compatible endpoint.

The Bottom Line

Local AI coding is real and works. It’s not marketing — you can genuinely replace a chunk of your API usage with a local model that costs nothing per token.

The trade-off: it’s slower, less capable on hard problems, and requires hardware investment. For boilerplate, scaffolding, code explanation, and routine tasks — local is fine. For everything else, cloud is still winning.

Run both. Use local by default, escalate to cloud when you need to. That’s the practical setup.

Hardware constraints are real. Start with the smallest model that works for your machine before buying anything.