Local AI Lab
Claims about local models, checked against my own VRAM
Why bother
Every claim about what a local model can do is worth exactly as much as the hardware budget of the person making it. Benchmarks are run on cards nobody has at home, and the interesting question is not whether a model is good but whether it is good enough on twelve gigabytes.
The setup
An RTX 3060 with 12 GB of VRAM, Ollama and LM Studio, and a fixed set of agentic development tasks to run everything against so the comparison means something.
Models under evaluation include Qwen, Gemma, Llama 3.1, DeepSeek R1 and Mistral Nemo.
What it actually tells me
Which development workflows survive without a frontier model and which quietly fall apart. Tool calling and MCP integrations are where the gap shows first: a model that writes acceptable code will still lose the thread across a multi-step agentic loop long before it runs out of context.
It is also where I find out what a workflow costs when the marginal token is free, which is the other half of the question TokenMeter asks.