GGUF vs Safetensors vs GPTQ: Local Model File Formats Explained
GGUF, Safetensors and GPTQ are not three versions of the same thing. GGUF and Safetensors are container formats, while GPTQ is a quantisation method whose weights are…
TAGGED GUIDES
#llama.cpp
Explore the guides, ideas, and practical explanations in this topic.
9 published articlesHow to Choose Quantization Settings for Local Models Without Breaking Output Quality
Start with Q4_K_M, then let the model repository override that default. This guide explains what GGUF quantization actually changes, how to read the documented quality, size, and…
Prompt Caching and Prefix Caching in Local LLMs: How They Reduce Memory and Latency
Prompt caching and prefix caching both reuse the key-value (KV) state a model already computed for an unchanged prompt prefix, so a new request can skip that…
Local AI: A Complete Guide to Running Language Models on Your Computer
Local AI means running a language model on hardware you control instead of sending prompts to a hosted service. This guide explains how local inference works, how…
How Context Length Affects Local AI Memory Usage
Raising a local model's context window from 2K to 32K changed generation speed by essentially nothing in a published llama.cpp sweep, but added about 1.7 GB of…
Why a Local AI Model Runs Slowly: A Troubleshooting Checklist
A slow local AI model is usually not one problem. Separate model loading, prompt processing, and token generation first, then check GPU offload, context length, quantization, and…
Connecting WordPress to Local AI: Architecture, Access, and Troubleshooting
Connecting WordPress to local AI means running an OpenAI-compatible inference server on hardware you control and registering it as an AI connector so the WordPress AI Client…
Why WordPress Cannot Connect to a Local AI Server
When WordPress fails to reach a local Ollama or llama.cpp server, the server is usually healthy and the connector is the problem. Reported causes cluster around the…
How to Record a Reproducible Local Model Benchmark
A reproducible local model benchmark is a record, not a number. It needs the exact model bytes and hashes, the runtime version, the hardware and power state,…