Why we self-host
Why we run our own LLMs - and skip the frontier
We self-host open-weights LLMs on our own hardware as our daily driver across projects - client web builds, CMS platforms, and the homelab that runs our own inference stack. It's the default we bring to new work. Here's the honest reasoning, trade-offs and all.
Reasons
Your data stays yours
No third-party API sees your documents, your prompts, or your answers. Everything - ingestion, embeddings, inference - runs on infrastructure you control. For teams in regulated industries, that isn't a nice-to-have; it's the point.
Costs that stay flat
Frontier APIs bill per token, forever - and the rate card moves when they feel like it. A self-hosted model has a fixed, predictable running cost. The more you query, the cheaper it gets per answer. Volume stops being a fear and starts being a win.
Latency and reliability you tune
No shared frontier API queue, no vendor rate limits, no surprise outages in someone else's cloud. We size the hardware, pin the model, and watch the metrics - your system's behaviour is yours to control, end to end.
No training on your data
When you call a frontier API, you inherit their data policy, their terms, and their roadmap. With an open-weights model on your own metal, the answer to 'who sees my data?' is simply: us - and only us.
Open weights, honest upgrades
The open-weights ecosystem moves fast. When a better model drops, we evaluate it, and we upgrade on our timeline - not when a vendor deprecates an API version. No lock-in, no renegotiation.
Works offline, works anywhere
Field sites, private networks, air-gapped environments - a local LLM runs where the internet doesn't. Your knowledge base keeps answering in places a hosted API simply can't reach.
The honest trade-offs
A local model is not a frontier model. On open, general knowledge and raw reasoning, the biggest hosted models still lead - and we will tell you when a problem genuinely needs them. But for the work most businesses actually need - answering over your knowledge, drafting in your voice, moving your data - a well-chosen open-weights model paired with a good RAG pipeline is most of the way there, at a fraction of the cost, with your data in the building. We pick the model, size the hardware, and prove it with evals before anyone relies on it.
Sometimes the right answer is a hybrid: local for the steady state, frontier for the hard edges. We'll design for that too - but the default starts on your own metal.
How it works in practice
- 01. We build the pipeline first - ingestion, chunking, embeddings, retrieval - so the model is answering from your knowledge, not its own imagination.
- 02. We shortlist open-weights models for your task, run them on our hardware, and compare them with evals built from your real questions.
- 03. You get a system you own: model, pipeline, and infrastructure, with monitoring, guardrails, and an upgrade path when the next model lands.
Wondering what this looks like for your stack?
Tell us what you'd want a model to answer about your business - we'll tell you exactly how we'd build it.
hello@coffeedrivendevelopment.co.nzNo frontier model required.