Sam Altman has said training GPT-4 cost north of $100 million. DeepSeek then claimed it trained a frontier-class model for a reported $5.6 million in compute — a number that helped knock roughly $589 billion off Nvidia's market value in a single January 2025 session. Both figures are quoted constantly, and both are useless to the founder who walks into a call and says, "We want our own custom LLM." Because almost nobody who says that sentence means either of those things. They mean something far smaller, far cheaper, and far better — and the distance between what they say and what they need is exactly where budgets go to die.
"Custom LLM" is four different things, and buyers usually mean the cheapest one
The phrase covers at least four jobs that share no price tag, no timeline, and no risk profile. One: pretraining a model from scratch — the GPT-4 tier, tens to hundreds of millions of dollars and a research org to spend it. Two: continued pretraining, where you take open weights and pour domain text over them — cheaper, still a serious data-and-compute project. Three: fine-tuning, usually with LoRA or similar, where you nudge an open model's behavior on a few thousand examples for compute that ranges from a few hundred to a few thousand dollars depending on model size and dataset. Four: changing zero weights — wrapping a frontier model in retrieval, prompts, structured outputs, and tools through an API. When a normal company says "custom LLM," they almost always need number four. They have been sold the story of number one.
The conventional narrative — own the weights, own the moat — is mostly wrong
The lazy consensus goes like this: frontier models are commodities, your data is your edge, so train the model on your data and the edge becomes defensible and yours. It sounds right. It is the kind of thing that survives a board meeting. It is also, for the overwhelming majority of businesses, a category error. Model weights are the most expensive, most perishable, most replaceable layer in the entire stack. The frontier moves every few months; whatever you train today is depreciating against a model you could have rented next quarter for less. You do not get a moat by owning a snapshot of last year's capability. You get a liability with a GPU bill attached.
The moat, when there is one, lives nowhere near the weights. It lives in the data you can retrieve and keep current, the way you chunk and rank and ground it, the evaluation harness that tells you when a change made things worse, and the product wrapped around all of it. None of that requires owning a model. All of it requires engineering discipline that the "train your own LLM" pitch quietly skips over, because discipline is harder to sell than a GPU cluster.
When fine-tuning is actually the right tool — and when it's theater
Fine-tuning earns its place on a specific, narrow class of problem: high-volume, stable, well-defined tasks where you want a smaller, cheaper, faster model to behave a particular way. Classification. Extraction into a rigid schema. Tone and format adherence at scale. Shaving latency and per-call cost off a task you run a million times a day. If you have thousands of clean, labeled examples of the exact behavior you want repeated, fine-tuning is legitimate engineering, not theater.
The mistake — and it is the single most common one we see — is fine-tuning to inject knowledge. Teams want the model to "know" their product catalog, their policies, their last quarter's tickets, so they fine-tune on those documents and are baffled when the model confidently invents things and forgets the rest. Fine-tuning teaches a model how to behave, not what is true this morning. Knowledge that changes belongs in retrieval, where you can update it without retraining anything and trace every answer back to a source. Confusing the two is how a six-figure project ends up worse than a weekend of prompt engineering.
When you should train from scratch: almost never
Pretraining your own model makes sense if you are a frontier lab, or if you genuinely operate in a data regime the public models have never seen — a proprietary modality, a specialized scientific corpus, a language or symbol system absent from the open web — and you have the capital and the research staff to sustain it. That is a real but tiny club. For everyone else, "we'll train our own foundation model" is a sentence that should trigger a hard conversation about what specific capability the frontier models are actually failing at. In practice the answer is usually "none, we just haven't built the retrieval and tooling yet."
What we actually build when a client asks for a custom LLM
Our Python and AI-ML practice has spent the past couple of years shipping production AI pipelines, and the request almost always arrives framed as a model problem. It almost always resolves into a systems problem. What gets built is retrieval-augmented generation and AI search, agentic workflows, document processing, and LLM integration on top of frontier models from OpenAI, Anthropic, Google, or open weights like Llama and Mistral — orchestrated with tools like LangChain, LangGraph, and LlamaIndex, grounded in a vector store such as Pinecone, Qdrant, Weaviate, or pgvector. The "custom" part is the data plumbing and the product logic, not a private set of weights. For one entertainment client we built a generative AI application that reads screenplays and generates visual scene compositions — work that used to take a sketch artist months, compressed into minutes. There is no bespoke foundation model in it. There is a great deal of bespoke engineering around a borrowed one.
The part the "train your own" pitch never mentions is evaluation, and it is the part that decides whether any of this survives contact with real users. You instrument the system — Langfuse, Phoenix, LangSmith — so that every prompt change, retrieval tweak, or model swap is measured against a fixed set of cases instead of vibes. We review accuracy the way we review code: every change defended, nothing merged on a hunch, because a senior engineer signs off on the pull request. AI tools speed that loop up; they don't replace the judgment in it. A team that can't tell you whether last week's change helped or hurt does not have a custom LLM. It has a slot machine with a system prompt.
The honest test: a custom model, or a custom system?
Here is the question that settles it, and it costs nothing to ask before you spend anything. Is the frontier model failing because it lacks a capability, or because it lacks your context? If it's context — your documents, your data, your rules, your formats — you have a retrieval and tooling problem, and training a model is the most expensive possible way to not solve it. If it's a genuine capability gap on a narrow, repetitive task at scale, fine-tune a small open model and measure the result. If it's neither, and you're three months from a foundation-model project, someone is selling you the GPU bill as the strategy.
So commit to the unglamorous version. For nearly every company, the custom LLM worth building is a custom system — retrieval, context, tools, and evals wrapped around a model you rent and replace as the frontier moves. The weights are the commodity. Your data, your product, and the engineering that connects them are the asset. Spend accordingly, and stop paying foundation-lab prices to relearn what RAG already does for the cost of doing it properly.
Last updated July 9, 2026