Your AI Feature Works in Demo and Breaks in Production. Here's Why — And How We Fix It.

It is 2:17 AM. Your phone lights up with the third Slack alert in an hour. A customer just screenshot the AI assistant on your dashboard confidently telling them their account balance is $4,300 negative. It is not. Your support inbox has 11 new tickets. The OpenAI billing page, when you finally have the courage to open it, shows you have spent $3,800 this month against a $500 budget. The demo three months ago was flawless. Your investors loved it. Your team shipped on time. And somewhere between the staging environment and the third week of real users, the thing quietly stopped working.

If you are reading this, you have probably lived some version of that night. Maybe the symptoms are different — drifted answers, latency creeping past four seconds, an agent that loops forever on edge cases, a RAG pipeline that retrieves the wrong document just often enough to embarrass you. The pattern underneath is the same. Your AI feature passed every test and now it is failing the only test that matters: real users, in real conditions, at scale.

This post is for the founder or CTO who has already shipped. You do not need to be sold on AI. You already paid the bill. What you need is a clear, honest map of why production AI breaks differently from normal software, and what an experienced engineering team actually does about it. We have walked into enough of these situations to know the territory.

The four failure modes nobody warned you about

After cleaning up a string of these projects, the same four patterns keep surfacing. They almost never show up alone. By the time the founder calls us, two or three are running at once.

1. The demo-to-production gap

This is the most common, and the most painful, because it is the one that makes you doubt your team. The system passes every test you wrote. It passes every test your developers wrote. And then it meets a real user who types in lowercase with three typos, pastes a screenshot, asks a follow-up that references something from yesterday, and the whole thing falls apart.

The reason is structural. Tests are written against the inputs the team imagined. Production receives the inputs reality produces. Those two distributions are never the same. The industry has started calling the result the green CI trap — your continuous integration pipeline is green, your monitoring dashboards are flat, your system is broken. It is broken because the tests confirm what you told the AI to do, not what the business actually needs from it. The behaviour passes; the meaning does not.

2. Epistemic debt — the code nobody on your team can explain

Traditional technical debt is messy code that everyone understands but nobody has time to clean up. The AI era introduced a worse cousin: code that works, but nobody on your team can explain why it works. A prompt was tuned three months ago, the model was upgraded, a retrieval threshold was nudged from 0.78 to 0.81 to fix a Friday bug, and now the whole pipeline is held together by decisions whose original logic has evaporated.

You see this most when you try to change something. A small tweak to the system prompt mysteriously breaks an unrelated feature. A model version bump from your vendor doubles your error rate. Nobody can predict what a change will do, so nobody changes anything, and the system slowly calcifies around the original mistakes. This is what your team means when they say "we can't really touch that part."

3. Silent drift and token bleed

The first two failure modes scream. This one whispers, which makes it worse. Your accuracy is slowly degrading because the input distribution is shifting. Your output token count is creeping up because conversation history is accumulating across calls. Nobody notices, because there is no Prometheus chart for "the model is gradually getting more wrong."

The cost side of this is the part founders feel first. We have seen production agents where a single architectural decision — re-billing the full conversation history on every step of a tool-using loop — drove costs up tenfold over the napkin estimate. One real, public example: a 50-engineer team set up an AI code review agent for pull requests and discovered it was eating about 400,000 input tokens per PR. Month-one bill: roughly $8,400 for what was supposed to be a cheap automation. The model was not the problem. The architecture was. The naïve agent loop has a cost shape that grows quadratically with steps, because every previous step's output gets re-sent on the next call. Nobody mentions this in the tutorial.

4. Integration brittleness

The fourth failure mode is mechanical. Your AI does not exist alone — it talks to your database, your auth system, your CRM, your billing provider, your file storage. Each of those handshakes is a place where the dev shop that built your system probably took a shortcut. Hardcoded credentials. A retry loop with no backoff. A tool definition that does not validate its inputs. A webhook that silently fails when the response is over a certain size. None of it shows up until it breaks in front of a customer.

Why this keeps happening to good companies

Take a step back. None of the failure modes above are the fault of a stupid engineer. They are the predictable result of a market that, between 2023 and 2025, learned to ship AI features at demo speed and never quite learned to ship them at production speed.

The honest version goes like this. Most AI projects in the last three years were built by teams treating LLM applications like ordinary web apps with a fancy API call inside. That model is wrong. A regular web service is deterministic — same input, same output. An LLM application is not. It is a non-deterministic system embedded inside a deterministic one, and the parts that break are mostly at the seams. Eval, observability, guardrails, cost attribution, fallback behaviour, model versioning — all of these are not nice-to-haves, they are the load-bearing infrastructure of a production AI system. The teams that skipped them shipped fast. The teams that built them shipped well. The bill arrives later for the first group.

The 10-point diagnostic you can run on your own system this week

You do not need to call anyone yet. Before you do, run this checklist on whatever you currently have in production. If you cannot say "yes, we have that" to most of these, you have found your problem.

1. You have an eval dataset built from real production logs, not invented test cases. At least 50 real conversations, scored by a human on the outcomes that matter to your business.
2. You log every LLM request and response, with metadata: user ID, feature name, model version, prompt version, input tokens, output tokens, latency, cost.
3. You can answer the question "how much did user X cost us this month" in under five minutes.
4. You have hard budget circuit breakers per feature, not just a global OpenAI cap. If one feature blows its budget, only that feature stops.
5. Your system prompt is versioned in source control like any other code, and prompt changes go through code review.
6. You have a guardrail layer that is separate from the model — input validation, PII redaction, output checks. Not "we asked the model not to do that in the prompt."
7. You know your P95 latency, not just your average. Averages hide the user experience for everyone in the tail.
8. You ran your agent against adversarial inputs before shipping. Typos, prompt injection attempts, contradictory instructions, missing context.
9. You have a defined fallback for every external call the AI makes. The database is down, the API returned 500, the tool input failed validation — what happens?
10. You have an owner. A specific engineer who is responsible for the AI system's behaviour in production, not just for shipping the next feature.

In our experience, the typical broken system we get called into score about 2 to 4 out of 10. The ones that work in production score 8 or above. The gap is the entire ballgame.

What a rescue project actually looks like

When a founder calls us after one of those 2 AM nights, we do not start with code. Code is the symptom. We start with a five-day audit. Here is roughly what happens.

Days 1–2: instrumentation. Before we change anything, we make the system observable. Logging, tracing, token attribution per user and per feature, P95 dashboards. You cannot fix what you cannot measure, and most broken AI systems are broken precisely because nobody could see them clearly.

Day 3: failure mapping. We build the eval dataset from your real logs — the actual conversations, not the imagined ones. We grade them. We find the failure modes. We rank them by business impact, not by how interesting they are technically.

Day 4: the rip-and-replace list. Some things should be patched. Some things need to be torn out and rebuilt. We tell you which is which, in plain English, and we tell you what each one will cost.

Day 5: the plan. A two-week stabilisation sprint, scoped and priced. At the end of two weeks you have a system that scores 8 out of 10 on the checklist above, or your money back. That is what we mean when we say senior engineers — it is not a job title, it is an accountability.

We do this work from Karachi, with engineers who have spent the last decade shipping software that has to actually work the next morning. The offshore rate is real. The seniority is also real. The combination is unusual, which is why we wrote this post the way we did.

A note on the broader pattern

There is a version of this article that ends with "and that is why you need to hire Brillnex." This is not that article. The honest ending is this: if you are stuck in any of the four failure modes above, the most important thing you can do this week is not hire anyone. It is to run the 10-point diagnostic on your own system, in writing, and force your current team to answer it.

If they can, you have a manageable problem and you should stay the course. If they cannot — if the answers come back vague, defensive, or worse, confident but unsupported — you have a different problem, and it is not one more sprint will fix.

Either way, you are now looking at your AI system the way it deserves to be looked at: as production infrastructure, not as a demo. That is the only mental model that survives contact with real users. Everything else is a story your team tells itself until the 2 AM Slack alert.

Frequently Asked Questions

Why does my AI feature work perfectly in testing but fail in production?

Tests reflect the inputs your team imagined. Production reflects the inputs reality produces. Real users type in lowercase, with typos, with context your tests never include. The fix is not more tests written by your team — it is an evaluation dataset built from real production logs and scored on real business outcomes. If you do not have one, you are still in the demo phase even if you have shipped.

My OpenAI bill is 5x to 10x what we estimated. Where is the money going?

Almost always in three places. First, system prompts get billed on every single call, not just the first one. Second, agent loops re-bill the full conversation history on every step — a 10-step agent run does not cost 10x a single call, it costs more like 55x because of how the math compounds. Third, retries from poorly-handled errors are silently doubling or tripling your real request count. The fix is per-user, per-feature cost attribution. Without it, you cannot tell whether you have a runaway agent, a prompt problem, or one power user inflating the whole bill.

What is LLM hallucination and can it actually be prevented in production?

A hallucination is when the model produces an answer that sounds confident but is factually wrong. It cannot be fully prevented, because non-determinism is built into how these models work. It can be made very rare in well-built systems through three layers: a grounded retrieval system that gives the model real facts to anchor on, a separate validation layer that cross-checks output against authoritative data before the user sees it, and a guardrail layer that catches the obvious failures. Teams that rely only on a clever system prompt to prevent hallucinations are the teams that end up screenshotted by angry customers.

What is "epistemic debt" and why is it different from normal technical debt?

Normal technical debt is code you understand but have not had time to clean up. Epistemic debt is code that works, but nobody on your team can explain why. AI systems accumulate it fast because prompts and retrieval thresholds get nudged over time, model versions change, and the original reasoning behind each tweak gets lost. The danger is that nobody can predict what a change will do, so the system silently calcifies. You notice it the first time a small tweak breaks something completely unrelated.

How long does it take to fix a broken AI integration?

In our experience, a typical mid-sized AI rescue takes about five days of audit and instrumentation, followed by a two-week stabilisation sprint. That gets a 2-out-of-10 system to roughly 8-out-of-10 on the diagnostic checklist. Larger systems, or systems where the underlying data is also broken, can take six to eight weeks. The five-day audit is the part that should never be skipped — every team that has tried to shortcut straight to "rebuild" has spent more, not less.

Should I just rebuild the whole thing from scratch?

Usually no. Roughly 70% of what your current team built is probably salvageable. The audit identifies the 30% that has to go. Founders who order a full rebuild without the audit step usually end up with a second broken system, because the same architectural assumptions that broke version one quietly carry over into version two. Diagnosis first, then surgery — not the other way around.

Is offshore really the right call for this kind of cleanup work?

It depends entirely on whether "offshore" means "junior engineers at a body shop" or "senior engineers at a focused team." The first option is what gave offshore a bad reputation, and we agree with that reputation. The second option is what we do — engineers who have spent a decade on production systems, working from Karachi at offshore rates because the cost of living here is different, not because the engineering is. The diagnostic checklist above is also a good interview question for any team you are evaluating. If they cannot walk you through it confidently, the location does not matter.

Brillnex Systems is an AI integration company based in Karachi, helping startups and businesses fix, harden, and ship custom AI features that survive contact with real users. If you are in the middle of one of the failure modes above and want a second pair of senior eyes on it, get in touch.