Shipping AI that does not invent prices

The fastest way to lose trust in an AI product is to show a number that is wrong. A price, a date, a balance, an eligibility rule. Users forgive a clumsy sentence. They do not forgive being charged the wrong amount.

Most AI failures in production are not model failures. They are boundary failures: the model was allowed to be the source of a fact it could not know.

Separate facts from language

An LLM is good at language and bad at being a database. Treat it that way.

• Facts — prices, validity, activation codes, stock, eligibility — come from a structured source and are never generated.
• Language — explanation, comparison, summarisation, tone — can come from a model.
• Decisions — sorting, filtering, ranking — belong in deterministic code you can test.

When a user asks "what is the cheapest 5GB Jazz plan?", the cheapest plan is a query result. The sentence around it can be generated. If the model returns a plan that is not in the result set, the answer is wrong even if it sounds right.

The BundleWala constraint

BundleWala is a catalogue of 286 mobile plans across five Pakistani carriers. The assistant had to answer questions such as "cheapest 5GB for Jazz", "postpaid under 1000" and "only monthly" — in English, Urdu script and Roman Urdu.

We made it fully deterministic. The query is parsed into a structured scope (network, category, budget, data, minutes, SMS, validity, postpaid/free), the catalogue is scored, and the answer is assembled from templates. There is no model API, no per-message cost, and no way for the assistant to invent a price.

That was not a purist choice. It was the only design that satisfied three requirements at once:

1. Exact facts — every number comes from the catalogue.
2. Offline — the app works with no connection and no server round-trip.
3. Zero marginal cost — the free-tier constraint was explicit from day one.

The trade-off is honesty about scope. The assistant can answer questions the parser understands. For everything else, it says so and points to the guided finder.

Constrain the output

If you do use a model, do not ask for prose and hope. Give it a schema and validate the result.

type Answer = {
  intent: "recommend" | "compare" | "explain" | "unknown";
  packageIds: string[];
  summary: string;
  caveats: string[];
};

Then check every packageId against the retrieved set. If an id is not there, drop it and retry once with the valid ids. If the model still fails, return the deterministic answer. The model is a formatter with a bounded input, not an authority.

For extraction and classification, the same rule applies: a small schema, a validation step and a fallback.

Evaluate with the questions users actually ask

A demo is five prompts. Production is the long tail.

Before launch, collect 100–300 real questions from support, search logs or user interviews. Label the expected scope and the acceptable answer. Then measure:

• Scope accuracy — did the system understand the network, budget and category?
• Fact accuracy — are the plans and prices in the catalogue?
• Refusal quality — does it say "I don't know" clearly when it should?
• Latency — including the slowest 5% of requests.
• Cost per answer — tokens, retries and infrastructure.

The last two are where AI features quietly become unshippable. A 94% answer quality with a 6-second p95 and an unpredictable bill is not a product.

Design the failure state

The failure state is part of the feature. Decide what happens when:

• the provider times out;
• the answer fails validation;
• the catalogue is stale;
• the user is offline;
• the question is out of scope.

In BundleWala, an out-of-scope question gets a short explanation and a link to the guided finder. The guided finder is deterministic, so it still works. In a RAG product, the fallback might be "here are the three documents I found; I could not answer from them" with the sources visible.

Silence is the worst fallback. A confident wrong answer is the second worst.

When an LLM is the right tool

Use a model when the input is unstructured and the output is language or a classification that can be checked. Examples:

• turn a messy brief into a structured job post;
• summarise a long support thread;
• extract fields from a document;
• draft a first response that a human edits;
• classify intent before a deterministic flow.

Do not use a model as the only source of a fact, a price, a permission or a legal statement. Ground it, validate it, and keep a path that does not depend on it.

A short checklist

• [ ] Facts come from a structured source, not the prompt.
• [ ] The output has a schema and a validator.
• [ ] Every referenced entity is checked against the retrieved set.
• [ ] There is a deterministic fallback.
• [ ] There is an evaluation set with real questions.
• [ ] Cost and latency are measured per answer.
• [ ] The UI makes uncertainty visible.
• [ ] The feature degrades gracefully when the provider is down.

AI is useful in production when it is treated like any other dependency: bounded, measured and replaceable. The magic is in the product decision, not the prompt.