Building AI products

What exactly is a problem AI can actually solve? Let's try to understand. In a simple way. It is tempting to sprinkle AI onto everything.

But a language model is one specific tool. Most problems are not shaped for it. So before you build, sort the problem into two piles. A good fit for AI.

Or a job another tool does better. The first sign of a good fit is a fuzzy, language shaped task. Summarize this. Sort these by topic.

Draft a reply. There is no single formula. The second sign is that roughly right is already useful. A decent first draft beats a blank page.

You fix it from there. Now the bad fits. If a problem has one exact answer that must be correct every time, a model is the wrong tool. It guesses.

For a bank balance, use a database. The second bad fit is when a mistake is costly and nobody would catch it. The model is sometimes confidently wrong. If that slips through, the fuzziness becomes a danger.

So ask, is there a safety net? A person who reviews it, or a cheap check that flags bad output. A net turns a risky task into a workable one. One last test.

Can you show, not tell? If you can write a few examples of good output, the task fits. If you cannot describe good, the model cannot either. So the gist.

A fuzzy task, where roughly right helps, and a wrong answer can be caught. Match all three, and you have a real fit. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is the magic wand test? Let's try to understand. In a simple way. When you dream up an AI feature, two questions get tangled together.

Is it valuable? And can we actually build it? People argue the second and forget the first. So wave a magic wand.

Pretend the hard AI part already works flawlessly, like real magic. The model never makes a mistake. Now ask just one thing. With that perfect version in your hands, would people actually want it?

Does it solve a real pain? If the answer is no, even with a flawless magic version, then stop. No amount of engineering can save a feature nobody wants. But if the answer is yes, you have found something worth the hard work.

The value is real, so now the effort is justified. Why does this work? It splits two different risks. Desirability, do they want it.

And feasibility, can we build it. The wand removes feasibility on purpose. One trap. Do not confuse a cool demo with a solved problem.

The wand makes everything look magical, so judge the outcome, not the sparkle. The best part, this test is free. No code, no month of building. Just a thought you run before you spend anything.

It blocks the opposite mistake. Starting from AI can do this, then forcing a feature around it. That is tech first. The wand keeps you user first.

So, the magic wand test. Pretend the AI is perfect, then ask if anyone still wants it. Value first, building second. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is good enough? Let's try to understand. In a simple way. Here is a hard truth about AI products.

The model will make mistakes. Always. Waiting for a perfect model means waiting forever. So the real question is not, is it perfect.

It is, how good must it be, before real users see it. That depends on the stakes. A wrong movie pick? The user shrugs.

Wrong medical advice? Someone can get hurt. So a playlist feature might be fine at eight right answers in ten. A medical feature might need ninety nine in a hundred, plus a human double checking.

Now the mechanism. Before you build anything, write the bar down as a number. Something like, nine out of ten answers get rated useful by users. That number becomes your target.

Every version you build gets measured against it. Pass, and you ship. Fail, and you keep working. No endless debates.

Also compare against today. If humans doing the job get it right seven times in ten, an AI hitting eight is already a real win. Skip this step, and two bad things can happen. You ship a sloppy product and burn trust.

Or you chase perfect and ship nothing. One last dial. Can the user spot a mistake and fix it in seconds? Then the bar can drop.

A draft is forgiving. An autopilot is not. So, good enough is a number you pick before building. Set by the cost of one mistake, and the bar humans already reach.

Quick check now. One question is coming up. Let's see if it clicked.

How does picking a model work? Let's try to understand. In a simple way. When you build with AI, you do not get one model.

You get a menu. Small ones, medium ones, and giant ones. The giant model is the smartest. It can handle messy, difficult reasoning.

But every answer is slow, and every answer costs real money. The small model is the opposite. Answers come back fast, and they cost almost nothing. But on hard problems, it starts to stumble.

So here is the trap. Grab the biggest model for everything, and your product feels slow and burns cash. Grab the smallest, and it gives wrong answers. The pick comes down to three questions.

How hard is this task? How fast must the answer come back? And what can one answer cost? Here is the recipe most teams use.

Start with the smartest model. First, prove the feature works at all. Then step down. Run the same tests on smaller models, one size at a time.

Watch the quality score as you go. At some point the score drops below your bar. Step back up one size. The smallest model that still clears the bar.

That is your pick. One more trick. A product is many jobs. Route the easy ones, like sorting emails, to a small model.

Save the giant for the hard ones. So, picking a model is a trade. Speed and cost on one side, smarts on the other. Start big, prove it, then shrink.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a prompt prototype? Let's try to understand. In a simple way. You have an idea for an AI feature.

The old instinct says, start building. Code, screens, a database. Weeks of work before you learn anything. Here is the faster way.

Open a plain chat window. Type the instructions for your feature, as if the app already existed. Now paste in a real example. A real customer email.

A real support ticket. Not a made up toy case. Real inputs show real behavior. Then judge the output yourself.

Would you show this answer to a user? You just tested your product idea. In minutes. With zero code.

That is the mechanism. The prompt plus real examples behaves like your future app. Same model, same instructions, same inputs. The answers you see are a preview.

When the output is bad, do not quit. Reword the instructions. Add an example of a perfect answer. Each retry costs seconds, not weeks.

After twenty tries, you have a verdict. If the model fails here, with your best hand holding, no app built around it will save it. And if it works? Save that winning prompt.

It becomes the first draft of your real system prompt. The prototype grows into the product. So, a prompt prototype is your idea tested in a chat window, on real examples, before any code. The cheapest experiment in AI.

Quick check now. One question is coming up. Let's see if it clicked.

How does custom knowledge work? Let's try to understand. In a simple way. A model learns from public text, and then its knowledge is frozen.

So it has never seen your return policy, your product docs, or your prices. Ask it about your refund rules anyway, and it will not stay silent. It answers in a confident voice, and simply makes the details up. The fix is surprisingly direct.

Paste your documents into the prompt, right next to the question. Now the model reads your text before it answers. This is an open book exam. The model does not memorize your company.

It just reads the pages you hand it, every single time. But there is a catch. The prompt has a size limit, and your knowledge base is huge. Thousands of pages will never fit in one request.

So we add a search step. When a question comes in, the system first finds the few passages that match it. Maybe three paragraphs out of thousands. Those passages get pasted into the prompt, with the question underneath.

The model answers from what it just read, not from memory. And here is the best part. Change a document, and the very next answer uses the new version. No retraining.

Ever. So, custom knowledge is not taught, it is handed over. Search your documents, paste the best bits, and let the model read. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is structured output? Let's try to understand. In a simple way. Ask a model to pull a name and a price out of an email.

It happily replies with a friendly paragraph. Lovely for a human to read. But here is the catch. In a real product, no person reads that answer.

Your code does. And code needs exact fields, in exact places. Parsing free text is a nightmare. One day the model adds an extra sentence, or moves a comma, and your parser crashes.

The feature dies. The fix is to make the model speak JSON. A tiny format of named keys and values. So every answer arrives with each field in its own labeled slot.

Step one lives in the prompt. You show the exact shape you want. Reply only with JSON, with these two keys. Nothing else.

Step two is stronger. Most APIs have a structured output mode. While generating, the model may only pick tokens that keep the JSON valid. Everything else is simply blocked.

Still, always validate. Your code parses the reply and checks every key and every type. Because the shape can be perfect while a value inside is wrong. And when a check fails?

Send the error back and ask the model to fix it. One automatic retry catches most bad replies. So, structured output means the model answers in JSON. A promised shape, labeled slots, and code that can finally build on the answer.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a golden set? Let's try to understand. In a simple way. You tweak your prompt, try it on one example, and it looks better.

So you ship it. But did the change break the nine other cases you did not try? You have no idea. A golden set fixes this.

It is a small, fixed collection of real inputs, each paired with the answer you would be proud to ship. So one entry might be a real customer email, plus the perfect summary of it. Written once, by a human, and saved. Twenty to fifty entries is plenty.

But choose them to cover the range. Easy typical cases, tricky edge cases, and the ugly ones that broke things before. Now the mechanism. Every time you change the prompt, or swap the model, you run the whole set through it, and compare each output to its saved answer.

That comparison becomes a score. Say, forty one out of fifty. Change something, run again, and the score tells you if things got better or worse. And this only works because the set stays fixed.

Same exam every time, so any change in the score comes from your change, not from new questions. The set does grow, though. Slowly. Every time a real user hits a new failure, that case earns a place in the set, with its correct answer.

So, a golden set is your product's exam. Real inputs, saved best answers, taken by every change before it ships. Quick check now. One question is coming up.

Let's see if it clicked.

How does measuring factualness work? Let's try to understand. In a simple way. An AI answer always sounds sure of itself.

Right or wrong, same confident tone. So you cannot judge it by feel. You need a number. Here is the trick.

Do not grade the whole answer at once. Split it into small factual claims. One sentence, one checkable statement. Now take each claim and hold it against a trusted source.

Your own documents. A claim only counts as correct if the source actually backs it. So every claim gets a verdict. Supported, the source says it.

Contradicted, the source says otherwise. Or not found, the answer just made it up. Then count. Supported claims, divided by all claims.

Nine out of ten check out? That answer is ninety percent grounded. Factualness is now a number. Who does the checking?

People are the gold standard, but they are slow and expensive. So teams use a second AI as the judge. It reads one claim, one source passage, and votes. But can you trust the judge?

Spot check it. Take a small sample, have a human grade the same claims, and compare. If they mostly agree, the judge can run alone. And once it is a number, you can compare.

New prompt, new model, same questions. If the score drops, you know before your users do. So, measuring factualness. Split the answer into claims, check each one against a source, and count what survives.

Quick check now. One question is coming up. Let's see if it clicked.

How does thumbs up feedback work? Let's try to understand. In a simple way. Under every AI answer sit two tiny buttons.

Thumbs up, thumbs down. They look like decoration. They are actually your cheapest source of truth about quality. So what does one click actually do?

It saves the whole exchange. The user's question, the answer the AI gave, and the rating. All stored together. Now, a warning.

Most users never click anything. And unhappy users click far more often than happy ones. So the signal is thin, and it leans negative. That is fine.

The thumbs down pile is the goldmine. Each one is a real user, on a real question, showing you exactly where the product failed. Now the mechanism. A human reads through the down votes and groups them.

Maybe thirty of them are all refund questions. That is not noise. That is a pattern. A pattern points at a fix.

Maybe the prompt needs a refund rule. Maybe a missing document gets added. You fix the pattern, not one angry user. Then close the loop.

The failed cases become test cases, so you can prove the fix works. And watch the thumbs down rate on refunds drop. One myth to drop. A thumbs up does not retrain the model on the spot.

Nothing learns automatically. People read, people fix. The buttons just aim their attention. So, thumbs feedback is a loop.

Collect ratings on real answers. Mine the down votes for patterns. Fix the pattern, and check that the rate drops. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is latency? Let's try to understand. In a simple way. Latency is the wait.

The gap between the user hitting send, and the answer appearing on screen. Users feel that gap before they read a single word. And patience is short. Under one second feels instant.

A few seconds feels sluggish. Around ten seconds, people decide the feature is broken, and they leave. So a brilliant answer that takes twenty seconds can lose. Not because it is wrong.

Because the user gave up before it ever arrived. And an AI answer is usually a draft. The user reads it, fixes it, moves on. A long wait for something you will edit anyway feels terrible.

That changes the math. Fast answers mean more tries per minute. Ask, skim, rephrase, ask again. Three quick attempts often beat one slow, polished answer.

Now the mechanism. Engineers track two clocks. Time to the first word, and time to the full answer. Users judge the wait almost entirely by the first clock.

That is why answers stream. The model generates word by word anyway, so show each word the moment it exists. The total time does not change. The wait almost disappears.

The second lever is length. Every extra word takes time to generate. Ask for a short answer, and the model simply finishes sooner. Brevity is a speed feature.

So, faster can beat smarter, because attention is the scarce thing. Stream the first word early, keep answers short, and keep the user in the loop. Quick check now. One question is coming up.

Let's see if it clicked.

How does graceful failure work? Let's try to understand. In a simple way. You test your AI app with clean, sensible questions.

Then real users arrive. And real users never stay on the happy path. Someone sends an empty message. Someone pastes an entire contract.

Someone types in another language, or pure gibberish. None of this is an attack. It is just Tuesday. Here is the danger.

The model rarely says, this is broken. It answers anyway, with full confidence. Garbage in, confident garbage out. So the fix is not a smarter model.

The fix is designing the failure path on purpose, before launch. You decide what happens when things go wrong. First guard, at the door. Plain code checks the input before the model ever sees it.

Empty? Too long? Wrong format? Reject it early, with a clear message.

Second, give the model a way out. Tell it in the prompt: if the request is off topic or unclear, say so and ask again. An honest, I am not sure, beats a made up answer. Third, plan for the model itself failing.

Calls time out. Services go down. Your app needs a fallback: a safe default answer, or a simple, please try again. And log every weird input you rejected.

Each one shows you how real users actually behave. Every break is a free lesson. So, users will break it, guaranteed. Guard the door, give the model an out, and keep a fallback ready.

Broken input, graceful answer. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the cost of one answer? Let's try to understand. In a simple way. Every answer your AI gives has a price tag.

Tiny, but real. If you cannot calculate it, you cannot know if the product makes money. The provider bills you per token. Tokens in, tokens out.

So one answer's cost is just two token counts, times two prices. Here is the catch. The input is not just the user's question. Your instructions, pasted documents, chat history.

All of it rides along on every call. And the two prices are not equal. Output tokens usually cost a few times more than input tokens. Long, chatty answers are the expensive part.

So the formula is simple. Input tokens times the input price. Plus output tokens times the output price. That is the cost of one answer.

Say an answer reads two thousand tokens and writes three hundred. At typical prices, a fraction of a cent. Sounds like nothing. Now multiply.

Twenty answers per user per day. Ten thousand users. A month of that. Suddenly your fraction of a cent is thousands of dollars.

This is why teams trim the hidden input. A shorter system prompt. Fewer pasted passages. A capped history.

Same answer, cheaper call. Then compare cost to value. One cent to save a support ticket? Great deal.

If an answer costs more than it earns, you have a problem. So, count tokens in, count tokens out, multiply by their prices, and then multiply by scale. That number decides if your product survives. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is model drift? Let's try to understand. In a simple way. Launch day.

Your AI feature passes every test. Answers are sharp, users are happy. You move on. Story over?

Not quite. Three months later, complaints trickle in. Wrong answers, odd replies. Nothing crashed.

No error fired. Quality just slid, quietly, while nobody was watching. First cause. The model did not change.

The world did. New products, new slang, new kinds of questions your prompt has never seen. That gap has a name. Drift.

Your setup was tuned for yesterday's inputs. Today's inputs look different, so the same prompt misses more often. Second cause. The model itself changed.

Providers retire old versions and update models behind the same name. Same code on your side, a different brain on theirs. Defense one. Pin an exact model version in your code, not just a family name.

Then upgrades happen when you choose, after you test. Defense two. Take the test set you already built, and re-run it on a schedule. Every week, same questions, and you write down the score.

Now degradation has a number. Forty five out of fifty. Then forty three. Then thirty eight.

A falling line is your alarm, long before users leave. When the alarm fires, read the new failures. Usually the fix is fresh examples, updated documents, or a retested newer model. Not panic.

So, good models do not stay good by default. The world drifts, models change. Measure on a schedule, and catch the slide early. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the human-in-the-loop check? Let's try to understand. In a simple way. Your AI product writes an answer.

But before that answer reaches a real customer, it stops at one last gate. A person looks at it first. Why? Because some mistakes are too expensive to ship.

A wrong refund. A bad medical hint. An email to the wrong client. And the model cannot tell which mistake it just made.

So the mechanism is a review queue. Drafts pile up in a list, and a reviewer works through them with three buttons. Approve, edit, or reject. Approve sends the draft out untouched.

Edit fixes it, and ships the fixed version. Reject throws it away. Nothing leaves the building without a click. But you do not review everything.

You route by risk. Low stakes answers go straight out. High stakes ones, refunds, legal, health, always wait in the queue. The gate can loosen.

When the AI keeps passing review, you switch to sampling. Check one draft in ten, instead of every single one. And every edit gets logged. That log shows exactly where the AI still fails, on real cases.

It tells you when the gate is safe to loosen. The catch is speed. A human gate adds waiting, and reviewers cost money. That is the trade.

You pay minutes to avoid disasters. So, the human-in-the-loop check is a gate before the send button. Approve, edit, or reject. Full review at first.

Sampling once trust is earned. Quick check now. One question is coming up. Let's see if it clicked.