#Evaluation
Posts tagged Evaluation.
Why temperature changes the answer, not the knowledge
Turning temperature down does not make a model more accurate. It makes it more repeatable — and confusing the two is how...
Why models hallucinate, and what actually reduces it
Hallucination is not a glitch that a better model will one day remove. It is what generation does when it has nothing to...
Choosing between a large model and a small one
Most production traffic does not need the largest model available. Routing by task rather than defaulting to the top of...
Why agents fail on long tasks
An agent that handles five steps beautifully can fall apart at twenty. The reason is rarely reasoning — it is that the c...
When a workflow beats an agent
If you already know the steps, do not ask a model to rediscover them on every request. A fixed pipeline with model calls...
Reviewing AI-generated UI: a checklist
Generated interfaces fail in a predictable set of places. Knowing which ones turns review from a vague unease into a lis...
Streaming responses without breaking your UI
Streaming makes a slow response feel fast. It also means rendering text that is syntactically incomplete, and handling a...
How to evaluate an AI feature before you ship it
You cannot unit test "is this a good answer", but you can build a set of real cases with known-good outputs. An afternoo...
The cost model of an AI feature
Per-token pricing looks trivial until you multiply by retries, conversation history and the context you resend on every...
Writing prompts your team can maintain
Prompts are code with none of the tooling. Treat them like code anyway — version them, review them, test them, and expla...
What to tell users when the model is wrong
Your AI feature will be confidently wrong in front of a user. What the interface does in that moment decides whether the...
AI adoption without a rewrite
The best first AI feature is small, measurable and easy to switch off. Start where a wrong answer is cheap and the value...