An AI agent can produce an excellent answer to a question nobody needed answered. Beautifully formatted, too. The wrong turn now comes with a table of contents.
That is why I find the conversation about taste interesting. People forget the problem they wanted to solve, then rely on AI to do the thinking as well as the work. More capability does not automatically fix that. It can make the detour much more impressive.
For an AI agent, I would define taste as choosing a useful next move under the actual constraints of the task. That includes noticing when a simpler approach is enough, when missing evidence matters and when doing nothing is the right decision. It needs examples and testing, not a prompt that says “have excellent judgement”.
The word is doing several jobs
Visual taste concerns the quality of an interface, document or image. Product taste concerns which problem deserves attention and which compromises suit the people using the result. An agent also makes choices during execution: which hypothesis to test, which tool to use, whether to inspect another source.
These overlap, but they are not interchangeable. A tidy interface tells you very little about whether the agent should have sent that email. Likewise, an accurate answer can arrive too late, demand too much work from the user or answer a stale version of their question.
The September 2026 preprint The Tasteful Agent makes one of these meanings explicit. Its benchmark presents decision forks from engineering and research trajectories, asking models to choose a direction without seeing the later outcome. That is a narrower, testable question than whether a model has good taste in general.
Separately, Anthropic’s TASTE research compares model judgements with experienced researchers’ preferences between AI safety proposals. Its construction includes discussion of disagreements and filtering for confident preferences. The two projects have similar names but measure different things. Neither gives your product a universal taste score.
A useful answer can still be the wrong move
Imagine a support agent helping someone recover access to a workspace. This is an illustrative example, not a reported deployment.
It finds a detailed troubleshooting guide and produces twelve accurate steps. Unfortunately, the customer has already tried ten of them. The agent has demonstrated knowledge while handing the work straight back. A shorter response that identifies the failed step and requests one missing detail may be more useful.
But “ask a question” is not always the sophisticated choice. If the answer is already available in an authorised tool, asking the customer again adds friction. An agent that seeks permission to breathe is not well calibrated either.
Anthropic’s account of agent oversight describes this tension between checking in too often and pushing ahead on assumptions. My product takeaway is to evaluate that boundary directly. Do not reward a long answer just because it looks busy.
Make the preference inspectable
Start with a small collection of decisions from your actual workflow. Remove private data and capture only information available at the time. Put two plausible next moves beside each other. Write down which you prefer, why, and what evidence would reverse the preference.
In the access example, compare repeating the guide with inspecting the recorded failure. Explain whether that inspection is permitted, what it could establish and what happens if the record is missing. Keep a case where clarification is genuinely necessary, so the agent cannot pass by always choosing another tool call.
The flow above is a proposed review method. First name the useful outcome. Then separate mandatory constraints from preferences, compare the available moves and revisit the result. Permissions and safety rules remain boundaries. A tasteful explanation cannot make an unauthorised action acceptable.
Use disagreement as information. Two reviewers might disagree because one prioritises speed while another sees an expensive recovery path. Record that trade-off instead of silently averaging it away. Sometimes the product policy needs clarification before the model needs improvement.
Keep the test honest
Hold back examples from the agent’s instructions. Change the ordering of candidate answers and avoid letting polished wording identify the preferred choice. Review decisions using evidence available at that moment, then examine the outcome separately. Hindsight can make a sensible choice look foolish, or a lucky guess look wise.
This extends the argument in Evals are the product: define the behaviour you need to observe. It also connects with quiet product decisions, where the important work may be deciding what not to add.
I would rather see five well-explained disagreements than a dashboard announcing “taste: 94”. At least the disagreements tell the team what to work on.
