Your AI follows the refund policy you actually wrote

A new study ran 3,456 refund cases through eight AI models. They paid fair claims, until one sentence told them not to. Where your policy lives matters.

Two weeks ago Karen Spinner published a careful little experiment. Spinner wrote 36 refund and claims disputes, most of them borderline, a few clearly covered, and ran them through eight current AI models under different instructions. 3,456 decisions in all.

The headline most people will take from it is the scary one. We think the more useful finding is the first one.

With neutral instructions, the models paid deserving customers 93% to 100% of what they asked for, on average. The clearly covered claims stayed protected across almost every condition. Read plainly, that’s good news for anyone thinking about AI in support. Given the facts and a fair brief, current models make about the call a decent support lead would make.

Then Spinner changed one or two sentences.

A sentence is a policy

When the instructions told the models to keep payouts down, full payment on sound claims at the first offer fell from 92% to 50%. Deserving customers were refused outright 40 times in 359 decisions. The most sensitive model went from paying 100% of sound requests to 27%. In the other direction, a single “be generous” line pushed payments on weak claims from 35% to 61%.

Nobody rewrote the refund policy. Nobody changed the 30-day window or the damaged-item rule. Somebody added a sentence about tone and priorities, and the money moved.

That’s the lesson for an owner, and it isn’t really about AI being unreliable. The model did exactly what it was told. The trouble is that “be mindful of costs” and “returns accepted within 30 days” were sitting in the same paragraph, written in the same voice, and the model treated both as policy. Because both were.

Anyone who has managed a support team has seen the human version. A manager says “we’re watching refunds this quarter” in a Monday meeting, and by Friday the team is turning down claims the written policy covers. The difference with an AI is speed and scale. A human team drifts over weeks. A prompt change ships to every conversation at once.

Keep the money decision out of the mood

So here’s the argument. Whatever an AI says about refunds can be shaped by instructions, and that’s fine for wording. Whether a refund actually happens shouldn’t be.

In practice that splits into three parts.

Write the policy as rules, not attitude. “Refunds within 30 days of delivery, unused items, original payment method” is something code can check. “Try to retain revenue where reasonable” is not, and it will quietly bend every borderline case it touches. If you want to be stricter, change the rule, out loud, where you can see it.

Let the AI propose and let code decide. The model is good at the part a rules engine is bad at: reading a messy email, finding the order, understanding that “it showed up cracked” means damaged in transit. It should hand that over as a proposal (this order, this amount, this reason). Then plain code checks the proposal against what the business has actually allowed. If it fits, it goes. If it doesn’t, a person sees it.

Watch it decide before you let it act. Spinner’s study is basically a test bench, and every business should have one. Run the agent on your own past disputes, the ones where you know what the right answer was, and look at where it lands before any customer is involved. If a wording change moves the outcomes, you find out on old tickets instead of real ones.

What this looks like for us

This is close to how we built IMCeleste. Celeste, our front desk agent, practices on a business’s own history before answering anyone, and the owner reads those practice runs. Celeste goes live one job at a time, so “where’s my order” and booking a slot can be fully automatic while refunds are still waiting on a person’s approval. And anything that changes money or an account is a proposal that our code checks against the permissions the owner has granted. A clever sentence in a prompt can change how the agent says no. It can’t change whether the refund goes through.

The study’s last point is worth keeping too. Spinner’s advice to customers was to restate the facts neutrally and ask for a human in writing when a refusal feels off. That’s good advice, and it also tells you what a well-run front desk should offer without being asked: a fair reading of the facts, and a real person within reach when the decision is a judgment call.

If you’re weighing an AI for your front desk and refunds are the part that worries you, we’re happy to walk through it.