What six months of AI use is doing to your team's judgement

Written by Sonya Cullington from Behavioural AI 

Most businesses have been using AI daily for six months to a year now. The tools are embedded. Outputs are faster. Deadlines that used to slip are being hit. If you look at the numbers most companies track, adoption is a success story.

Something else is happening underneath, and it shows up in the work.

The person who used to catch the client's name being misspelled is missing it. The manager who used to challenge a weak recommendation is signing it off. The senior specialist who used to spot a mistake in a report from across the room is now asking the junior, "does this look right to you?"

Confidence in the AI-assisted work is high. The judgement that used to sit behind the confidence is quieter than it was.

This is what I call the Judgement Gap. It's the distance between how confident someone is when they're using AI and how competent they actually are without it, or when they're checking it. It widens slowly, which is why most training doesn't catch it. AI literacy courses teach people how to prompt, how to spot hallucinations, how to work with the tools. They don't measure what six or twelve months of use has done to the person's own judgement, and they don't touch what needs to change in the team to keep that judgement intact.

The evidence for this is now real. In one study, a full 20-hour AI literacy course didn't shift automation bias in the people who took it. Their accuracy on the underlying task fell from 85 per cent to 73 per cent. In another, specialists using AI to support diagnostic work saw detection rates drop from 28.4 per cent to 22.4 per cent after sustained exposure. Confidence held. Competence, when tested unaided, fell. The people most fluent with the tools weren't always the ones producing the most accurate work.

For an SME, this isn't a governance problem. It's a commercial one. It's the mistake that goes out to a client. It's the assumption in the report that no one caught. It's the senior member of the team who's quietly stopped being the check on the junior's work, because they're using the same tool the junior is.

Training on the tools doesn't close the Judgement Gap. Measuring it does, and then changing how the team works around AI. That's what a diagnostic is for. At individual level, it gives you a calibration profile: a clear picture of where your confidence and your competence have moved apart. At team level, it gives you a risk map showing where in the business judgements are getting rubber-stamped, and which of those decisions really shouldn't be.

I've been running this diagnostic with NHS trusts and public-sector clients under the Judgement Integrity Framework. From this month it's available to Greater Birmingham SMEs through the Chamber. If you're reading this and wondering whether any of the pattern I've described is showing up in your business, it probably is. Finding out where is a half-day exercise. What you do about it is the work.

Sonya Cullington is a cyberpsychologist and behavioural AI governance advisor. She sits on the Executive Committee of the UK Digital Health Care and chairs its work on AI in the workforce. A founding-cohort offer for Chamber members is currently live on the Member Marketplace.