Every inspection has things you couldn’t fully inspect. The attic hatch was screwed shut. The crawlspace had standing water. The A/C wasn’t run because it was 48 degrees outside. Your client needs to know about each of those, and they’re not defects. They’re limitations.
In AEYES you can say it the way you’d say it to a colleague: "Couldn’t enter the attic, hatch blocked by storage." AEYES puts it on the attic as a limitation, not as a finding. To do that, it has to answer two questions about what you said:
- Is this a limitation at all, or a defect?
- How much of the system did it affect? Was the system not inspected, or was it partially inspected?
We had rules for both questions. What we didn’t have was any idea how often they were right.
We measured it for the first time
So we checked. We took about three months of dictated notes from real inspections, pulled out a mix of likely limitations, borderline cases and ordinary defect notes, and labelled 166 of them by hand. The labels were written without seeing what AEYES had decided, so they couldn’t steer us.
The first question held up well. When AEYES said "limitation", it was right almost 9 times out of 10, and it caught most of the real ones too.
The second question did not. On "not inspected or partial", AEYES and our labels agreed only 58% of the time. That’s not much better than a coin toss.
We expected to find a weak spot in the software. We found two questions nobody had ever answered.
The two rules nobody had decided
When we went through the disagreements, most of them weren’t the software misreading a sentence. They came down to two rules.
Seen, but not operated. "Water heater visually inspected, not operated, gas off." Is that not inspected, or partial? The rules said partial: you still looked at it. Our labels mostly said not inspected. That one disagreement caused almost two thirds of the misses. We settled it: if you looked at it but couldn’t run it, it’s partial. You did inspect it, and the report should say what you saw.
Outside the inspection. Irrigation, radon, a standby generator. When an inspector said one of those wasn’t inspected, AEYES quietly dropped it, on the theory that it was never part of the job. But inspectors say it out loud precisely because they want it in the report. Now it becomes a limitation like any other.
With those two rules settled, the agreement on "not inspected or partial" went from 58% to 84%.
What’s still hard
The rest is mostly language, and there is a lot of it. We found "locked gas meter" phrased four different ways, and phrases like "sealed shut", "blocked by plants" and "limited view of" that the rules had never heard.
The one genuinely hard case is the difference between "I couldn’t test it" and "I tested it and it failed". "Disposal did not operate" sounds like a limitation, but it’s a defect: you tried it, and it didn’t work. That’s the one place where a smarter model might genuinely help, and it’s what we’re looking at next. First, Travis goes through every case where we and the software disagreed, because an inspector’s call beats ours.
Why this matters more than it sounds
A missing limitation is how a client ends up surprised by something you were never able to see. Getting it into the report, with the right extent, is part of protecting the inspector. So before we let a smarter model near it, we wanted to know what "right" means. Most of the answer turned out to be two decisions, not a model.
