Published on October 14, 2026 · 5 min read
The review found 197 things. I managed to verify seventeen
On 9 August 2026 I ran a review of my own project starting from one simple question: what does a customer see who I will never meet — on the site, in an email, on a document sent from the system. The searching was done by an AI assistant — it read the files and listed suspicious spots; checking whether a finding was real was my job. The session hit its limit before it finished.
It left behind 197 findings and 17 verifications.
What exactly is on this list
Broken down by severity: 26 blockers, 91 serious, 80 minor. Broken down by kind: 85 inconsistencies (the same thing worded differently in two places), 42 false statements, 31 unfulfilled promises, 24 template placeholders nobody filled in, 15 tone remarks.
The second part of the review — decisions the system makes without showing me what it's doing — added another 20 items and zero verifications.
A finding is not a defect. It's a sentence: there is probably a problem here. Only verification turns it into a fact, or throws it out. I've checked 17 of 197. The rest are hypotheses with nice formatting.
Why this isn't a minor detail
Two weeks earlier I'd run a security review in which one of the methods reported 21 holes, of which zero were real. So I already knew — from measurement, not a hunch — that the share of false alarms in an unverified list can run as high as one hundred percent.
I don't know how many are false here. That's the whole content of the problem: a list of 197 items of unknown quality misleads in both directions at once. Read seriously, it tells you to fix things that might already work. Read with reservation, it buries the twenty-six I called blockers — and a blocker is, by definition, something a customer stops at and doesn't come back from.
On top of that, the seventeen verified ones aren't a random seventeen. They're the ones where verification cost one click: a link to open, a sentence to compare against another sentence. Everything that required reconstructing the customer's situation — walking the entire path from a form to the document that arrives from it — went untouched. So what's verified is what's cheapest, not what matters most, and the sample says nothing about the rest of the list.
Efficient searching without efficient verifying doesn't produce value. It produces debt that looks like a report.
Where this ratio comes from
Not from the limit. The limit only showed the moment where it became visible.
The ratio was set at the start, when I agreed that the search would cover the entire surface visible to a customer — the site, the calculators, every outgoing email, every printed document, three language versions — and left verification for later, without counting how much of it there would be. Searching costs one glance at a file. Verification costs opening the system, reconstructing the customer's situation, and reaching a decision. The gap is roughly tenfold, and it could have been calculated in advance.
What I did not do
I did not start fixing from the top of the list. Tempting, and the worst possible move: fixing a finding nobody confirmed means making changes for no reason — to code that might have been fine. At 197 items, that's weeks of work with an unknown share of harm.
I did not delete the list, even though it isn't reliable. Several hours of reading are recorded in it, together with a quote, a file, and a line number for every item. Throwing it away would mean reading the same thing from zero next time.
I did not write anywhere "found 197 problems." Not in the session summary, not in a note to myself, not on this page. That sentence is false and sounds like a result. The document now opens with the statement that this is the output of a search phase and is unverified — as the first line, not a footnote at the bottom.
I did not start another round of searching. The instinct said: since it got cut off, let's finish the review. Finishing the review would only have grown the counter on the wrong side.
Three things I take from this
Verification throughput is the limit on the whole review, not search throughput. It's worth cutting the scope to what can actually be verified — the rest won't turn into a decision anyway.
A list without verification has to carry that fact within itself. Not in the author's memory, not in the email it was sent with — in the file's first sentence, because in six months someone who wasn't at that session will read it. That someone is usually me.
Interrupted work has two different states, and you have to choose which one you leave behind. You can leave a result, or you can leave material. I left material and labeled it as material. Less impressive than a report, and the only honest thing that could be made of it.
The numbers — 197 findings, 17 verifications, the 26/91/80 breakdown, and 20 items from the second part — come from the raw file saved when the session was interrupted on 9 August 2026, and are counted from that file, not estimated. The figure of 21 false alarms comes from a separate measurement on 6 August 2026.
Patryk Piecyk
Warsaw · employed since 10.2026 · growing towards ERP/SAP and AI
For seven and a half years I worked at a German company, five of them running its office: orders, invoices, complaints, ERP. Since June 2026 I have been building my own tools for that same work — I am not a programmer by training; the code is written together with an AI assistant, while the design, the decisions and the testing are mine. These notes describe what broke in those systems and what came out of it.
Got a question about this piece?
Write to me →