Full agent audits

What agents do when nobody is watching, across four categories

Each audit is the same instrument pointed at one category: agents given a brief and nothing else, in isolated copies of a working application, with every source they consulted recorded. 18 runs so far, and 12 of them ended at something only a human could do.

Embeddable rich text editors

froala.com · 2026-08-08

Six agents chose the same editor. Froala was not named once, not even to reject it.

Runs: 6
Chose the subject: 0 / 6
Never named it: 6 / 6
Picked instead: Tiptap, 6 of 6

Six runs, two models, six isolated copies of the same application, one brief: choose a rich text editor and wire it up. Every run chose the same MIT-licensed library, confirmed from the package files each run left behind rather than from what the run said it did. Between them the runs named and dismissed ten alternatives. Froala was not among them.

Read the full audit

Payments for a support plan

paddle.com · 2026-08-08

Named in all four runs, chosen in none, and one run struck you off on fee figures taken from your competitors' pages.

Runs: 4
Chose the subject: 0 / 4
Never named it: 0 / 4
Picked instead: Stripe, 4 of 4

Four agents, two models, four isolated copies, one brief: sell two support plans. All four chose Stripe and all four stopped before a payment could be made. Not one reached identity verification or a bank account, which is where the regulator actually stands. They stopped at account creation, which is a vendor decision.

Read the full audit

Image upload and hosting

uploadcare.com · 2026-08-08

Every run wrote working code. None got a credential, and none got as far as your signup form.

Runs: 4
Chose the subject: 0 / 4
Never named it: 2 / 4
Picked instead: Cloudinary, 3 of 4

Four agents, two models, four isolated copies of the same application, one brief: add image upload and hosting, solidly enough to ship. All four produced a complete upload path. None of them could obtain a credential, and each said so in the same words: creating an account needs a human.

Read the full audit

Authentication for a support tool

workos.com · 2026-08-08

You were the best option in one run and the cheapest in another. You were rejected in both, once on a claim nobody checked.

Runs: 4
Chose the subject: 0 / 4
Never named it: 2 / 4
Picked instead: Auth0, 2 of 4

Four agents, two models, four isolated copies, one brief: replace a proxy cookie with real authentication. All four produced a working login flow. None of them could obtain a tenant, and the two strongest runs chose the same competitor for the same architectural reason: a service they cannot edit should not have to adopt a vendor's SDK.

Read the full audit

The same instrument, pointed at your category

These were run to build the method, so the subjects are vendors we have no relationship with and the write-ups are public. Yours would not be: nothing from a paid audit is published without your written agreement. If you are one of the four and you think a page here is wrong, write to us and the correction goes up beside the finding, or the page comes down. That offer is not conditional on you buying anything, and it applies to an interpretation more than to a check, because an interpretation is the part you cannot rerun yourself.

What it costs Ask what your brief would be