Full agent audits

Agent integration studies across four categories

Agents received a brief in isolated copies of a working application. Each study records the sources they consulted. Across 18 runs, 12 reported a step that needed a human.

Embeddable rich text editors

froala.com · 2026-08-08

Six agents chose the same editor. Froala was not named once, not even to reject it.

Runs: 6
Chose the subject: 0 / 6
Never named it: 6 / 6
Picked instead: Tiptap, 6 of 6

Six runs, two models, six isolated copies of the same application, one brief: choose a rich text editor and wire it up. Every run chose the same MIT-licensed library, confirmed from the package files each run left behind rather than from what the run said it did. Between them the runs named and dismissed ten alternatives. Froala was not among them.

Read the full audit

Payments for a support plan

paddle.com · 2026-08-08

Four runs mentioned Paddle. All four chose Stripe.

Runs: 4
Chose the subject: 0 / 4
Never named it: 0 / 4
Picked instead: Stripe, 4 of 4

Four runs, two models, one task: add a checkout for two support plans. All four chose Stripe; none completed a payment. Their reports describe an account or credential handoff, not a completed integration.

Read the full audit

Image upload and hosting

uploadcare.com · 2026-08-08

Every run wrote working code. None got a credential, and none got as far as your signup form.

Runs: 4
Chose the subject: 0 / 4
Never named it: 2 / 4
Picked instead: Cloudinary, 3 of 4

Four agents, two models, four isolated copies of the same application, one brief: add image upload and hosting, solidly enough to ship. All four produced a complete upload path. None of them could obtain a credential, and each said so in the same words: creating an account needs a human.

Read the full audit

Authentication for a support tool

workos.com · 2026-08-08

You were the best option in one run and the cheapest in another. You were rejected in both, once on a claim nobody checked.

Runs: 4
Chose the subject: 0 / 4
Never named it: 2 / 4
Picked instead: Auth0, 2 of 4

Four agents, two models, four isolated copies, one brief: replace a proxy cookie with real authentication. All four produced a working login flow. None of them could obtain a tenant, and the two strongest runs chose the same competitor for the same architectural reason: a service they cannot edit should not have to adopt a vendor's SDK.

Read the full audit

Commission an integration audit

These public studies cover vendors I have no relationship with. Nothing from a paid audit is published without your written agreement.

If a finding about your product is wrong, write to me. I publish the correction beside the finding or take the page down. You do not need to buy an audit to request a correction.

What it costs Ask what your brief would be