Methodology2026-06-14 · 6 min read

How we test AI SOP creators

The exact 12-step billing workflow, the nine-criterion rubric, and the weights we apply to every tool on this site.

In short
Every tool runs the same 12-step billing workflow, graded on nine weighted criteria. The AI first draft is scored untouched. Then we grade the editing pass. Then the publish flow. The weights are published and every raw score is on the tool's review page.

The workflow

We record the same customer-onboarding invoice run in every tool. Twelve clicks, one form field, two navigations across an admin area with realistic PII on screen. Same browser, same window size, same starting state. The workflow is chosen because it produces enough surface area to test capture, redaction, and editing without turning into a screencast.

What we grade

  1. 01AI generation quality on the untouched first draft, 20% weight.
  2. 02Capture method and speed, 15%.
  3. 03Editing and formatting, 12%.
  4. 04Security and redaction, 12%.
  5. 05Sharing and export, 10%.
  6. 06Integrations, 10%.
  7. 07Team collaboration, 8%.
  8. 08Pricing and value at 10 seats, 8%.
  9. 09Ease of use, measured time to first published SOP, 5%.

How the score is computed

Each criterion is scored 1 to 10 against a written scale. The final score is the weighted average, rounded to two decimals. On segment pages the same scores are re-weighted, never re-scored, and the weights are always normalized to 100% so numbers stay comparable across pages.

What we do not do

  • Take vendor talking points at face value. Every score comes from our own run.
  • Hide the tools where our client loses. Where Haiku is not the best pick, we say so on the page that would rank it.
  • Update quietly. Every rescore lands with a dated change log on the methodology page.

For the full weight table and the raw per-criterion scores, see the methodology page.

More field notes