How a CRO testing program runs: audit, hypothesize, test, scale
A CRO testing program runs as a loop of four steps. Audit the funnel to find where the money leaks, rank the fixes by expected revenue impact, A/B test the biggest one against the control, then roll the winner sitewide and feed what you learned into the next round. Step five is step one again. No redesigns on faith.
Key takeaways
- Every account runs the same 4 steps: audit, hypothesize, test, scale, and each step hands a concrete output to the next.
- The audit ranks leaks by dollars, so the first test goes to the biggest leak, not the page someone finds ugliest.
- A change ships only when it beats the control in an A/B test. Winners have numbers. Opinions do not ship.
- On one account at $41.19M over the trailing 365 days, up 162% year over year, the conversion rate barely moved: the growth came from a testing system across every traffic angle, offer and bundle.
- Step five is step one again: when a round ends, the next leak is already ranked.
What are the steps of a CRO testing program?
Four steps, run as a loop: audit, hypothesize, test, scale. Each step produces one output, and that output is the input for the next step.
- Audit. Analytics, session recordings and funnel data, to find where the money actually leaks. Output: a leak map, ranked by dollars.
- Hypothesize. Rank every fix by expected revenue impact. The biggest leak gets the first test, always. Output: a test backlog.
- Test. A/B test each change against the control. Winners have numbers. Opinions do not ship. Output: a verified winner.
- Scale. Roll the winner sitewide and feed what it taught into the next round. Output: a sitewide rollout.
Then step five is step one again. The next leak is already ranked, so the program never stops to plan a relaunch.
Why run CRO as a testing loop instead of a redesign?
Because a redesign bets the whole page on opinion, and a test bets one change on data. A loop of tests also keeps paying after the first win, because every round starts from what the last one proved.
A redesign changes many things at once, so even when the numbers move, nobody knows which change moved them. A test changes one thing against a control, so the answer is readable and the lesson carries into the next round.
The work almost always opens with a campaign, while the money is leaking from the page that campaign points to. I wrote about that pattern in an essay on LinkedIn:
After sixty audited accounts, the pattern is almost always the same: the creative is doing its job, and the page is quietly killing the click.
Hamid Chakir, It Was Never a Bad Agency. It Was a Bad Structure.
What happens in the audit step?
The audit finds where the money actually leaks, not where the page looks ugly. I work from analytics, session recordings and funnel data, and the output is a leak map ranked by dollars.
What I look at
- Funnel data: each step from ad click to landing page, product page, cart, checkout and order, and the step where the biggest share of visitors drops out.
- Analytics by page and traffic source: which pages paid traffic lands on, and how each of them converts.
- Session recordings: on the pages with the biggest drop, where visitors stall, hesitate at the buy box or scroll past the offer.
- The offer itself: the bundles, the subscription option and the choices the buy box asks a visitor to make.
Ranking by dollars is the point of the step. A clumsy section that few buyers ever reach can cost less than a clean-looking step that every paid visitor passes through.
How do you decide what to test first?
Rank every fix by expected revenue impact, and give the biggest leak the first test, always. The output is a test backlog in that order.
A useful hypothesis names three things: the leak, the change, and the number it should move. Expected impact is a rough product of how much traffic hits the leak, how much of it the fix could win back, and what an order is worth.
That estimate only sets the order of the backlog. The winner is decided by the test.
How do you run the A/B test?
Every change runs as an A/B test against the control. Winners have numbers. Opinions do not ship.
The control is the page as it runs today. Traffic splits between control and variant at the same time, so ad creative, promotions and seasonality hit both sides equally and the only difference is the change being tested.
Before anything ships I read conversion rate and revenue per visitor together, because a variant that sells more units at a lower order value can look like a win and still earn less. The full reading method is in how to read an A/B test before you ship it, and raw test screenshots are on the portfolio under receipts.
What happens after a test wins?
The winner rolls out sitewide, and what it taught feeds the next round. Compounding, not one-off wins. The output is a sitewide rollout.
Sitewide means the idea travels. A buy box structure that wins on one product page goes to the other product pages, and an angle that wins on one listicle becomes the starting point for the next pre-sell page.
Then step five is step one again. The rollout changes the funnel, the next audit reads the new data, and the next leak is already ranked.
What does a testing system produce over a year?
Growth that compounds across the whole store. The account behind my LinkedIn essay did $41.19M in the trailing 365 days, up 162% year over year.
The part that matters most: the conversion rate barely moved. In the essay I credit the growth to a testing system running across every traffic angle, every offer and every bundle, month after month.
That is why I judge a program on revenue, with conversion rate as one input. The dashboard is on the portfolio under receipts, and the account is written up in the $41.19M DTC store case study.
How does the loop map onto different scopes of work?
The loop stays the same on every account; the scope decides where it runs. Three accounts from the results ledger on the portfolio show the range.
| Account | Scope | Ledger figures |
|---|---|---|
| Physician's Choice | Sitewide testing program across 580K monthly sessions: PDPs, landing pages, offer and subscription flow | Store CVR 3.95%, sub adoption about 50% (levels, not lifts) |
| Natural Rems | Full funnel rebuild: PDP restructure, offer architecture, post-click flow | Conversion rate +40%, average order +25% |
| Serene Herbs | Listicle and landing page program plus PDP optimization for paid traffic | Conversion rate +30%, average order +20% |
Physician's Choice runs the loop at full width: every step works across the whole store, from PDPs and landing pages to the offer and the subscription flow. No baseline is published for its ledger figures, so they are program levels, not lifts.
Below are two Physician's Choice landing pages from the proof wall, live at time of writing. The Fiber Gummies landing page pairs its hero headline with a creator panel showing the product in hand, and puts a buy box with quantity tiers and a delivery plan below it. The Easy Mix Fiber landing page opens on the headline 'A cleaner, gentler daily fiber that actually works.' with flavor and offer pickers beneath.
Natural Rems was a full funnel rebuild: PDP restructure, offer architecture and the post-click flow. A rebuild still follows the loop: the audit decides what gets rebuilt, and the new funnel is measured against the one it replaced.
Serene Herbs is a listicle and landing page program plus PDP optimization for paid traffic. There the loop runs on the pre-sell pages cold traffic lands on and on the product page behind them. Both ledgers moved conversion rate and average order, the two halves of revenue per visitor.
FAQ
What is a leak map in CRO?
A leak map is the output of the audit: a list of the places where a store loses buyers, ranked by how many dollars each one costs. It turns a vague sense that the site underperforms into an ordered list, so the first test goes to the biggest leak instead of the page someone dislikes most.
What happens when an A/B test loses?
The control stays live, and the result still pays. A losing test keeps a bad idea off the site and tells you either the leak is somewhere else or the fix was wrong. The learning goes back into the backlog, and the next test on the ranked list runs.
Is a full redesign ever the right call?
Sometimes the scope is a rebuild, like the full funnel rebuild on Natural Rems: PDP restructure, offer architecture and post-click flow. The rule still holds: no redesigns on faith. A rebuild has to answer ranked leaks from the audit, and it has to prove itself in the numbers against what it replaced.
Can a smaller store run a CRO testing program?
Yes, with more discipline on the ranking step. With less traffic, each test takes longer to read, so fewer tests run and each one has to count. That makes the audit and the dollar ranking more valuable, because they point the limited traffic at the biggest leak first.
Why judge a testing program on more than conversion rate?
Because a program can grow revenue while the headline rate stays flat. On the $41.19M account, up 162% year over year, the conversion rate barely moved. The growth came from a testing system running across every traffic angle, offer and bundle, so I judge a program on revenue, with conversion rate as one of its inputs.