A new sign-up page made the numbers look like a crash, but most of that was just a change in counting. The real story was one step further in, where people were quietly slipping away.
/ The problem
A company redesigned the sign-up page inside its product. Right after, the number of sales calls booked from inside that product dropped by about three-quarters.
That is the kind of number that gets a new design pulled on Monday morning. But nobody had checked the one thing that mattered first: were the calls really gone, or had the new page just stopped counting them the old way? On a report, those two look exactly the same.
/ What we did
We followed one person's whole path, from signing up, to answering the questions, to booking a call, to buying. Then we rebuilt that path for the old page and the new page, over the same stretch of time.
That meant going back to the original records instead of trusting the report: the product's own records, for who signed up and who qualified; the booking tool, for who actually booked a call; and the sales tracker, for who bought. We matched all three by email address, so we could tell a real drop-off apart from someone who simply wasn't being counted.
We only counted a booking as coming from the product if it carried a tracking tag, a small marker in the link that shows exactly where someone came from. Before we added that rule, we had tried counting by timing instead, and it gave the product credit for bookings that actually came from somewhere else.
Investigation · where everyone goes
4%
of everyone who reached the questions cleared both checks
/ What changed
Most of the drop wasn't real. About the same number of calls were booked before and after, and the same share turned into sales. The "crash" was the new page counting differently, not people leaving.
But one piece was real. The old page let people book without leaving the form, and 92% of those who qualified went through. The new page sends them to a separate scheduling page, and only 25% make it. That is a real leak the redesign caused, and it had been hiding inside a number everyone had already written off as a counting mistake.
The bigger problem came one step earlier. Of everyone who reached the qualifying questions, 76% weren't allowed to work in the country. Of the ones who got past that, 82% said they weren't ready to spend. Only 4% cleared both.
The conversation changed. It stopped being "did the new design break something?" and became two clear questions: fix the handoff to the scheduling page, and take a hard look at who is showing up in the first place.
/ Decisions worth explaining
Guessing which bookings came from the product, based on timing, felt reasonable and made the product look good. But the tag is the only thing that proves where someone actually came from.
The old one counted a finished form. The new one counts a tagged booking. Close, but not the same, and calling it a clean comparison would have hidden that seam.
A floor is the least something really happened, not the full amount. 96% of bookings carry no tracking tag at all, so we could only count the part we could prove, not everything that actually took place. Comparing two floors against each other is still fair. Calling a floor a total would not be.
The closing rate matched before and after, but the "after" side rests on just a handful of sales, too few to lean on. It backs up the earlier number without proving it, and the report says so, instead of letting a matching figure do work it can't.