Field note
Nine months to prove we were right
Two platforms disagreed on externally reported metrics. Three people spent nine months rebuilding them by hand from feeder systems.
On one programme I worked on, the old and new platforms disagreed on a set of externally reported metrics. Both were internally consistent. Neither could prove itself, because each was arguing from its own transformation logic and the dispute was precisely about whose logic was right. There was no arbiter inside either system.
The only way to settle it was to go outside both and rebuild the metrics by hand from the feeder systems — manual extraction, manual integration, manual deduplication, manual aggregation, manual calculation. Three people, nine months. The finding was that the new platform was 99.4% accurate and the old one was materially worse, which is to say the answer the team had believed from the start was correct.
It cost the programme nine months anyway. And the organisation had to restate key metrics to the markets, with everything that implies legally and reputationally.
Where this feeds the guide: The numbers will not match.