Why Did It Drop?

An analytics agent that explains why a product metric moved, and says when it can't. Every case below is a real week of Google's demo-store data. Most have an incident planted in them, with the answer known. You can see what the agent concluded, what was really there, and the evidence it saw.

How it decides

  1. Is the change real? This week against the usual week-to-week swing.
  2. Can the data be trusted? Tracking checks look for signatures no shopper produces: duplicate purchase events, purchases with no checkout, orders with zero revenue.
  3. Where is it? Segments that moved differently from the rest of the site, more than they usually do, in a way that fits a failure and made the asked metric worse.
  4. What does it mean? A model (gemini-3.5-flash-lite) reads that evidence and writes the diagnosis, citing every number by reference. Code fills in the figures and checks each claim; anything unsupported becomes "can't tell".

What the results mean

The release gate was strict: at most 5% wrong causes, at least 75% of incidents right, at most 10% false alarms on quiet weeks. The held-out run failed it. Most misses came from the data's own real campaign shifts (new visitors in early December, Home-page traffic over New Year), named instead of the planted incident. The model also added no decision value over the deterministic rules; its job is the explanation. Every run, with its errors · How the benchmark was built

Data: Google's public GA4 demo-store export (Google Merchandise Store, Nov 2020 to Jan 2021, obfuscated by Google). Built by Akshay Kumar Dhar · Code and data