Peer groups that actually catch anomalies
Compare a merchant to the wrong 'normal' and the anomaly disappears. Measured: 91% of cases caught with peers built on identity, 24% with peers built on behavior.
The problem
[PLACEHOLDER: business framing — every anomaly detector answers “is this unusual?”, which is meaningless until you say “compared to what?”. 2 short paragraphs, e-commerce/payments framing.]
Every anomaly detector needs a definition of normal. Compare a corner tobacconist against all merchants and it looks strange every day of the week — low tickets, high volume. Compare it to other tobacconists and its real behavior becomes legible. That is what peer groups are for.
The tempting shortcut is to let the data draw the groups: cluster merchants by how they transact, then flag whoever deviates. This case study measures what that shortcut costs.
Why peer groups exist
Six merchant categories, 1.25M transactions. Median ticket runs from €7 at a tobacconist to €221 at an electronics retailer — over 30× apart. A €400 transaction is a Tuesday for one and an event for the other. Any detector without this context is measuring the category, not the behavior.
The trap
If peers are clustered on the same transactional behavior the detector then scores, the anomalous merchants are grouped with each other — and inside that group, they are perfectly normal. The detector eats its own signal.
Each point is a merchant, averaged over the scoring window. The flagged ones have drifted away from their category — small shops now pushing large tickets. The drift lives in the tail: a 95th-percentile ticket of about €240 for the drifting tobacconists (typical: €16) and €190 for the drifting pharmacies (typical: €44), a band of their own between fuel stations (€92) and electronics retailers (€638). Cluster by behavior and 18 of the 24 anomalous merchants — 12 tobacconists, 6 pharmacies — fall into a single cluster whose members are 100% anomalous. On the dimensions that drew that cluster, nothing inside it deviates from the local median: the local median is them.
The experiment
One detector, one feature set, one scoring rule — only the definition of the comparison group changes:
- No peers — every merchant against the whole population.
- Prior peers — the merchant’s declared category. A quasi-static identity attribute that the scored behavior cannot contaminate.
- Behavior peers — k-means over the same features the detector scores, fitted on the scoring window. Given the true number of categories as k, so it is not handicapped.
Scoring is a robust z (median/MAD) within peer, aggregated as the mean of the top-2 deviations. Evaluation is at analyst alert budgets — the top-k merchants reviewed per day — because a detector is judged by what a team can work, not by AUC.
Results
At 20 alerts per day: 90.6% of anomalous merchant-days caught with prior-based peers, 24.0% with behavior-based peers, 0% with no peers at all. Precision at that budget: 67.2% vs 17.8%. Same features, same model, same budget — the only difference is what “normal” was measured against.
The no-peer baseline is the sharper warning: it catches nothing. Its alert queue fills with supermarkets and electronics stores doing exactly what supermarkets and electronics stores do, because at population level they are the extremes.
One flagged pharmacy, two verdicts. The chart tracks its daily 95th-percentile ticket, where the shift lives. After the shift it averages €315 — against €46 for a typical pharmacy on the same days, an obvious outlier — but €243 for its behavior cluster, which formed around exactly this kind of tail. The detector’s verdict follows: alerted on 26 of its 29 anomalous days with prior peers, on 6 of 29 with behavior peers (20 alerts/day). The cluster absorbs the merchant on the very dimensions that make it anomalous.
Limitations & what didn’t work
[PLACEHOLDER: this section is the user’s to shape — the technical reviewer’s own caveats belong here in his voice.]
- This is a sandbox, and that is a deliberate trade. Proving a mechanism requires knowing the true peer structure, which no public dataset publishes. The generator is seeded and documented; absolute numbers are a property of it, and only the comparison between the three arms carries over. Confirmation on real data is the next step.
- The injected anomaly is a single pattern (low-ticket merchants pushing high-ticket volume). Detectors that win on one pattern routinely lose on another.
- Feature choice decides what any peer definition can see. The shift is modest on the median ticket (+43% for tobacconists, +52% for pharmacies) and dramatic in the tail (15× and 4× at the 95th percentile). A first version of this page plotted the median and its captions overstated what the charts showed; both were redone on the tail.
- Category is assumed honest. In production a misdeclared merchant category is itself a fraud vector, and it would quietly poison the prior peers this method depends on.
- k-means was given every advantage — correct k, standardized log features. It still failed, which is the point: this is not a tuning problem.
- Peer stability over long horizons is untested, as is the boundary question of how much stable structural behavior may legitimately enter a peer definition without reintroducing circularity.
What this would mean for your business
[PLACEHOLDER: client-facing closing — alert queues your team can work, fewer false alarms per real case, and the reason to ask “compared to what?” before buying any anomaly model.]