Organisations are collecting massive amounts of data. Most of the time, they focus on finding the trends that will help drive business decisions. Yet, in some cases, the most valuable information is found in those cases that deviate from the trend.
Anomaly detection, or outlier analysis, helps to expose unusual events and errors. Within data sets, the data patterns represent business as usual. An unexpected change within these data patterns, or an event that does not conform to the expected data pattern, is considered an anomaly.
Anomaly detection helps companies detect critical incidents, such as technical glitches in malfunctioning equipment. But the same technologies are used to detect fraudulent and criminal behaviour – which are often behaviours that deviate subtly from common patterns.
Anomaly detection can be applied in different contexts in a wide range of industries. In a business context, we successfully deployed anomaly detection to investigate internal procurement spend to uncover savings opportunities, process deviations, and possible fraud. In a more technical context, we have used it to detect leakages in a manufacturing pipeline.