Search notes:

Anomaly detection

Anomaly detection, aka outlier detection: identification of items, events and/or observations that do not conform to an expected pattern or other items in a dataset.

Anomaly detection might be used in

Intrusion detection
Fraud detection
Fault detection
System health monitoring
Preprocessing a data set before data preparation

Techniques

Density-based (k-nearest neighbor, local outlier factor …)
Subspace- and correlation-based (high-dimensional data)
One-class support vector machines.
Replicator neural networks
Bayesian Networks.
Hidden Markov models (HMMs).
Cluster analysis
Deviations from association rules and frequent itemsets.
Fuzzy logic

These techniques are applied

Unsupervised,
Supervised or
Semi-supervised

Outlier detection to test data quality

Outlier data can be views as »totally different« from the rest of the data. Hakwins (Outlier detection using replicator neural networks, 2002) defined it more formally as

An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism.

With such a »definition«, the observation of outlier data might be used to asses data quality.

Links

ODDS provide access to a large collection of outlier detection datasets with ground truth (if available).

Index