Data Mining for Anomaly Detection

author:Jaideep Srivastava, University of Minnesota
author:Varun Chandola, University of Minnesota
author:Vipin Kumar, University of Minnesota
author:Aleksandar Lazarevic, United Technologies Research Center
author:Arindam Banerjee, The University of Texas at Austin
published: Oct. 10, 2008,   recorded: September 2008,   views: 1222
Categories
You might be experiencing some problems with Your Video player.

Slides

Slides
0:00 Data Mining for Anomaly Detection
0:35 Outline
1:37 Introduction
2:07 What are Anomalies?
3:33 Real World Anomalies
3:47 Simple Examples
5:15 Related problems
5:53 Key Challenges
9:03 Aspects of Anomaly Detection Problem
9:36 Input Data (1)
9:59 Input Data (2)
10:31 Input Data – Nature of Attributes
11:06 Input Data – Complex Data Types
11:53 Data Labels
13:37 Type of Anomaly
14:08 Point Anomalies
14:25 Contextual Anomalies
15:42 Collective Anomalies
16:35 Output of Anomaly Detection
17:23 Evaluation of Anomaly Detection – F-value
19:57 Evaluation of Outlier Detection – ROC & AUC
21:39 Applications of Anomaly Detection
22:22 Intrusion Detection
23:59 Fraud Detection
25:07 Healthcare Informatics
26:08 Industrial Damage Detection
28:08 Image Processing
29:13 Taxonomy (1)
31:10 Classification Based Techniques (1)
33:09 Classification Based Techniques (2)
34:45 Supervised Classification Techniques
35:53 Manipulating Data Records
39:06 Rule Based Techniques
41:48 New Rule-based Algorithms: PN-rule Learning
42:58 New Rule-based Algorithms: CREDOS
44:23 Using Neural Networks
45:14 Using Support Vector Machines
46:13 Semi-supervised Classification Techniques
46:58 Using Replicator Neural Networks
49:00 Using Support Vector Machines
50:37 Taxonomy (2)
51:04 Nearest Neighbor Based Techniques (1)
52:09 Nearest Neighbor Based Techniques (2)
53:59 Nearest Neighbor Based Techniques (3)
55:16 Distance based Outlier Detection
56:25 Advantages of Density based Techniques
58:25 Local Outlier Factor (LOF)

Related content

Visitors who watched this lecture also watched...
48:34
Machine Learning for Intrusion Detection

666 views - Pavel Laskov, 2007
01:43:27
Statistical techniques for fraud detection, prevention, and evaluation

2291 views - David J. Hand, 2007
03:24:20
Lectures on Clustering

5737 views - Ulrike von Luxburg, 2007
04:59:19
Machine Learning, Probability and Graphical Models

18444 views - Sam Roweis, 2006
03:54:31
Support Vector Machines

12760 views - Chih-Jen Lin, 2006
01:36:27
PhD Thesis Defense: Dynamics of large networks

10188 views - Jure Leskovec, 2008
03:06:58
Data Mining

686 views - Rao Kotagiri, 2009
01:43:02
Fuzzy Logic

16715 views - Michael Berthold, 2005
02:23:50
Mining Massive RFID, Trajectory, and Traffic Data Sets

2185 views - Jiawei Han, Jae-Gil Lee, Hector Gonzalez, Xiaolei Li, 2008
02:39:32
Data Mining and Knowledge Discovery

1473 views - Nada Lavrač, 2007

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.

 Watch videos:   (click on thumbnail to launch)

Watch Part 1
Part 1 1:00:58 Flash video Slides Windows Media video
!NOW PLAYING
Watch Part 2
Part 2 0:58:26 Flash video Windows Media video
Watch Part 3
Part 3 0:23:56 Flash video Windows Media video

Description

Anomaly detection corresponds to discovery of events that typically do not conform to expected normal behavior. Such events are often referred to as anomalies, outliers, exceptions, deviations, aberrations, surprise, peculiarities or contaminants in different application domains Detection of anomalies is a common problem in many domains, such as detecting fraudulent credit card transactions, insurance and tax fraud detection, intrusion detection for cyber security, failure detection, direct marketing, and medical diagnostics. Although anomalies are by definition infrequent, in many examples their importance is quite high compared to other events, making their detection extremely important. This tutorial will provide an overview of the research done in the increasingly important field of anomaly detection. The tutorial will cover the existing literature from a variety of perspectives, such as nature of input/output, and the availability of supervision. Anomalies will be divided into three broad groups: (i) Point anomalies, (ii) Contextual anomalies, and (iii) Structural anomalies, and a wide variety of anomaly detection methods appropriate for each type of anomaly will be presented. Additionally, the tutorial will discuss several application domains, such as intrusion detection, fraud detection, industrial damage detection, healthcare informatics, where anomaly detection plays a central role.

Link this page  

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 Lucas Fettucci, July 28, 2009 at 5:16 p.m.:

Part 1 does not work: AJAX error: parsererror


Comment2 shenjun, December 5, 2009 at 4:49 p.m.:

i am instresting in that letcure!I wish get more information about the kwnolege in this field!

Write your own review or comment:

make sure you have javascript enabled or clear this field: