NATO Advanced Study Institute on Mining Massive Data Sets for Security

Information Theo-retic and Alge-braic Methods for Network Anomaly Detection

author: Naftali Tishby , Hebrew University of Jerusalem

Description

The tutorial will discuss two central issues: (i) Information Theoretic principles and algorithms for extracting predictive statistics in distributed networks and (ii) algebraic and spectral methods for network anomaly detection. The first part will deal with the concept of predictive information - the mutual information between the past and future of a process, its sub-extensive properties, and algorithms for estimating it from data.We will argue that the information theoretic predictability quantifies the complexity of a process and provides effective ways for detecting anomalies and surprises in the process. Using the Information Bottleneck algorithms one can extract approximate sufficient statistics from the past to the future of the process and use them as anomaly detectors on multiple time scales. In the second part we will discuss ways for analyzing network activity using spectral methods (distributed PCA and network Laplacian analysis) for identifying regular temporal patterns of connected network components. By combining the two approaches, we will suggest new techniques for network anomaly detectors for security.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	Algebraic and Information Theoretic Methods forNetwork Anomaly Detection
1:50	Outline
11:56	Statement of the problem
16:13	"...drowning in data but starving for knowledge"
16:38	Biological neural networks
17:24	Biochemical interactions
17:32	Gene expression analysis
18:16	Example: Wireless Sensor Networks
18:45	An Object Moving Through the Network
19:46	Graph Thoretical Formulation
25:01	Undirected graph - Symmetric matrix
26:59	Security Issues
30:07	Undirected graph - Symmetric matrix
31:34	Security Issues
31:35	Algebraic Methods - Static Networks (1)
35:12	Algebraic Methods - Static Networks (2)
41:46	Algebraic Methods - Static Networks (3)
48:32	Laplacian eigenvector decomposition
53:50	Application: Using Spectral Embedding for Novelty Detection in communication networks
55:41	Reordering the nodes based on Spectral decomposition
56:40	Simple illustration
58:14	Distances between graphs (1)
59:35	Distances between graphs (2)
60:56	Distances between graphs (3)
62:05	Distances between graphs (4)
62:25	Distances between graphs (2)
70:16	Diffusion on Graphs
75:10	Computational comment
79:54	Diffusion on time dependent graphs
82:27	Predictive Information
84:19	Why Predictability? (1)
84:28	Why Predictability? (2)
86:13	Predictive Information (with Bialek and Nemenman, 2001)

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.