On the Hardness of Finding Symmetries in Markov Decision Processes

author: Balaraman Ravindran, Department of Computer Science and Engineering, Indian Institute of Technology

Description

In this work we address the question of finding symmetries of a given MDP. We show that the problem is Isomorphism Complete, that is, the problem is polynomially equivalent to verifying whether two graphs are isomorphic. Apart from the theoretical importance of this result it has an important practical application. The reduction presented can be used together with any off-the-shelf Graph Isomorphism solver, which performs well in the average case, to find symmetries of an MDP. In fact, we present results of using NAutY (the best Graph Isomorphism solver currently available), to find symmetries of MDPs.

You might be experiencing some problems with Your Video player.

*Slides*
0:00	On the Hardness of Finding Symmetries in Markov Decision Processes
0:27	Outline
0:39	Overview
2:01	Outline - Introduction
2:01	Stochastic Sequential Decision Making
2:28	Solution of an MDP
2:33	Outline - Introduction
2:34	Reduced Model - Formal definition
3:40	Reduced Model - Significance
4:24	Symmetry informally
4:59	Symmetry Formal Definition
6:02	Problem
6:26	Outline - Symmetries in MDPs
6:29	Problem Simplification
6:44	Isomorphism Completeness
7:07	List of relevant Isomorphism Complete Problems
7:45	Outline
8:34	Set Bijections
9:52	Set Bijections example
11:01	An example (1)
11:14	An example (2)
11:55	An example (3)
13:43	An example (4)
13:47	An example (3)
14:01	An example (4)
14:45	An example (5)
16:15	Construction provides the Generators of AutM
16:36	Significance
16:46	Nauty - No Automorphisms, Yes?
17:15	Nauty Integration
17:24	Outline - Symmetries in MDPs
17:25	G -Reduced Image Algorithm IJCAI 07
17:49	Outline - Symmetries in MDPs
17:51	G -Reduced Image Algorithm IJCAI 07
17:52	Experimental Setup - Probabilistic GridWorld
17:54	Experimental Setup - GridWorld Soccer
18:28	Results - Probabilistic GridWorld
18:59	Results - GridWorld Soccer
19:45	Summary
20:24	Thank You!

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.