Decentralized Control of Multi-Robot Partially Observable Markov Decision Processes using Belief Space Macro-actions



Project Description  (top)


Decentralized Control of Multi-Robot Partially Observable Markov Decision Processes using Belief Space Macro-actions by Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Christopher Amato, Shih-Yuan Liu, Miao Liu, and Jonathan P. How


This work focuses on solving multi-robot planning problems in continuous spaces with partial observability given a high-level domain description. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems. However, representing and solving Dec-POMDPs is often intractable for large problems.

This work extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP) to take advantage of high-level representations that are natural for multi-robot problems and to facilitate scalable solutions to large discrete and continuous problems. The Dec-POSMDP formulation uses task macro-actions created from lower-level primitive actions that allow asynchronous decision-making, which is crucial in multi-robot domains. We also present algorithms for solving Dec-POSMDPs, which are more scalable than previous methods since they can incorporate closed-loop belief space macro-actions in planning. The proposed algorithms are then evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent realistic domains and provide high-quality solutions for large-scale problems.




Videos  (top)



Related publications  (top)

  • Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-Actions, (Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Christopher Amato, Jonathan P. How) In Proc. of the IEEE Int'l Conf. on Robotics and Automation (ICRA-15), May 2015. 
  • Stick-Breaking Policy Learning in DEC-POMDPs (Miao Liu, Chris Amato, Xuejun Liao, Lawrence Carin, Jonathan P. How), In Proc. of the 24th Intīl Joint Conf. on Artificial Intelligence (IJCAI-15), July 2015.