fighterxi / phd-bibliography Public

forked from eleurent/phd-bibliography

Notifications You must be signed in to change notification settings
Fork 0
Star 0

References on Optimal Control, Reinforcement Learning and Motion Planning

0 stars 207 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 145 Commits
README.md		README.md
reinforcement-learning.graphml		reinforcement-learning.graphml
reinforcement-learning.svg		reinforcement-learning.svg

Repository files navigation

Bibliography

Table of contents

Optimal Control

Dynamic Programming

(book) Dynamic Programming, Bellman R. (1957).
(book) Dynamic Programming and Optimal Control, Volumes 1 and 2, Bertsekas D. (1995).
(book) Markov Decision Processes - Discrete Stochastic Dynamic Programming, Puterman M. (1995).

Approximate Planning

ExpectiMinimax Optimal strategy in games with chance nodes, Melkó E., Nagy B. (2007).
Sparse sampling A sparse sampling algorithm for near-optimal planning in large Markov decision processes, Kearns M. et al. (2002).
MCTS Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search, Rémi Coulom, SequeL (2006).
UCT Bandit based Monte-Carlo Planning, Kocsis L., Szepesvári C. (2006).
OPD Optimistic Planning for Deterministic Systems, Hren J., Munos R. (2008).
OLOP Open Loop Optimistic Planning, Bubeck S., Munos R. (2010).
LGP Logic-Geometric Programming: An Optimization-Based Approach to Combined Task and Motion Planning, Toussaint M. (2015). 🎞️
AlphaGo Mastering the game of Go with deep neural networks and tree search, Silver D. et al. (2016).
AlphaGo Zero Mastering the game of Go without human knowledge, Silver D. et al. (2017).
AlphaZero Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm, Silver D. et al. (2017).
TrailBlazer Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning, Grill J. B., Valko M., Munos R. (2017).
MCTSnets Learning to search with MCTSnets, Guez A. et al. (2018).
ADI Solving the Rubik's Cube Without Human Knowledge, McAleer S. et al. (2018).

Control Theory

(book) Constrained Control and Estimation, Goodwin G. (2005).
PI² A Generalized Path Integral Control Approach to Reinforcement Learning, Theodorou E. et al. (2010).
PI²-CMA Path Integral Policy Improvement with Covariance Matrix Adaptation, Stulp F., Sigaud O. (2010).
iLQG A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems, Todorov E. (2005).
iLQG+ Synthesis and stabilization of complex behaviors through online trajectory optimization, Tassa Y. (2012).

Model Predictive Control

(book) Model Predictive Control, Camacho E. (1995).
(book) Predictive Control With Constraints, Maciejowski J. M. (2002).
Linear Model Predictive Control for Lane Keeping and Obstacle Avoidance on Low Curvature Roads, Turri V. et al. (2013).
MPCC Optimization-based autonomous racing of 1:43 scale RC cars, Liniger A. et al. (2014). 🎞️ | 🎞️
MIQP Optimal trajectory planning for autonomous driving integrating logical constraints: An MIQP perspective, Qian X., Altché F., Bender P., Stiller C. de La Fortelle A. (2016).

Safe Control

Robust Control

Minimax analysis of stochastic problems, Shapiro A., Kleywegt A. (2002).
Robust DP Robust Dynamic Programming, Iyengar G. (2005).
Robust Planning and Optimization, Laumanns M. (2011). (lecture notes)
Robust Markov Decision Processes, Wiesemann W., Kuhn D., Rustem B. (2012).
Coarse-Id On the Sample Complexity of the Linear Quadratic Regulator, Dean S., Mania H., Matni N., Recht B., Tu S. (2017).
Tube-MPPI Robust Sampling Based Model Predictive Control with Sparse Objective Information, Williams G. et al. (2018). 🎞️

Risk-Averse Control

A Comprehensive Survey on Safe Reinforcement Learning, García J., Fernández F. (2015).
RA-QMDP Risk-averse Behavior Planning for Autonomous Driving under Uncertainty, Naghshvar M. et al. (2018).

Constrained Control

ICS Will the Driver Seat Ever Be Empty?, Fraichard T. (2014).
RSS On a Formal Model of Safe and Scalable Self-driving Cars, Shalev-Shwartz S. et al. (2017).
BFTQ Safe Transfer across Reinforcement Learning Tasks, Carrara N. et al. (2018).
MPC-HJI On Infusing Reachability-Based Safety Assurance within Probabilistic Planning Frameworks for Human-Robot Vehicle Interactions, Leung K. et al. (2018).

Uncertain Dynamical Systems

Simulation of Controlled Uncertain Nonlinear Systems, Tibken B., Hofer E. (1995).
Trajectory computation of dynamic uncertain systems, Adrot O., Flaus J-M. (2002).
Simulation of Uncertain Dynamic Systems Described By Interval Models: a Survey, Puig V. et al. (2005).
Design of interval observers for uncertain dynamical systems, Efimov D., Raïssi T. (2016).

Sequential Learning

Multi-Armed Bandit

UCB1/UCB2 Finite-time Analysis of the Multiarmed Bandit Problem, Auer P., Cesa-Bianchi N., Fischer P. (2002).
kl-UCB The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond, Garivier A., Cappé O. (2011).
KL-UCB Kullback-Leibler Upper Confidence Bounds for Optimal Sequential Allocation, Cappé O. et al. (2013).
LUCB PAC Subset Selection in Stochastic Multi-armed Bandits, Kalyanakrishnan S. et al. (2012).
Track-and-Stop Optimal Best Arm Identification with Fixed Confidence, Garivier A., Kaufmann E. (2016).
M-LUCB/M-Racing Maximin Action Identification: A New Bandit Framework for Games, Garivier A., Kaufmann E., Koolen W. (2016).
LUCB-micro Structured Best Arm Identification with Fixed Confidence, Huang R. et al. (2017).
Bayesian Optimization in AlphaGo, Chen Y. et al. (2018)

Reinforcement Learning

Reinforcement learning: A survey, Kaelbling L. et al. (1996).

Value-based

NFQ Neural fitted Q iteration - First experiences with a data efficient neural Reinforcement Learning method, Riedmiller M. (2005).
DQN Playing Atari with Deep Reinforcement Learning, Mnih V. et al. (2013). 🎞️
DDQN Deep Reinforcement Learning with Double Q-learning, van Hasselt H., Silver D. et al. (2015).
DDDQN Dueling Network Architectures for Deep Reinforcement Learning, Wang Z. et al. (2015). 🎞️
PDDDQN Prioritized Experience Replay, Schaul T. et al. (2015).
NAF Continuous Deep Q-Learning with Model-based Acceleration, Gu S. et al. (2016).
Rainbow Rainbow: Combining Improvements in Deep Reinforcement Learning, Hessel M. et al. (2017).
Ape-X DQfD Observe and Look Further: Achieving Consistent Performance on Atari, Pohlen T. et al. (2018). 🎞️

Policy-based

Policy gradient

REINFORCE Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning, Williams R. (1992).
Natural Gradient A Natural Policy Gradient, Kakade S. (2002).
Policy Gradient Methods for Robotics, Peters J., Schaal S. (2006).
TRPO Trust Region Policy Optimization, Schulman J. et al. (2015). 🎞️
PPO Proximal Policy Optimization Algorithms, Schulman J. et al. (2017). 🎞️
DPPO Emergence of Locomotion Behaviours in Rich Environments, Heess N. et al. (2017). 🎞️

Actor-critic

AC Policy Gradient Methods for Reinforcement Learning with Function Approximation, Sutton R. et al. (1999).
NAC Natural Actor-Critic, Peters J. et al. (2005).
DPG Deterministic Policy Gradient Algorithms, Silver D. et al. (2014).
DDPG Continuous Control With Deep Reinforcement Learning, Lillicrap T. et al. (2015). 🎞️ 1 | 2 | 3 | 4
A3C Asynchronous Methods for Deep Reinforcement Learning, Mnih V. et al 2016. 🎞️ 1 | 2 | 3
SAC Soft Actor-Critic : Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, Haarnoja T. et al. (2018). 🎞️

Derivative-free

CEM Learning Tetris Using the Noisy Cross-Entropy Method, Szita I., Lörincz A. (2006). 🎞️
CMAES Completely Derandomized Self-Adaptation in Evolution Strategies, Hansen N., Ostermeier A. (2001).
NEAT Evolving Neural Networks through Augmenting Topologies, Stanley K. (2002). 🎞️

Model-based

Dyna Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming, Sutton R. (1990).
UCRL2 Near-optimal Regret Bounds for Reinforcement Learning, Jaksch T. (2010).
PILCO PILCO: A Model-Based and Data-Efficient Approach to Policy Search, Deisenroth M., Rasmussen C. (2011). (talk)
DBN Probabilistic MDP-behavior planning for cars, Brechtel S. et al. (2011).
GPS End-to-End Training of Deep Visuomotor Policies, Levine S. et al. (2015). 🎞️
DeepMPC DeepMPC: Learning Deep Latent Features for Model Predictive Control, Lenz I. et al. (2015). 🎞️
SVG Learning Continuous Control Policies by Stochastic Value Gradients, Heess N. et al. (2015). 🎞️
Optimal control with learned local models: Application to dexterous manipulation, Kumar V. et al. (2016). 🎞️
BPTT Long-term Planning by Short-term Prediction, Shalev-Shwartz S. et al. (2016). 🎞️ 1 | 2
Deep visual foresight for planning robot motion, Finn C., Levine S. (2016). 🎞️
VIN Value Iteration Networks, Tamar A. et al (2016). 🎞️
VPN Value Prediction Network, Oh J. et al. (2017).
An LSTM Network for Highway Trajectory Prediction, Altché F., de La Fortelle A. (2017).
DistGBP Model-Based Planning with Discrete and Continuous Actions, Henaff M. et al. (2017). 🎞️ 1 | 2
Prediction and Control with Temporal Segment Models, Mishra N. et al. (2017).
Predictron The Predictron: End-To-End Learning and Planning, Silver D. et al. (2017). 🎞️
MPPI Information Theoretic MPC for Model-Based Reinforcement Learning, Williams G. et al. (2017). 🎞️
Learning Real-World Robot Policies by Dreaming, Piergiovanni A. et al. (2018).

Temporal Abstraction

Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning, Sutton R. et al. (1999).
Intrinsically motivated learning of hierarchical collections of skills, Barto A. et al. (2004).
Learning and Transfer of Modulated Locomotor Controllers, Heess N. et al. (2016). 🎞️
Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving, Shalev-Shwartz S. et al. (2016).
FuNs FeUdal Networks for Hierarchical Reinforcement Learning, Vezhnevets A. et al. (2017).
Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments, Paxton C. et al. (2017). 🎞️

Partial Observability

PBVI Point-based Value Iteration: An anytime algorithm for POMDPs, Pineau J. et al. (2003).
cPBVI Point-Based Value Iteration for Continuous POMDPs, Porta J. et al. (2006).
POMCP Monte-Carlo Planning in Large POMDPs, Silver D., Veness J. (2010).
A POMDP Approach to Robot Motion Planning under Uncertainty, Du Y. et al. (2010).
Solving Continuous POMDPs: Value Iteration with Incremental Learning of an Efficient Space Representation, Brechtel S. et al. (2013).
Probabilistic Decision-Making under Uncertainty for Autonomous Driving using Continuous POMDPs, Brechtel S. et al. (2014).
MOMDP Intention-Aware Motion Planning, Bandyopadhyay T. et al. (2013).
The value of inferring the internal state of traffic participants for autonomous freeway driving, Sunberg Z. et al. (2017).

Transfer

Virtual to Real Reinforcement Learning for Autonomous Driving, Pan X. et al. (2017). 🎞️
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots, Tan J. et al. (2018). 🎞️
ME-TRPO Model-Ensemble Trust-Region Policy Optimization, Kurutach T. et al. (2018). 🎞️
Kickstarting Deep Reinforcement Learning, Schmitt S. et al. (2018).
Learning Dexterous In-Hand Manipulation, OpenAI (2018). 🎞️

Multi-agent

Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems, Albrecht S., Stone P. (2017).
MILP Time-optimal coordination of mobile robots along specified paths, Altché F. et al. (2016). 🎞️
MIQP An Algorithm for Supervised Driving of Cooperative Semi-Autonomous Vehicles, Altché F. et al. (2017). 🎞️
SA-CADRL Socially Aware Motion Planning with Deep Reinforcement Learning, Chen Y. et al. (2017). 🎞️
Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment, Galceran E. et al. (2017).
Online decision-making for scalable autonomous systems, Wray K. et al. (2017).
MAgent MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence, Zheng L. et al. (2017). 🎞️
Cooperative Motion Planning for Non-Holonomic Agents with Value Iteration Networks, Rehder E. et al. (2017).
COMA Counterfactual Multi-Agent Policy Gradients, Foerster J. et al. (2017).
FTW Human-level performance in first-person multiplayer games with population-based deep reinforcement learning, Jaderberg M. et al. (2018). 🎞️

Representation Learning

DeepDriving DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving, Chen C. et al. (2015). 🎞️
On the Sample Complexity of End-to-end Training vs. Semantic Abstraction Training, Shalev-Shwartz S. et al. (2016).
Learning sparse representations in reinforcement learning with sparse coding, Le L., Kumaraswamy M., White M. (2017).
World Models, Ha D., Schmidhuber J. (2018). 🎞️
Learning to Drive in a Day, Kendall A. et al. (2018). 🎞️
MERLIN Unsupervised Predictive Memory in a Goal-Directed Agent, Wayne G. et al. (2018). 🎞️ 1 | 2 | 3 | 4 | 5 | 6
Variational End-to-End Navigation and Localization, Amini A. et al. (2018). 🎞️
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks, Lee M. et al. (2018). 🎞️

Other

Is the Bellman residual a bad proxy?, Geist M., Piot B., Pietquin O. (2016).
Deep Reinforcement Learning that Matters, Henderson P. et al. (2017).
Automatic Bridge Bidding Using Deep Reinforcement Learning, Yeh C. and Lin H. (2016).
Shared Autonomy via Deep Reinforcement Learning, Reddy S. et al. (2018). 🎞️
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review, Levine S. (2018).

Learning from Demonstrations

Imitation Learning

QMDP-RCNN Reinforcement Learning via Recurrent Convolutional Neural Networks, Shankar T. et al. (2016). (talk)
DQfD Learning from Demonstrations for Real World Reinforcement Learning, Hester T. et al. (2017). 🎞️
Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy, Barnes D., Maddern W., Posner I. (2016). 🎞️
GAIL Generative Adversarial Imitation Learning, Ho J., Ermon S. (2016).
From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots, Pfeiffer M. et al. (2017). 🎞️
Branched End-to-end Driving via Conditional Imitation Learning, Codevilla F. et al. (2017). 🎞️ | talk
UPN Universal Planning Networks, Srinivas A. et al. (2018). 🎞️
DeepMimic DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills, Peng X. B. et al. (2018). 🎞️
R2P2 Deep Imitative Models for Flexible Inference, Planning, and Control, Rhinehart N. et al. (2018). 🎞️

Applications to Autonomous Driving

ALVINN, an autonomous land vehicle in a neural network, Pomerleau D. (1989).
End to End Learning for Self-Driving Cars, Bojarski M. et al. (2016). 🎞️
End-to-end Learning of Driving Models from Large-scale Video Datasets, Xu H., Gao Y. et al. (2016). 🎞️
End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies, Eraqi H. et al. (2017).
Driving Like a Human: Imitation Learning for Path Planning using Convolutional Neural Networks, Rehder E. et al. (2017).
Imitating Driver Behavior with Generative Adversarial Networks, Kuefler A. et al. (2017).
PS-GAIL Multi-Agent Imitation Learning for Driving Simulation, Bhattacharyya R. et al. (2018). 🎞️

Inverse Reinforcement Learning

Projection Apprenticeship learning via inverse reinforcement learning, Abbeel P., Ng A. (2004).
MMP Maximum margin planning, Ratliff N. et al. (2006).
BIRL Bayesian inverse reinforcement learning, Ramachandran D., Amir E. (2007).
MEIRL Maximum Entropy Inverse Reinforcement Learning, Ziebart B. et al. (2008).
LEARCH Learning to search: Functional gradient techniques for imitation learning, Ratliff N., Siver D. Bagnell A. (2009).
CIOC Continuous Inverse Optimal Control with Locally Optimal Examples, Levine S., Koltun V. (2012). 🎞️
MEDIRL Maximum Entropy Deep Inverse Reinforcement Learning, Wulfmeier M. (2015).
GCL Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization, Finn C. et al. (2016). 🎞️
RIRL Repeated Inverse Reinforcement Learning, Amin K. et al. (2017).
Bridging the Gap Between Imitation Learning and Inverse Reinforcement Learning, Piot B. et al. (2017).

Applications to Autonomous Driving

Apprenticeship Learning for Motion Planning, with Application to Parking Lot Navigation, Abbeel P. et al. (2008).
Navigate like a cabbie: Probabilistic reasoning from observed context-aware behavior, Ziebart B. et al. (2008).
Planning-based Prediction for Pedestrians, Ziebart B. et al. (2009). 🎞️
Learning for autonomous navigation, Bagnell A. et al. (2010).
Learning Autonomous Driving Styles and Maneuvers from Expert Demonstration, Silver D. et al. (2012).
Learning Driving Styles for Autonomous Vehicles from Demonstration, Kuderer M. et al. (2015).
Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks, Sharifzadeh S. et al. (2016).
Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments, Wulfmeier M. (2016). 🎞️
Planning for Autonomous Cars that Leverage Effects on Human Actions, Sadigh D. et al. (2016).
A Learning-Based Framework for Handling Dilemmas in Urban Automated Driving, Lee S., Seo S. (2017).

Motion Planning

Search

Dijkstra A Note on Two Problems in Connexion with Graphs, Dijkstra E. W. (1959).
A* A Formal Basis for the Heuristic Determination of Minimum Cost Paths , Hart P. et al. (1968).
Planning Long Dynamically-Feasible Maneuvers For Autonomous Vehicles, Likhachev M., Ferguson D. (2008).
Optimal Trajectory Generation for Dynamic Street Scenarios in a Frenet Frame, Werling M., Kammel S. (2010). 🎞️
3D perception and planning for self-driving and cooperative automobiles, Stiller C., Ziegler J. (2012).
Motion Planning under Uncertainty for On-Road Autonomous Driving, Xu W. et al. (2014).
Monte Carlo Tree Search for Simulated Car Racing, Fischer J. et al. (2015). 🎞️

Sampling

RRT* Sampling-based Algorithms for Optimal Motion Planning, Karaman S., Frazzoli E. (2011). 🎞️
LQG-MP LQG-MP: Optimized Path Planning for Robots with Motion Uncertainty and Imperfect State Information, van den Berg J. et al. (2010).
Motion Planning under Uncertainty using Differential Dynamic Programming in Belief Space, van den Berg J. et al. (2011).
Rapidly-exploring Random Belief Trees for Motion Planning Under Uncertainty, Bry A., Roy N. (2011).
PRM-RL PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning, Faust A. et al. (2017).

Optimization

Trajectory planning for Bertha - A local, continuous method, Ziegler J. et al. (2014).
Learning Attractor Landscapes for Learning Motor Primitives, Ijspeert A. et al. (2002).

Reactive

PF Real-time obstacle avoidance for manipulators and mobile robots, Khatib O. (1986).
VFH The Vector Field Histogram - Fast Obstacle Avoidance For Mobile Robots, Borenstein J. (1991).
VFH+ VFH+: Reliable Obstacle Avoidance for Fast Mobile Robots, Ulrich I., Borenstein J. (1998).
Velocity Obstacles Motion planning in dynamic environments using velocity obstacles, Fiorini P., Shillert Z. (1998).

Architecture and applications

A Review of Motion Planning Techniques for Automated Vehicles, González D. et al. (2016).
A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles, Paden B. et al. (2016).
Autonomous driving in urban environments: Boss and the Urban Challenge, Urmson C. et al. (2008).
The MIT-Cornell collision and why it happened, Fletcher L. et al. (2008).
Making bertha drive-an autonomous journey on a historic route, Ziegler J. et al. (2014).

About

References on Optimal Control, Reinforcement Learning and Motion Planning

Report repository

Releases

No releases published

Packages

No packages published