Design of Experiments for Reinforcement Learning

GLOSSARY
ACKNOWLEDGMENT
FOREWARD
1. INTRODUCTION
2. REINFORCEMENT LEARNING
2.1 Applications of reinforcement learning
2.1.1 Benchmark problems
2.1.2 Games
2.1.3 Real-world applications
2.1.4 Generalized domains
2.2 Components of reinforcement learning
2.2.1 Domains
2.2.2 Representations
2.2.3 Learning algorithms
2.3 Heuristics and performance effectors
3. DESIGN OF EXPERIMENTS
3.1 Classical design of experiments
3.2 Contemporary design of experiments
3.3 Design of experiments for empirical algorithm analysis
4. METHODOLOGY
4.1 Sequential CART
4.1.1 CART modeling
4.1.2 Sequential CART modeling
4.1.3 Analysis of sequential CART
4.1.4 Empirical convergence criteria
4.1.5 Example: 2-D 6-hump camelback function
4.2 Kriging metamodeling
4.2.1 Kriging
4.2.2 Deterministic kriging
4.2.3 Stochastic kriging
4.2.4 Covariance function
4.2.5 Implementation
4.2.6 Analysis of kriging metamodels
5. THE MOUNTAIN CAR PROBLEM
5.1 Reinforcement learning implementation
5.2 Sequential CART
5.3 Response surface metamodeling
5.4 Discussion
6. THE TRUCK BACKER-UPPER PROBLEM
6.1 Reinforcement learning implementation
6.2 Sequential CART
6.3 Response surface metamodeling
6.4 Discussion
7. THE TANDEM TRUCK BACKER-UPPER PROBLEM
7.1 Reinforcement learning implementation
7.2 Sequential CART
7.3 Discussion
8. DISCUSSION
8.1 Reinforcement learning
8.2 Experimentation
8.3 Innovations
8.4 Future work
APPENDICES
A. Parameter effects in the game of Chung Toi
B. Design of experiments for the mountain car problem
C. Supporting tables

Christopher Gatti received his PhD in Decision Sciences and Engineering Systems from Rensselaer Polytechnic Institute (RPI). During his time at RPI, his work focused on machine learning and statistics, with applications in reinforcement learning, graph search, stem cell RNA analysis, and neuro-electrophysiological signal analysis. Prior to beginning his graduate work at RPI, he received a BSE in mechanical engineering and an MSE in biomedical engineering, both from the University of Michigan. He then continued to work at the University of Michigan for three years doing computational biomechanics focusing on the shoulder and knee. He has been a gymnast since he was a child and is currently an acrobat for Cirque du Soleil.