[paper reading][JMLR 2019] Bayesian Optimization for Policy Search via Online-Offline Experimentation

目录

• 1 Introduction
• 2 Empirical Context and the Simulator
• 3 Response Surface Modeling with the Gaussian Process
  • 3.1 The Multi-Task Gaussian Process
• 4 Multi-Task Response Surface Models with the Simulator

• JMLR 2019
• https://www.jmlr.org/papers/volume20/18-225/18-225.pdf
• combine online and offline
• multi-task GP, kernel for multi-task

1 Introduction

• A/B: simple space
• BO: large spaces
• data: expensive
• offline systems, "sim", bias

2 Empirical Context and the Simulator

• very biased
  

3 Response Surface Modeling with the Gaussian Process

• points that are nearby in space are given high covariance. The degree of smoothness depends on the kernel variance and lengthscales

3.1 The Multi-Task Gaussian Process

• offline and online tasks
• the same spatial kernel
• separability between the task covariance and the spatial covariance
• "linear combination" of 2 latent functions
• extreme, 0, irrelevant, independent GPs that have the same cov function
• decomposition, incomplete, \(O(DP)\)
  • 2 tasks: not necessary
• more complex?
• now: shared kernel, leverage, √

4 Multi-Task Response Surface Models with the Simulator