Proj CDeepFuzz Paper Reading: Automatic Differentiation in Machine Learning: a Survey

Abstract

背景：

  1. AD涉及computational fluid dynamics, atmospheric sciences, engineering design optimization
  2. AD与DL长时间并不相互交流，直到dynamic computational graphs和differentiable programming出现

本文：

  1. survey the intersection of AD and ML
  2. cover AD有直接影响的applications
  3. define the main differentiation 及其内部关系

1. Intro

2. What AD Is Not

2.1 AD Is Not Numerical Differentiation

2.2 AD is Not Symbolic Differentiation

3. AD and Its Main Modes

3.1 Forward Mode

3.1.1 Dual Numbers

3.2 Reverse Mode

3.3 Origins of AD and Backpropagation

4. AD and Machine Learning

4.1 Gradient-Based Optimization

4.2 Neural Networks, Deep Learning, Differentiable Programming

4.3 Computer Vision

4.4 Natural Language Processing

4.5 Probabilistic Modeling and Inference

5. Implementations

5.1 Elemental Libraries

5.2 Compilers and Source Code Transformation

5.3 Operator Overloading

6. Conclusions

posted @ 2023-08-29 16:03  雪溯  阅读(6)  评论(0)  编辑  收藏  举报