Proj CDeepFuzz Paper Reading: Automatic differentiation in ML: Where we are and where we should be going

Abstract

本文：Myia
Github: https://github.com/mila-iqia/myia
Task: Review automatic differentiation for array programming in machine learning, propose a new graph-based IR to support fully-general AD, a proof-of-concept compiler toolchain Myia
包括：

1. operator overloading
2. source transformation
3. a new graph-based IR to support fully-general AD, supports function calls, higher-order functions and recursion

• 特点：
  1. 可以使用closure这一点使得PyTorch能够使用source transformation来完成AD而无需tape
  2. 可以使用函数语言的工具对编译器进行优化
  3. 允许高阶导数

1. Intro

2. Background and prior work

2.1 Automatic differentiation

2.1.1 Operator overloading

2.1.2 Source transformation

2.2 Dataflow programming

2.3 Programming languages and compilers

2.3.1 Python

3. Graph-based direct intermediate representation

3.1 IR specification

3.2 Source transformation

4. Myia

4.1 Python front end

4.2 Type inference

4.3 Optimization

5. Conclusion