Proj CDeepFuzz Paper Reading: Audee: Automated testing for deep learning frameworks

Abstract

本文：Audee
Github: https://github.com/jikechao/Audee
Task: test DL frameworks and localizing bugs
Method:

1. search-based
2. 3 different mutation strategies by exploring combinations of model structures, parameters, weights and inputs
3. Differential Testing for logical bugs: cross-reference check across multiple frameworks
4. NaN errors: a heuristic-based approach to generate DNN that tends to output outliers
5. localizing bugs: causal-testing to localize layers and parameters
   Targets bugs: logical bugs, crashes and NaN errors

实验：
对象：Tensorflow, PyTorch, CNTK, Theano
效果：

1. cover 25 widely-used APIs
2. effective in detecting inconsistencies, crashes and NaN errors
3. 26 bugs, 7 confirmed