Proj CDeepFuzz Paper Reading: Audee: Automated testing for deep learning frameworks
Abstract
本文:Audee
Github: https://github.com/jikechao/Audee
Task: test DL frameworks and localizing bugs
Method:
- search-based
- 3 different mutation strategies by exploring combinations of model structures, parameters, weights and inputs
- Differential Testing for logical bugs: cross-reference check across multiple frameworks
- NaN errors: a heuristic-based approach to generate DNN that tends to output outliers
- localizing bugs: causal-testing to localize layers and parameters
Targets bugs: logical bugs, crashes and NaN errors
实验:
对象:Tensorflow, PyTorch, CNTK, Theano
效果:
- cover 25 widely-used APIs
- effective in detecting inconsistencies, crashes and NaN errors
- 26 bugs, 7 confirmed