Proj CDeepFuzz Paper Reading: Combinatorial Testing for Deep Learning Systems
Abstract
本文:DeepCT
Task: Testing DL Models with Combinatorial Testing
Method:
- 将输出值的空间离散化为区间,以便覆盖每个区间,对不同层内的神经元交互进⾏采样,并减少必须执⾏的测试输⼊的数量。
- a set of coverage criteria: Neuron-activation configuration, t-way combination sparse coverage, t-way combination dense coverage, (p, t)-completeness coverage
- LP constraint solving basd test generation
实验:
数据集: MNIST, DNN1, DNN2
Discussion:
- 在实际的深度学习系统中神经元的数量通常⾮常⼤,这会极⼤地阻碍基于约束的测试⽣成技术。
- Random Testing does not provide confidence when local-robustness cannot be detected.