论文阅读之Regularized Evolution for Image Classifier Architecture Search

Regularized Evolution for Image Classifier Architecture Search这篇文章实际上解决的是针对NAS这个问题用EA算法的缺陷，一个很强的motivation是NAS是需要对搜到的网络进行重新训练的，但是即使是一个比较差的网络，在训练的过程中也可能因为偶然因素变得很好，如果是传统的EA算法，在种群中保留最好的，这样可能会导致的问题是这个因为偶然性表现比较好的会作为一个精英并且一直延续下去，导致大量的没有意义的搜索，减少了exploration的能力。作者一个很大的贡献就是改进了EA算法，如下所示：

如果是传统的EA算法的话，只是移除了最老的，而不是最差的，作者说这样可能为有上面所说的问题，原文是：

We can speculate that aging may help navigate the training noise in evolutionary experiments, as follows. Noisy training means that models may sometimes reach high accuracy just by luck. In non-aging evolution (NAE, i.e. standard tournament selection), such lucky models may remain in the population for a long time—even for the whole experiment. One lucky model, therefore, can produce many children, cause reducing exploration. Under aging evolution (AE), on the other hand, all models have a short lifespan, so the population is wholly renewed frequently, leading to more diversity and more exploration. In AE, because models die quickly, the only way an architecture can remain in the population for a long time is by being passed down from parent to child through the generations. Each time an architecture is inherited it must be re-trained.

作者的意思比较明了，值得注意的是最后返回所有val arch上最好的结果。
还有就是，上面这张算法图给我们展示了标准的tournament selection的EA算法是一个怎么样的流程。比如一开始初始化种群，然后随机选取多少个，成为tournament size，从中选择最好的，对其进行变异，然后淘汰最老的或者最差的。
当然作者也试验了一下他们搜索的结果，对比了一下引入age和不引入age的实验结果

绝大多数结果都是比较好的