Abstract

A popular method for Neural Architecture Search (NAS) is based on growing networks via small local changes to the network's architecture called network morphisms. These methods start with a small seed network and progressively grow the network by adding new neurons in an automated way. However, it remains a challenge to efficiently determine which parts of the network are best to grow. Here we propose a NAS method for growing a network by using a Gauss-Newton approximation of the loss function to efficiently learn and evaluate candidate network morphisms. We compare our method with state of the art NAS methods for CIFAR-10 and CIFAR-100 classification tasks, and conclude our method learns similar quality or better architectures at a smaller computational cost.

Metadata

publication

arXiv preprint arXiv:2411.05855, 2024

year

2024

publication date

2024/11/7

authors

Neal Lawton, Aram Galstyan, Greg Ver Steeg

link

https://arxiv.org/abs/2411.05855

resource_link

https://arxiv.org/pdf/2411.05855

journal

arXiv preprint arXiv:2411.05855