Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs

Posted 4 months ago
Publication:
arXiv
arXiv® is a curated research-sharing platform open to anyone. It publishes scientific papers in the fields of mathematics, physics, astronomy, electrical engineering, computer science, quantitative biology, statistics, mathematical finance, and economics, which can be accessed online..
Publication Year:
2020
Link:
Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
Authors:
Ari Morcos Matthew L. Leavitt
Category:
Computers & Internet

The properties of individual neurons are often analyzed in order to understand the biological and artificial neural networks in which they're embedded. Class selectivity-typically defined as how different a neuron's responses are across different classes of stimuli or data samples is commonly used for this purpose. However, it remains an open question whether it is necessary and/or sufficient for deep neural networks (DNNs) to learn class selectivity in individual units.

We investigated the causal impact of class selectivity on network function by directly regularizing for or against class selectivity. Using this regularizer to reduce class selectivity across units in convolutional neural networks increased test accuracy by over 2% for ResNet18 trained on Tiny ImageNet.

Read the whole paper here. Download this paper here.