Encoding Involutory Invariance in Neural Networks


Anwesh Bhattacharya, Marios Mattheakis, and Pavlos Protopapas. Submitted. “Encoding Involutory Invariance in Neural Networks.” In . Publisher's Version Copy at https://j.mp/3qs4tgJ
2106.12891.pdf1.54 MB


In certain situations, Neural Networks (NN) are trained upon data that obey underlying physical symmetries. However, it is not guaranteed that NNs will obey the underlying symmetry unless embedded in the network structure. In this work, we explore a special kind of symmetry where functions are invariant with respect to involutory linear/affine transformations up to parity p = ±1. We develop mathe- matical theorems and propose NN architectures that ensure invariance and universal approximation properties. Numerical experiments indicate that the proposed mod- els outperform baseline networks while respecting the imposed symmetry. An adaption of our technique to convolutional NN classification tasks for datasets with inherent horizontal/vertical reflection symmetry has also been proposed.

Last updated on 08/27/2021