Geometric Properties of Data Representations in Deep Neural Networks

The research paper explores the role of intrinsic dimensionality in deep neural networks, specifically focusing on the geometric properties of data representations. It investigates how the intrinsic dimensionality changes across layers of neural networks and its impact on generalization performance.

Key takeaways for engineers/specialists include the discovery of a 'hunchback' shape for intrinsic dimensionality across layers of Convolutional Neural Networks (CNNs), with a strong correlation between the ID in the final layer and performance on unseen data. The findings indicate that deep networks compress information into low-dimensional manifolds to generalize effectively, involving non-linear transformations for achieving linearly separable representations.

Read full paper: https://arxiv.org/abs/1905.12784

Tags: Deep Learning, Machine Learning, Explainable AI