Speaker: David Schwab, City University of New York Abstract: Lossy compression, in which disentangling relevant and irrelevant information plays a critical role, is a central concept in both physics and machine learning. In this talk, I will describe a line of work that begins with compression in the context of the renormalization group from statistical physics and then explores the role compression plays in modern neural networks. I will first describe an early connection we found between coarse graining in physics and representation learning with neural networks. I will then turn my focus to the information bottleneck (IB), an information theoretic formulation of relevance-aware compression. While IB has long been considered an appealing framework in principle, applying it to real systems, and neural networks in particular, is challenging. I will describe approaches my group has taken to modify the IB objective in order to broaden its applicability, with a particular focus on decodable information and its relation to learning optimal supervised representations.Â