Datamodels: Predicting Predictions from Training Data



Andrew Ilyas (MIT)

Andrew Ilyas is a fourth-year PhD student at MIT, advised by Aleksander Madry and Constantinos Daskalakis. His research focuses on robust and reliable machine learning, with an emphasis on the ways in which (often unintended) correlations present in training data can manifest at test-time. He is supported by an Open Philanthropy Project AI Fellowship.



Short Abstract: Machine learning models tend to rely on an abundance of training data. Yet, understanding the underlying structure of this data—and models' exact dependence on it---remains a challenge. In this talk, we will present a framework for directly modeling predictions as functions of training data. This framework, given a dataset and a learning algorithm, pinpoints---at varying levels of granularity---the relationships between train and test point pairs through the lens of the corresponding model class. Even in its most basic version, our framework enables many applications, including discovering data subpopulations, quantifying model brittleness via counterfactuals, and comparing learning algorithms. Based on joint work with Sung Min Park, Logan Engstrom, Harshay Shah, Guillaume Leclerc, and Aleksander Madry.