ML Safety



Dan Hendrycks (UC Berkeley)

I recently received my PhD from UC Berkeley where I was advised by Dawn Song and Jacob Steinhardt. I am now the director of the Center for AI Safety. I am interested in ML Safety. In 2018 I received my BS from UChicago. My research is supported by the NSF GRFP and the Open Philanthropy AI Fellowship. I helped contribute the GELU activation function (the most-used activation in state-of-the-art models including BERT, GPT, Vision Transformers, etc.), the out-of-distribution detection baseline, and distribution shift benchmarks.



Short Abstract: Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technologies, safety for ML should be a leading research priority. In response to emerging safety challenges in ML, such as those introduced by recent large-scale models, we provide a new roadmap for ML Safety and refine the technical problems that the field needs to address. We present four problems ready for research, namely withstanding hazards ("Robustness"), identifying hazards ("Monitoring"), reducing inherent model hazards ("Alignment"), and reducing systemic hazards ("Systemic Safety"). Throughout, we clarify each problem's motivation and provide concrete research directions.