Adversarial machine learning in the real-world: assessing and improving model robustness in domain-constrained data space



Maxime Cordy (University of Luxembourg)

Maxime Cordy is a Research Scientist at the Interdisciplinary Center for Security, Reliability and Trust (SnT), University of Luxembourg, in the domain of Artificial Intelligence (AI) and Software Engineering (SE), with a focus on security and quality assurance for machine learning, software verification and testing, and the engineering of data-intensive systems. He has published 80+ peer review papers in these areas. He is one of the four permanent scientists of the SnT’s SerVal group (SEcurity, Reasoning and VALidation). His research is inspired from and applies to several industry partners, mostly from the financial technology and smart energy sectors. He is deeply engaged in making Society benefit from results and technologies produced by research through the founding of a spin-off company and the leadership of private-public partnership projects at SnT. He has worked as a program committee member and reviewer for top-tier AI and SE conferences incl. IJCAI, ECCV, NeurIPS, ESEC/FSE, PLDI, ISSTA, CAISE, etc. He is distinguished reviewer board member of TOSEM and regular reviewer for other top-tier SE journals.



Short Abstract: Adversarial attacks are considered as one of the most critical security threats for Machine Learning (ML). In order to enable the secure deployment of ML models in the real world, it is essential to properly assess their robustness to adversarial attacks and develop means to make models more robust. Traditional adversarial attacks were mostly designed for image recognition and assume that every image pixel can be modified independently to its full range of values. In many domains, however, these attacks fail to consider that only specific perturbations could occur in practice due to the hard domain constraints that delimit the set of valid inputs. Because of this, they almost-always produce examples that are not feasible (i.e. could not exist in the real world). As a result, research has developed real-world adversarial attacks that either manipulate real objects through a series of problem-space transformations (i.e. problem-space attacks) or generate feature perturbations that satisfy predefined domain constraints (i.e. constrained feature space attacks). In this talk, we will review the scientific literature on these attacks and report on our experience in applying them to real-world cases.