back to list

Defenses against Data Poisoning Attacks on Private Machine Learning Training

Bachelor Thesis

Motivation

Attackers have great incentives to manipulate the results and models generated by machine learning algorithms as it is widely used for automated decision-making. In “poisoning attacks,” for instance, an attacker injects modified training data into the pipeline for training data, causing the final model to produce targeted misclassification on particular inputs. Poisoning attacks have recently improved, becoming more effective [1, 2, 3] and realistic [4, 5, 6]. Data poisoning attacks were ranked as the greatest concerning danger to industry machine learning systems in a recent study of industrial practitioners [7].

Secure multi-party computation (MPC) allows multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, by design, MPC protocols faithfully compute the training functionality, where the adversarial ML community can be tampered with in poisoning attacks [8].

In this thesis, we will perform the first systematic study of poisoning attacks and their countermeasures for private machine learning training. In poisoning attacks, attackers deliberately influence the training data to manipulate the results of a predictive model.

Goal

The student will, at the first stage, study and analyze different data poisoning attacks and defenses. Then he will design a new MPC-friendly defense method that is highly resilient against all poisoning attacks and implement it in the MPC framework CRYPTEN [9]. To do so, the combination of Arithmetic Sharing and Secret Sharing should be used. The student should demonstrate the defense effectiveness on a range of different datasets and models in private machine learning. In the end, this defense will be integrated with FL libraries like SionFL [10].

Requirements

  • High motivation for challenging engineering tasks
  • At least basic knowledge of secure two party computation and ML algorithms
  • Good programming skills in Python, Pytorch
  • High motivation + ability to work independently
  • Knowledge of the English language, Git, LaTeX, etc. goes without saying

References

  • [1] Xu Xiaojun, Qi Wang, Huichen Li, Nikita Borisov, Carl A. Gunter, and Bo Li. Detecting AI trojans using meta neural analysis. In IEEE SP, 2021.
  • [2] Huang Kunzhe, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. In ICLR, 2022.
  • [3] Qi Xiangyu, Tinghao Xie, Saeed Mahloujifar, and Prateek Mittal. Fight Poison with Poison: Detecting Backdoor Poison Samples via Decoupling Benign Correlations. arXiv preprint arXiv:2205.13616, 2022.
  • [4] Jagielski Matthew, Alina Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, and Bo Li. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In IEEE SP, 2018.
  • [5] Wang Bolun, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In IEEE SP, 2019.
  • [6] Tao, Guanhong, Yingqi Liu, Guangyu Shen, Qiuling Xu, Shengwei An, Zhuo Zhang, and Xiangyu Zhang. Model orthogonalization: Class distance hardening in neural networks for better security. In IEEE SP, 2022.
  • [7] Kumar Ram Shankar Siva, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. Adversarial machine learning-industry perspectives. In IEEE Security and Privacy Workshops (SPW), 2020.
  • [8] Chaudhari Harsh, Matthew Jagielski, and Alina Oprea. SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning. In IEEE Conference on Secure and Trustworthy Machine Learning, 2023.
  • [9] Knott Brian, Shobha Venkataraman, Awni Hannun, Shubho Sengupta, Mark Ibrahim, and Laurens van der Maaten. Crypten: Secure multi-party computation meets machine learning. In NeurIPS, 2021.
  • [10] Ben-Itzhak Yaniv, Helen Möllering, Benny Pinkas, Thomas Schneider, Ajith Suresh, Oleksandr Tkachenko, Shay Vargaftik, Christian Weinert, Hossein Yalame, and Avishay Yanai. ScionFL: Secure Quantized Aggregation for Federated Learning. arXiv preprint arXiv:2210.07376, 2022.

Supervisors