Cargando…

Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction

Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these...

Descripción completa

Detalles Bibliográficos
Autores principales: Majumdar, Puspita, Chhabra, Saheb, Singh, Richa, Vatsa, Mayank
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7931871/
https://www.ncbi.nlm.nih.gov/pubmed/33693421
http://dx.doi.org/10.3389/fdata.2020.590296
_version_ 1783660371649757184
author Majumdar, Puspita
Chhabra, Saheb
Singh, Richa
Vatsa, Mayank
author_facet Majumdar, Puspita
Chhabra, Saheb
Singh, Richa
Vatsa, Mayank
author_sort Majumdar, Puspita
collection PubMed
description Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these vulnerabilities, bias has been an Achilles’ heel problem. Many applications such as face recognition and language translation have shown high levels of bias in the systems towards particular demographic sub-groups. Unbalanced representation of these sub-groups in the training data is one of the primary reasons of biased behavior. To address this important challenge, we propose a two-fold contribution: a bias estimation metric termed as Precise Subgroup Equivalence to jointly measure the bias in model prediction and the overall model performance. Secondly, we propose a novel bias mitigation algorithm which is inspired from adversarial perturbation and uses the PSE metric. The mitigation algorithm learns a single uniform perturbation termed as Subgroup Invariant Perturbation which is added to the input dataset to generate a transformed dataset. The transformed dataset, when given as input to the pre-trained model reduces the bias in model prediction. Multiple experiments performed on four publicly available face datasets showcase the effectiveness of the proposed algorithm for race and gender prediction.
format Online
Article
Text
id pubmed-7931871
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-79318712021-03-09 Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction Majumdar, Puspita Chhabra, Saheb Singh, Richa Vatsa, Mayank Front Big Data Big Data Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these vulnerabilities, bias has been an Achilles’ heel problem. Many applications such as face recognition and language translation have shown high levels of bias in the systems towards particular demographic sub-groups. Unbalanced representation of these sub-groups in the training data is one of the primary reasons of biased behavior. To address this important challenge, we propose a two-fold contribution: a bias estimation metric termed as Precise Subgroup Equivalence to jointly measure the bias in model prediction and the overall model performance. Secondly, we propose a novel bias mitigation algorithm which is inspired from adversarial perturbation and uses the PSE metric. The mitigation algorithm learns a single uniform perturbation termed as Subgroup Invariant Perturbation which is added to the input dataset to generate a transformed dataset. The transformed dataset, when given as input to the pre-trained model reduces the bias in model prediction. Multiple experiments performed on four publicly available face datasets showcase the effectiveness of the proposed algorithm for race and gender prediction. Frontiers Media S.A. 2021-02-18 /pmc/articles/PMC7931871/ /pubmed/33693421 http://dx.doi.org/10.3389/fdata.2020.590296 Text en Copyright © 2021 Majumdar, Chhabra, Singh and Vatsa. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Big Data
Majumdar, Puspita
Chhabra, Saheb
Singh, Richa
Vatsa, Mayank
Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
title Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
title_full Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
title_fullStr Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
title_full_unstemmed Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
title_short Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
title_sort subgroup invariant perturbation for unbiased pre-trained model prediction
topic Big Data
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7931871/
https://www.ncbi.nlm.nih.gov/pubmed/33693421
http://dx.doi.org/10.3389/fdata.2020.590296
work_keys_str_mv AT majumdarpuspita subgroupinvariantperturbationforunbiasedpretrainedmodelprediction
AT chhabrasaheb subgroupinvariantperturbationforunbiasedpretrainedmodelprediction
AT singhricha subgroupinvariantperturbationforunbiasedpretrainedmodelprediction
AT vatsamayank subgroupinvariantperturbationforunbiasedpretrainedmodelprediction