Cargando…
Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction
Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7931871/ https://www.ncbi.nlm.nih.gov/pubmed/33693421 http://dx.doi.org/10.3389/fdata.2020.590296 |
_version_ | 1783660371649757184 |
---|---|
author | Majumdar, Puspita Chhabra, Saheb Singh, Richa Vatsa, Mayank |
author_facet | Majumdar, Puspita Chhabra, Saheb Singh, Richa Vatsa, Mayank |
author_sort | Majumdar, Puspita |
collection | PubMed |
description | Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these vulnerabilities, bias has been an Achilles’ heel problem. Many applications such as face recognition and language translation have shown high levels of bias in the systems towards particular demographic sub-groups. Unbalanced representation of these sub-groups in the training data is one of the primary reasons of biased behavior. To address this important challenge, we propose a two-fold contribution: a bias estimation metric termed as Precise Subgroup Equivalence to jointly measure the bias in model prediction and the overall model performance. Secondly, we propose a novel bias mitigation algorithm which is inspired from adversarial perturbation and uses the PSE metric. The mitigation algorithm learns a single uniform perturbation termed as Subgroup Invariant Perturbation which is added to the input dataset to generate a transformed dataset. The transformed dataset, when given as input to the pre-trained model reduces the bias in model prediction. Multiple experiments performed on four publicly available face datasets showcase the effectiveness of the proposed algorithm for race and gender prediction. |
format | Online Article Text |
id | pubmed-7931871 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-79318712021-03-09 Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction Majumdar, Puspita Chhabra, Saheb Singh, Richa Vatsa, Mayank Front Big Data Big Data Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these vulnerabilities, bias has been an Achilles’ heel problem. Many applications such as face recognition and language translation have shown high levels of bias in the systems towards particular demographic sub-groups. Unbalanced representation of these sub-groups in the training data is one of the primary reasons of biased behavior. To address this important challenge, we propose a two-fold contribution: a bias estimation metric termed as Precise Subgroup Equivalence to jointly measure the bias in model prediction and the overall model performance. Secondly, we propose a novel bias mitigation algorithm which is inspired from adversarial perturbation and uses the PSE metric. The mitigation algorithm learns a single uniform perturbation termed as Subgroup Invariant Perturbation which is added to the input dataset to generate a transformed dataset. The transformed dataset, when given as input to the pre-trained model reduces the bias in model prediction. Multiple experiments performed on four publicly available face datasets showcase the effectiveness of the proposed algorithm for race and gender prediction. Frontiers Media S.A. 2021-02-18 /pmc/articles/PMC7931871/ /pubmed/33693421 http://dx.doi.org/10.3389/fdata.2020.590296 Text en Copyright © 2021 Majumdar, Chhabra, Singh and Vatsa. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Big Data Majumdar, Puspita Chhabra, Saheb Singh, Richa Vatsa, Mayank Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction |
title | Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction |
title_full | Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction |
title_fullStr | Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction |
title_full_unstemmed | Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction |
title_short | Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction |
title_sort | subgroup invariant perturbation for unbiased pre-trained model prediction |
topic | Big Data |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7931871/ https://www.ncbi.nlm.nih.gov/pubmed/33693421 http://dx.doi.org/10.3389/fdata.2020.590296 |
work_keys_str_mv | AT majumdarpuspita subgroupinvariantperturbationforunbiasedpretrainedmodelprediction AT chhabrasaheb subgroupinvariantperturbationforunbiasedpretrainedmodelprediction AT singhricha subgroupinvariantperturbationforunbiasedpretrainedmodelprediction AT vatsamayank subgroupinvariantperturbationforunbiasedpretrainedmodelprediction |