Cargando…

Investigating toxicity changes of cross-community redditors from 2 billion posts and comments

This research investigates changes in online behavior of users who publish in multiple communities on Reddit by measuring their toxicity at two levels. With the aid of crowdsourcing, we built a labeled dataset of 10,083 Reddit comments, then used the dataset to train and fine-tune a Bidirectional En...

Descripción completa

Detalles Bibliográficos
Autores principales: Almerekhi, Hind, Kwak, Haewoon, Jansen, Bernard J.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: PeerJ Inc. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9455283/
https://www.ncbi.nlm.nih.gov/pubmed/36092019
http://dx.doi.org/10.7717/peerj-cs.1059
_version_ 1784785548844466176
author Almerekhi, Hind
Kwak, Haewoon
Jansen, Bernard J.
author_facet Almerekhi, Hind
Kwak, Haewoon
Jansen, Bernard J.
author_sort Almerekhi, Hind
collection PubMed
description This research investigates changes in online behavior of users who publish in multiple communities on Reddit by measuring their toxicity at two levels. With the aid of crowdsourcing, we built a labeled dataset of 10,083 Reddit comments, then used the dataset to train and fine-tune a Bidirectional Encoder Representations from Transformers (BERT) neural network model. The model predicted the toxicity levels of 87,376,912 posts from 577,835 users and 2,205,581,786 comments from 890,913 users on Reddit over 16 years, from 2005 to 2020. This study utilized the toxicity levels of user content to identify toxicity changes by the user within the same community, across multiple communities, and over time. As for the toxicity detection performance, the BERT model achieved a 91.27% classification accuracy and an area under the receiver operating characteristic curve (AUC) score of 0.963 and outperformed several baseline machine learning and neural network models. The user behavior toxicity analysis showed that 16.11% of users publish toxic posts, and 13.28% of users publish toxic comments. However, results showed that 30.68% of users publishing posts and 81.67% of users publishing comments exhibit changes in their toxicity across different communities, indicating that users adapt their behavior to the communities’ norms. Furthermore, time series analysis with the Granger causality test of the volume of links and toxicity in user content showed that toxic comments are Granger caused by links in comments.
format Online
Article
Text
id pubmed-9455283
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher PeerJ Inc.
record_format MEDLINE/PubMed
spelling pubmed-94552832022-09-09 Investigating toxicity changes of cross-community redditors from 2 billion posts and comments Almerekhi, Hind Kwak, Haewoon Jansen, Bernard J. PeerJ Comput Sci Data Mining and Machine Learning This research investigates changes in online behavior of users who publish in multiple communities on Reddit by measuring their toxicity at two levels. With the aid of crowdsourcing, we built a labeled dataset of 10,083 Reddit comments, then used the dataset to train and fine-tune a Bidirectional Encoder Representations from Transformers (BERT) neural network model. The model predicted the toxicity levels of 87,376,912 posts from 577,835 users and 2,205,581,786 comments from 890,913 users on Reddit over 16 years, from 2005 to 2020. This study utilized the toxicity levels of user content to identify toxicity changes by the user within the same community, across multiple communities, and over time. As for the toxicity detection performance, the BERT model achieved a 91.27% classification accuracy and an area under the receiver operating characteristic curve (AUC) score of 0.963 and outperformed several baseline machine learning and neural network models. The user behavior toxicity analysis showed that 16.11% of users publish toxic posts, and 13.28% of users publish toxic comments. However, results showed that 30.68% of users publishing posts and 81.67% of users publishing comments exhibit changes in their toxicity across different communities, indicating that users adapt their behavior to the communities’ norms. Furthermore, time series analysis with the Granger causality test of the volume of links and toxicity in user content showed that toxic comments are Granger caused by links in comments. PeerJ Inc. 2022-08-18 /pmc/articles/PMC9455283/ /pubmed/36092019 http://dx.doi.org/10.7717/peerj-cs.1059 Text en ©2022 Almerekhi et al. https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited.
spellingShingle Data Mining and Machine Learning
Almerekhi, Hind
Kwak, Haewoon
Jansen, Bernard J.
Investigating toxicity changes of cross-community redditors from 2 billion posts and comments
title Investigating toxicity changes of cross-community redditors from 2 billion posts and comments
title_full Investigating toxicity changes of cross-community redditors from 2 billion posts and comments
title_fullStr Investigating toxicity changes of cross-community redditors from 2 billion posts and comments
title_full_unstemmed Investigating toxicity changes of cross-community redditors from 2 billion posts and comments
title_short Investigating toxicity changes of cross-community redditors from 2 billion posts and comments
title_sort investigating toxicity changes of cross-community redditors from 2 billion posts and comments
topic Data Mining and Machine Learning
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9455283/
https://www.ncbi.nlm.nih.gov/pubmed/36092019
http://dx.doi.org/10.7717/peerj-cs.1059
work_keys_str_mv AT almerekhihind investigatingtoxicitychangesofcrosscommunityredditorsfrom2billionpostsandcomments
AT kwakhaewoon investigatingtoxicitychangesofcrosscommunityredditorsfrom2billionpostsandcomments
AT jansenbernardj investigatingtoxicitychangesofcrosscommunityredditorsfrom2billionpostsandcomments