CB-Fake: A multimodal deep learning framework for automatic fake news detection using capsule neural network and BERT

The progressive growth of today’s digital world has made news spread exponentially faster on social media platforms like Twitter, Facebook, and Weibo. Unverified news is often disseminated in the form of multimedia content like text, picture, audio, or video. The dissemination of such false news deceives the public and leads to protests and creates troubles for the public and the government. Hence, it is essential to verify the authenticity of the news at an early stage before sharing it with the public. Earlier fake news detection (FND) approaches combined textual and visual features, but the semantic correlations between words were not addressed and many informative visual features were lost. To address this issue, an automated fake news detection system is proposed, which fuses textual and visual features to create a multimodal feature vector with high information content. The proposed work incorporates the bidirectional encoder representations from transformers (BERT) model to extract the textual features, which preserves the semantic relationships between words. Unlike the convolutional neural network (CNN), the proposed capsule neural network (CapsNet) model captures the most informative visual features from an image. These features are combined to obtain a richer data representation that helps to determine whether the news is fake or real. We investigated the performance of our model against different baselines using two publicly accessible datasets, Politifact and Gossipcop. Our proposed model achieves significantly better classification accuracy of 93% and 92% for the Politifact and Gossipcop datasets, respectively, compared to 84.6% and 85.6% for the SpotFake+ model.