Offensive Language and Hate Speech Detection Using Transformers and Ensemble Learning Approaches

Abstract

Social networks play a vital role in facilitating communication and information sharing. However, these platforms are also witnessing a growing prevalence of hate content, which can pose a major threat to individuals and entire communities. In this paper, we propose a new method that addresses the problem of offensive language and hate speech detection using seven transformer models, including BERT, and six ensemble learning strategies (Majority Voting, Averaging, Highest-sum, Stacking, Boosting and Bagging). Specifically, a fine-tuning process is run for each pre-trained model on hate speech detection downstream task. Subsequently, various ensemble learning techniques are applied by combining the predictions of individual models in order to improve overall performance. Extensive experiments have been conducted on the publicly available Davidson-dataset to assess the performance of our proposed method. Evaluation demonstrates promising results in terms of various evaluation metrics, outperforming competitive state-of-the-art baselines.