Translated vs Non-Translated Method for Multilingual Hate Speech Identification in Twitter

Muhammad Okky Ibrohim, Indra Budi

Abstract


Nowadays social media is often misused to spread hate speech. Spreading hate speech is an act that needs to be handled in a special way because it can undermine or discriminate other people and cause conflict that leading to both material and immaterial losses. There are several challenges in building a hate speech identification system; one of them is identifying hate speech in multilingual scope. In this paper, we adapt and compare two methods in multilingual text classification which are translated (with and without language identification) and non-translated method for multilingual hate speech identification (including Hindi, English, and Indonesian language) using machine learning approach. We use some classification algorithms (classifiers) namely Support Vector Machine (SVM), Naive Bayes (NB), and Random Forest Decision Tree (RFDT) with word n-grams and char n-grams (character n-grams) as feature extraction. Our experiment result shows that the non-translated method gives the best result. However, the use of non-translated method needs to be reconsidered because this method needs more cost for data collection and annotation. Meanwhile, translated without language identification method give a poor result. To address this problem, we combine translated method with monolingual hate speech identification, and the experiment result shows that this approach can increase the multilingual hate speech identification performance compared to translate without language identification. This paper discusses the advantages and disadvantages for all method and the future works to enhance the performance in multilingual hate speech identification.

Keywords


social media; multilingual hate speech identification; machine learning.

Full Text:

PDF

References


Komnas HAM, Buku Saku Penanganan Ujaran Kebencian (Hate Speech). Komisi Nasional Hak Asasi Manusia, Jakarta, 2015.

G. H. Stanton, “The Rwandan genocide: Why early warning failed,†Journal of African Conflicts and Peace Studies, vol. 1(2), pp. 6–25, 2009.

Z. Waseem and D. Hovy, “Hateful symbols or hateful people? Predictive features for hate speech detection on twitter,†in Proceedings of the NAACL Student Research Workshop. San Diego, California: Association for Computational Linguistics, June 2016, pp. 88–93.

I. Alfina, R. Mulia, M. I. Fanany, and Y. Ekananta, “Hate speech detection in the Indonesian language: A dataset and preliminary study,†in International Conference on Advanced Computer Science and Information Systems (ICACSIS), 2017, pp. 233–238.

S. B. Shende and L. Deshpande, “A computational framework for detecting offensive language with support vector machine in social communities,†in 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), July 2017, pp. 1–4.

F. D. Vigna, A. Cimino, F. Dell’Orletta, M. Petrocchi, and M. Esconi, “Hate me, hate me not: Hate speech detection on facebook,†in Proceedings of the First Italian Conference on Cybersecurity (ITASEC17), 2017, pp. 86–95.

S. Tulkens, L. Hilte, E. Lodewyckx, B. Verhoeven, and W. Daelemans, “A dictionary-based approach to racism detection in dutch social media,†in First Workshop on Text Analytics for Cybersecurity and Online Safety (TACOS), 2016, pp. 11–17.

S. A. Ozel, E. Sarac, S. Akdemir, and H. Aksu, “Detection of cyberbullying on social media messages in Turkish,†in 2017 International Conference on Computer Science and Engineering, Oct 2017, pp. 366–370.

T. Davidson, D. Warmsley, M. W. Macy, and I. Weber, “Automated hate speech detection and the problem of offensive language,†in International AAAI Conference on Web and Social Media (ICWSM), 2017, pp. 512–515.

S. Agarwal and A. Sureka, “But I did not mean it!– Intent classification of racist posts on Tumblr,†in 2016 European Intelligence and Security Informatics Conference (EISIC), Aug 2016, pp. 124–127.

M. Sabou, K. Bontcheva, L. Derczynski, and A. Scharl, “Corpus annotation through crowdsourcing: Towards best practice guidelines,†in Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). European Language Resources Association (ELRA), 2014.

C. Goncalves, C. Goncalves, R. Camacho, and E. Oliveira, “The impact of pre-processing on the classification of MEDLINE documents,†in Proceedings of the 10th International Workshop on Pattern Recognition in Information Systems, 2010, pp. 53–61.

T. Baldwin and Y. Li, “An in-depth analysis of the effect of text normalization in social media.†in Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL (HLT-NAACL). The Association for Computational Linguistics, 2015, pp. 420–429.

P. C. Gaigole, L. H. Patil, and P. M. Chaudhari, “Preprocessing techniques in text categorization,†IJCA Proceedings on National Conference on Innovative Paradigms in Engineering & Technology 2013, vol. 3, no. 3, pp. 1–3, December 2013.

I. Kanaris, K. Kanaris, I. Houvardas, and E. Stamatatos, “Words vs. character n-grams for anti-spam filtering,†International Journal on Artificial Intelligence Tools, vol. 20, no. 10, pp. 1–20, 2006.

R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,†in Proceedings of the 14th International Joint Conference on Artificial Intelligence - Volume 2, ser. IJCAI’95. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1995, pp. 1137–1143.

V. Ganganwar, “An overview of classification algorithms for imbalanced datasets,†International Journal of Emerging Technology and Advanced Engineering, vol. 2(4), pp. 42–47, 2012.

M. Suzuki, N. Yamagishi, Y. Tsai, and S. Hirasawa, “Multilingual text categorization using character n-gram,†in 2008 IEEE Conference on Soft Computing in Industrial Applications, June 2008, pp. 49–54.

B. Plank, “ALL-IN-1: short text classification with one model for all languages,†CoRR, 2017.

L. Shi, R. Mihalcea, and M. Tian, “Cross-language text classification by model translation and semi-supervised learning,†in Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2010, pp. 1057–1067.

I. Alfina, S. H. Pratiwi, I. Budi, R. Mulia, and Y. Ekanata, “Detecting hate speech against religion in the Indonesian language,†Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 2018.

M. O. Ibrohim and I. Budi, “A dataset and preliminaries study for abusive language detection in indonesian social media,†Procedia Computer Science, vol. 135, pp. 222 – 229, 2018.

A. Bohra, D. Vijay, V. Singh, S. S. Akhtar, and M. Shrivastava, “A dataset of Hindi-English code-mixed social media text for hate speech detection,†in Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media. Association for Computational Linguistics, 2018, pp. 36–41.

M. Hossin and M. N. Sulaiman, “A review on evaluation metrics for data classification evaluations,†International Journal of Data Mining & Knowledge Management Process, vol. 5, pp. 1–11, 03, 2015.

P. Badjatiya, S. Gupta, M. Gupta, and V. Varma, “Deep learning for hate speech detection in tweets,†in International World Wide Web Conference Committee, 2017, p. 759760.

E. Sazany and I. Budi, “Deep Learning-Based implementation of hate speech identification on texts in Indonesian: Preliminary study,†in 2018 International Conference on Applied Information Technology and Innovation (ICAITI 2018), Padang, Indonesia, Sep. 2018.

M. O. Ibrohim, E. Sazany, and I. Budi, “Identify abusive and offensive language in indonesian twitter using deep learning approach,†Journal of Physics: Conference Series, 2018.




DOI: http://dx.doi.org/10.18517/ijaseit.9.4.8123

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development