Annotating hate speech: Three schemes at comparison
Published in Proceedings of the 6th Italian Conference on Computational Linguistics (CLIC-it 2019), 2019
Recommended citation: Fabio Poletto, Valerio Basile, Cristina Bosco, Viviana Patti, and Marco Antonio Stranisci. 2019. Annotating hate speech: Three schemes at comparison. In CEUR WORKSHOP PROCEEDINGS, vol. 2481, pp. 1-8. CEUR-WS. https://iris.unito.it/bitstream/2318/1716344/1/paper56.pdf
Annotated data are essential to train and benchmark NLP systems. The reliability of the annotation, i.e. low interannotator disagreement, is a key factor, especially when dealing with highly subjective phenomena occurring in human language. Hate speech (HS), in particular, is intrinsically nuanced and hard to fit in any fixed scale, therefore crisp classification schemes for its annotation often show their limits. We test three annotation schemes on a corpus of HS, in order to produce more reliable data.
Recommended citation: Fabio Poletto, Valerio Basile, Cristina Bosco, Viviana Patti, and Marco Antonio Stranisci. 2019. Annotating hate speech: Three schemes at comparison. In CEUR WORKSHOP PROCEEDINGS, vol. 2481, pp. 1-8. CEUR-WS.