Editors
Tirşe Erbaysal Filibeli, Tunga Güngör
Language
English
March 2025
72 pages
Free of charge
The report titled "Utilizing AI Against Hate Speech: A Guide to Annotation, Classification, and Detection" consists of two main sections: one detailing the development process of the AI tool pari, and another focusing on the labeling guide and model development stages.
In addition to the ethnic, religious, national, and sexual identities examined in the foundation’s media monitoring work since 2009, new groups targeted in agenda tracking on X were identified. The report explores how certain identity groups in Turkey are exposed to hate speech on social media, the process of collecting and labeling this data, and its transformation into an AI tool. Data collected through selected hashtags and keywords reveal the spread of hate speech against nine identity groups and the reinforcement of negative judgments about these identities. The first section describes the labeling process of the data collected for training pari. The following section, "Developing the AI Model," covers the processing of labeled data and model training.
Guidelines were created to address the subjective nature of data labeling and the limitations of the AI model, particularly for challenging elements such as sarcastic content and implicit hate speech. This framework serves both the current project and future research on hate speech.
Please review the complete report to explore the full process of collecting, labeling, and processing 16,254 tweets categorized as hate speech—as well as the hate speech categories and example tweets.
- Report title
- Utilizing AI Against Hate Speech: A Guide to Annotation, Classification, and Detection
- ISBN
- 9786057183590
- Page numbers
- 72
- Width
- 160 mm
- Height
- 225 mm
- Printing
- March 2025
- Language
- English
- Publication coordinators
- İnanç Arın, Didar Akar, Başak Can, Somaiyeh Dehghan, Elif Erol, Burak Işık, Sıla Kartal, Buket Kapısız, Nayat Karaköse, Yasemin Korkmaz, Arzucan Özgür, Nural Özel, Pelin Önal, Gökçe Uludoğan, Ayşecan Terzioğlu, Murat Tercan, İrem Topçu, Tuğba Özsoy, Berrin Yanıkoğlu, Elif Yararbaş, Umut Şen
- Project coordinators
- Başak Can, Elif Erol, Buket Kapısız, Yasemin Korkmaz, Pelin Önal, Tuğba Özsoy, Elif Yararbaş
- Editors
- Tirşe Erbaysal Filibeli, Tunga Güngör
- Translators
- Simon Charles Popay, Burcu Becermen
- Proofreader
- Neil Patrick Doherty
- Design and data visualization
- Yasemen Cemre Gürbüz
- Graphic application
- Selin Uluer
- Printed by
- Sena Ofset Ambalaj Sanayi ve Ticaret. Ltd. Şti.
Utilizing Digital Technology for Social Cohesion, Positive Messaging and Peace by Boosting Collaboration, Exchange and Solidarity project is supported by the European Union and the Friedrich Naumann Foundation.
Hrant Dink Foundation is solely responsible for the content in the publication, which does not reflect the views of supporters.
- Introduction
- The “Utilizing Digital Technology for Social Cohesion, Positive Messaging and Peace by Boosting Collaboration, Exchange and Solidarity” project
- Hate Speech Labeling Guide
- How do we decide whether discourse is hate speech?
- Labeling interface
- Determining the target group
- Determining the type of speech
- Determining the category of hate speech
- Assessing challenging examples
- Additional labeling headings
- Developing the AI model
- Data collection & annotation
- Developed AI tool for detecting and measuring hate speech
- Error analysis
- Limitations
- Conclusion
- Appendix A: Labeling interface
