Overview of RefutES at IberLEF 2024Automatic Generation of Counter Speech in Spanish

  1. Vallecillo-Rodríguez, María Estrella
  2. Cantero-Romero, María Victoria
  3. Cabrera-de-Castro, Isabel
  4. Ureña-López, Luis Alfonso
  5. Montejo-Ráez, Arturo
  6. Martín-Valdivia, María Teresa
Revista:
Procesamiento del lenguaje natural

ISSN: 1135-5948

Ano de publicación: 2024

Número: 73

Páxinas: 449-459

Tipo: Artigo

DIALNET GOOGLE SCHOLAR lock_openAcceso aberto editor

Outras publicacións en: Procesamiento del lenguaje natural

Resumo

Este artículo presenta la tarea RefutES 2024, organizada en IberLEF 2024 junto a la 40ª Conferencia Internacional de la Sociedad Española para el Procesamiento del Lenguaje Natural (SEPLN 2024). El objetivo principal de RefutES es promover la investigación sobre la generación automática de contranarrativas en español. La generación de contranarrativas es una nueva estrategia desarrollada para combatir los mensajes de odio en redes sociales que consiste en la generación de una respuesta que niega el mensaje offensivo. En esta tarea compartida, los participantes deben generar una respuesta a mensajes de odio que están dirigidos a diferentes colectivos en español. Esta respuesta debe de ser argumentada, respetuosa, no ofensiva y contender información específica y veraz. Además, los participantes tienen que presentar mediciones de las emisiones de carbono de sus sistemas, haciendo hincapié en la necesidad de prácticas de PNL sostenibles. En esta primera edición, un total de 6 equipos se registration en la tarea, 1 subió los resultados de las ejecuciones realizadas sobre los datos de test y 1 escribió el artículo con la descripción de su sistema.

Referencias bibliográficas

  • Benesch, S. 2014. Countering dangerous speech: New ideas for genocide prevention. URL: https://ssrn.com/abstract=3686876.
  • Bengoetxea, J., Y.-L. Chung, M. Guerini, and R. Agerri. 2024. Basque and Spanish counter narrative generation: Data creation and evaluation. In N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2132–2141, Torino, Italia, May. ELRA and ICCL. URL: https://aclanthology.org/2024.lrecmain. 192.
  • Bonaldi, H., Y.-L. Chung, G. Abercrombie, and M. Guerini. 2024. Nlp for counter-speech against hate: A survey and how-to guide.
  • Chiruzzo, L., S. M. Jiménez-Zafra, and F. Rangel. 2024. Overview of IberLEF 2024: Natural Language Processing Challenges for Spanish and other Iberian Languages. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2019. Unsupervised crosslingual representation learning at scale. CoRR, abs/1911.02116.
  • Courty, B., V. Schmidt, S. Luccioni, Goyal-Kamal, MarionCoutarel, B. Feld, J. Lecourt, LiamConnell, A. Saboni, Inimaz, supatomic, M. Léval, L. Blanche, A. Cruveiller, ouminasara, F. Zhao, A. Joshi, A. Bogroff, H. de Lavoreille, N. Laskaris, E. Abati, D. Blank, Z. Wang, A. Catovic, M. Alencon, M. Stęchły, C. Bauer, Lucas-Otavio, JPW, and MinervaBooks. 2024. mlco2/codecarbon: v2.4.1, May.
  • Dettmers, T., A. Pagnoni, A. Holtzman, and L. Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs, May. arXiv:2305.14314 [cs].
  • Fanton, M., H. Bonaldi, S. S. Tekiroğ lu, and M. Guerini. 2021. Human-in-the-loop for data collection: a multi-target counter narrative dataset to fight online hate speech. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics.
  • Furman, D., P. Torres, J. Rodríguez, D. Letzen, M. Martinez, and L. Alemany. 2023. High-quality argumentative information in low resources approaches improve counter-narrative generation. In H. Bouamor, J. Pino, and K. Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2942–2956, Singapore, December. Association for Computational Linguistics.
  • Halim, S. M., S. Irtiza, Y. Hu, L. Khan, and B. Thuraisingham. 2023. Wokegpt: Improving counterspeech generation against online hate speech by intelligently augmenting datasets using a novel metric. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1–10. IEEE.
  • Mathew, B., H. Tharad, S. Rajgaria, P. Singhania, S. K. Maity, P. Goyal, and A. Mukherjee. 2018. Thou shalt not hate: Countering online hate speech. In International Conference on Web and Social Media.
  • Qian, J., A. Bethke, Y. Liu, E. Belding, and W. Y. Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pages 4755–4764, Hong Kong, China, November. Association for Computational Linguistics.
  • Schieb, C. and M. Preuss. 2016. Governing hate speech by means of counterspeech on facebook. In 66th ica annual conference, at fukuoka, japan, pages 1–23.
  • Tekiroglu, S., H. Bonaldi, M. Fanton, and M. Guerini. 2022. Using pre-trained language models for producing counter narratives against hate speech: a comparative study. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3099–3114.
  • Touvron, H., L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models, July. arXiv:2307.09288 [cs].
  • Vallecillo-Rodríguez, M.-E., M.-V. Cantero-Romero, I. Cabrera-De-Castro, A. Montejo-Ráez, and M.-T. Martín-Valdivia. 2024. CONAN-MT-SP: A Spanish corpus for counternarrative using GPT models. In N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 3677–3688, Torino, Italy, May. ELRA and ICCL.
  • Vallecillo-Rodríguez, M. E., A. Montejo-Raéz, and M. T. Martín-Valdivia. 2023. Automatic counter-narrative generation for hate speech in spanish. Procesamiento del Lenguaje Natural, 71(0):227–245.
  • Zhang, T., V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi. 2019. BERTScore: Evaluating Text Generation with BERT. September.
  • Zhao, W., M. Peyrard, F. Liu, Y. Gao, C. M. Meyer, and S. Eger. 2019. Mover-Score: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance. In K. Inui, J. Jiang, V. Ng, and X. Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 563–578, Hong Kong, China, November. Association for Computational Linguistics.
  • Zubiaga, I., A. Soroa, and R. Agerri. 2024. Ixa at refutes 2024: Leveraging language models for counter narrative generation. In IberLEF (Working Notes). CEUR Workshop Proceedings.
Fonte de datos: Dialnet