Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • apa.csl
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The dark side of LLM-powered chatbots: Misinformation, biases, content moderation challenges in political information retrieval
Karlstad University, Faculty of Arts and Social Sciences (starting 2013), Department of Geography, Media and Communication (from 2013).ORCID iD: 0000-0002-0920-8153
Karlstad University, Faculty of Arts and Social Sciences (starting 2013), Department of Geography, Media and Communication (from 2013).ORCID iD: 0000-0002-8504-5691
Karlstad University, Faculty of Arts and Social Sciences (starting 2013), Department of Geography, Media and Communication (from 2013).ORCID iD: 0000-0003-4286-7764
Karlstad University, Faculty of Arts and Social Sciences (starting 2013), Department of Geography, Media and Communication (from 2013).ORCID iD: 0000-0003-1430-4964
Show others and affiliations
2025 (English)In: Selected Papers of Internet Research, SPIR, ISSN 2162-3317, Vol. 2024:, no AoIR2024Article in journal (Refereed) Published
Abstract [en]

This study investigates the impact of Large Language Model (LLM)-based chatbots, specifically in the context of political information retrieval, using the 2024 Taiwan presidential election as a case study. With the rapid integration of LLMs into search engines like Google and Microsoft Bing, concerns about information quality, algorithmic gatekeeping, biases, and content moderation emerged. This research aims to (1) assess the alignment of AI chatbot responses with factual political information, (2) examine the adherence of chatbots to algorithmic norms and impartiality ideals, (3) investigate the factuality and transparency of chatbot-sourced synopses, and (4) explore the universality of chatbot gatekeeping across different languages within the same geopolitical context. Adopting a case study methodology and prompting method, the study analyzes responses from Microsoft’s LLM-powered search engine chatbot, Copilot, in five languages (English, Traditional Chinese, Simple Chinese, German, Swedish). The findings reveal significant discrepancies in content accuracy, source citation, and response behavior across languages. Notably, Copilot demonstrated a higher rate of factual errors in Traditional Chinese while exhibiting better performance in Simplified Chinese. The study also highlights problematic referencing behaviors and a tendency to prioritize certain types of sources, such as Wikipedia, over legitimate news outlets. These results underscore the need for enhanced transparency, thoughtful design, and vigilant content moderation in AI technologies, especially during politically sensitive events. Addressing these issues is crucial for ensuring high-quality information delivery and maintaining algorithmic accountability in the evolving landscape of AI-driven communication platforms. 

Place, publisher, year, edition, pages
2025. Vol. 2024:, no AoIR2024
Keywords [en]
Algorithmic gatekeeping, comparative studies, algorithm auditing, generative information retrieval
National Category
Media and Communication Studies
Research subject
Media and Communication Studies
Identifiers
URN: urn:nbn:se:kau:diva-103165DOI: 10.5210/spir.v2024i0.13977OAI: oai:DiVA.org:kau-103165DiVA, id: diva2:1937762
Conference
AoIR2024: The 25thAnnual Conference of the Association of Internet Researchers. Sheffield, UK. 30 Oct - 2 Nov 2024.
Note

Selected Papers of #AoIR2024: The 25thAnnual Conference of the Association of Internet ResearchersSheffield, UK/ 30 Oct -2Nov 2024

Available from: 2025-02-14 Created: 2025-02-14 Last updated: 2025-10-16Bibliographically approved

Open Access in DiVA

fulltext(238 kB)58 downloads
File information
File name FULLTEXT01.pdfFile size 238 kBChecksum SHA-512
7d731ceee8e73920dca01f92b7b3faa3245a1d665317c3d6d1072ebd4efb188298d70f3ca0c4f2f368b070476fb339288380e85f4219e450dd65e5e9f5165ee8
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Authority records

Kuai, JoanneBrantner, CorneliaKarlsson, MichaelVan Couvering, Elizabeth

Search in DiVA

By author/editor
Kuai, JoanneBrantner, CorneliaKarlsson, MichaelVan Couvering, Elizabeth
By organisation
Department of Geography, Media and Communication (from 2013)
In the same journal
Selected Papers of Internet Research, SPIR
Media and Communication Studies

Search outside of DiVA

GoogleGoogle Scholar
Total: 58 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 51 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • apa.csl
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf