Limits of ChatGPT's conversational pragmatics in a Turing test on ethics, commonsense, and cultural sensitivity

Wagner, Wolfgang, Gaskell, George ORCID: 0000-0001-6135-9496, Paraschou, Eva, Lyu, Siqi, Michali, Maria and Vakali, Athena (2025) Limits of ChatGPT's conversational pragmatics in a Turing test on ethics, commonsense, and cultural sensitivity. Computers in Human Behavior: Artificial Humans, 5. p. 100191. ISSN 2949-8821

Text (1-s2.0-S2949882125000751-main) - Published Version Available under License Creative Commons Attribution Non-commercial. Download (2MB)

Abstract

Does ChatGPT deliver its explicit claim to be culturally sensitive and its implicit claim to be a friendly digital person when conversing with human users? These claims are investigated from the perspective of linguistic pragmatics, particularly Grice's cooperative principle in communication. Following the pattern of real-life communication, turn-taking conversations reveal limitations in the LLM's grasp of the entire contextual setting described in the prompt. The prompts included ethical issues, a hiking adventure, geographical orientation and body movement. For cultural sensitivity the prompts came from a Pakistani Muslim in English language, from a Hindu in English, and from a Chinese in Chinese language. The issues were deeply cultural involving feelings and emotions. Qualitative analysis of the conversation pragmatics showed that ChatGPT is often unable to conduct conversations according to the pragmatic principles of quantity, reliable quality, remaining in focus, and being clear in expression. We conclude that ChatGPT should be promoted as a machine and not a faux human and not be presented as a global LLM but be subdivided into culture-specific modules.

Item Type:	Article
Additional Information:	© 2025 The Author(s)
Divisions:	Methodology
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science B Philosophy. Psychology. Religion > BJ Ethics H Social Sciences
Date Deposited:	11 Sep 2025 14:20
Last Modified:	11 Sep 2025 14:20
URI:	http://eprints.lse.ac.uk/id/eprint/129487

Actions (login required)

View Item