Large language models (LLM) (e.g., ChatGPT) have garnered massive attention due to their potential to serve as interactive partners for a wide range of teaching and learning activities. In instructed second language acquisition, one potential use of LLMs is for pragmatics instruction. The current study sought to assess ChatGPT’s suitability for this task by analyzing its output in response to 144 DCTs (discourse completion tasks), which test a language learner’s ability to create apologies, requests, and refusals over differential levels of social and power distance, and degrees of imposition. Quantitatively, ChatGPT produced significantly shorter apologies than the other two speech acts (p < .001, R2 = .49), utilizing fewer pragmatic strategies (p < .001, R2 = .38). Closer social distance and greater imposition also resulted in longer responses, with higher imposition also eliciting ChatGPT to use more pragmatic strategies. Qualitatively, human raters scored ChatGPT’s apologies significantly lower in terms of their appropriateness, politeness, and language use, despite there being no statistical differences in ratings for requests and refusals. The main pedagogical implications are that, while ChatGPT offers an attractive option for students looking for more language exposure and interaction outside of the classroom, some degree of skepticism is still required due to its quantitative and qualitative inconsistencies.

Pragmatics has been defined as, "the study of how-to-say-what-to-whom-when" (Bardovi-Harling, 2013, p. 68). Despite its core function in communicative competence, classroom pragmatic instruction is typically extremely limited, and textbooks have been shown to contain very little metapragmatic information. Generative Al large language models have the potential to fill this gap as discourse-completion tasks (DCT) (scenarios which train learners in the use of pragmatic strategies) can be performed outside of the classroom. To ascertain the viability of ChatGPT as a tool for pragmatic instruction, we asked it to respond to 72 DCTs in order to assess its mastery of the speech acts of requests, apologies, and refusals at various levels of imposition, social, and power distance. We analyzed the length of responses and number of strategies employed, while a panel of 13 judges rated their appropriateness, politeness, and language use.
Results indicated that ChatGPT produced generally suitable responses, though significant differences appeared depending on speech act and degree of power distance. Our presentation will conclude with a discussion of common pitfalls to avoid in prompt composition and provide participants with specific items and guidelines to get the most accurate and useful responses from ChatGPT as a pragmatic instruction tool.