Abstract

The advent of generative artificial intelligence and machine learning has led to innovations in language testing that have disrupted the status quo, particularly in terms of tests for university admissions. One example is the Duolingo English Test (DET), which relies heavily on automated processes for both item generation and scoring and is promoted as a less expensive, more convenient alternative to traditional admissions tests such as the TOEFL iBT and IELTS.  
However, questions remain about whether such a test can adequately assess the language skills required for academic success and, consequently, the extent to which test scores on the DET and those on the TOEFL iBT and IELTS are comparable. Concordance tables provide some information about test comparability and may be the most accessible source of information for many stakeholders; however, concordance tables should be interpreted in light of evidence that the tests being compared measure closely related constructs (Dorans, 2004; Knoch & Fan, 2024).  
This paper addresses these issues by presenting the results of a comparative construct analysis of the TOEFL iBT, IELTS, and the DET, drawing on publicly available information about the three tests, including sample items and practice tests. In this presentation, I highlight similarities and differences between the three tests in terms of explicit statements of intended constructs, the cognitive and linguistic demands of test tasks in listening, reading, speaking, and writing, and scoring procedures. The presentation documents several specific ways in which the reliance on automated item generation and scoring limits construct coverage and reduces validity.  

Sara Cushing is a professor of applied linguistics at Georgia State University. She received her Ph.D. in applied linguistics from UCLA. She has published research in the areas of assessment, second language writing, and teacher education.  She has been invited to speak and conduct workshops on second language writing assessment around the world, most recently in Vietnam, Colombia, Thailand, and Norway. Her current research focuses on assessing integrated skills and the use of automated scoring for second language writing.