Abstract

The advent of generative artificial intelligence and machine learning have led to innovations in language testing that have disrupted the status quo, particularly in terms of tests for university admissions. An example is the Duolingo English Test (DET), which relies heavily on automated processes for both item generation and scoring and is promoted as a less expensive, more convenient alternative to traditional admissions tests such as TOEFL iBT and IELTS.  
However, questions remain about whether such a test can adequately assess the language needed for academic success and thus the degree to which test scores on DET and those of TOEFL iBT and IELTS have similar meanings. Concordance tables provide some information about test comparability, and may be the most accessible source of information for many stakeholders; however, concordance tables should be interpreted in light of evidence that the tests to be compared measure closely related constructs (Dorans, 2004; Knoch & Fan, 2024).  
This paper addresses these issues by presenting the results of a comparative construct analysis of the TOEFL iBT, IELTS and the DET, relying on publicly available information about the three tests, including sample items and practice tests. In the presentation, I highlight similarities and differences between the three tests in terms of explicit statements of intended constructs, cognitive and linguistic demands of test tasks in listening, reading, speaking, and writing, and scoring procedures. The presentation documents several specific ways in which the commitment to automated item generation and scoring limits construct coverage and reduces validity.  

Sara Cushing is a Professor of Applied Linguistics at Georgia State University. She received her Ph.D. in Applied Linguistics from UCLA.  She has published research in the areas of assessment, second language writing, and teacher education.  She has been invited to speak and conduct workshops on second language writing assessment throughout the world, most recently in Vietnam, Colombia, Thailand, and Norway.  Her current research focuses on assessing integrated skills and the use of automated scoring for second language writing.