Technology-Based Language Assessment: Benefits and Challenges
Technology-based language assessment (TBLA) generally refers to the use of electronic devices, systems, and software to assess language skills or abilities in a specified domain and to monitor or evaluate progress in language learning. In recent years, TBLA has come to play an increasingly dominant role in all stages of designing, developing, and delivering language tests, and thus includes much more than just presenting language test items and tasks on computers. In fact, most language tests and assessment instruments in current use employ digital technology in critically important ways, and some tests are already completely digitally based and administered exclusively via the Internet.
These profound changes are associated with lots of benefits that are obvious when comparing TBLA to traditional paper-based testing. The benefits include higher efficiency and flexibility in test administration (e.g., continuous and on-demand administration), faster turnaround of scores and score reports, and higher precision of test results. A particularly compelling benefit of TBLA is the possibility to deliver technology-enhanced item formats. These formats promise to be more authentic, more closely representing the real-world skills and abilities to which score interpretations refer; at the same time, they may be more engaging, more motivating for examinees (e.g., within the context of game-based or simulation-based approaches to language assessment and learning). On the other hand, TBLA poses a number of serious challenges that need to be carefully addressed in order to further advance the field and to develop truly innovative TBLA. At the heart of the challenges are validity and fairness issues, including construct-irrelevant score variance caused by unfamiliarity of item and response formats or by lack of accessibility or usability for all examinees.
The conference theme addresses both the benefits and the challenges of TBLA, and thus covers a whole range of conceptual, practical, and ethical issues that may be investigated building on diverse theoretical, empirical, and methodological perspectives or approaches.
We invite proposals for papers, work-in-progress presentations, posters and symposia in research areas related to one or more of the following suggested topics:
- Automatic item generation; web-based item development and item banking; item authoring and editing
- Design, development, and evaluation of technology-enhanced item formats; integration of multimedia into TBLA; item and task prototyping
- Automated test assembly, test composition software to build test forms
- Automated scoring of constructed-response items or tasks; onscreen scoring of examinee performances
- Design and implementation of game-based and simulation-based assessment
- Mobile assessment via tablets or smartphones
- Diagnostic language assessment using digital technology; linking TBLA to language instruction
- Test delivery models, in particular, computerized fixed, linear-on-the-fly, computerized adaptive, or multistage testing
- Test security systems for managing and maintaining security of test content
- Psychometric modeling approaches to examinee responses in TBLA environments; psychometric analysis of complex item formats (e.g., integrated tasks)
- Score generation and reporting; system support for communication with examinees and other stakeholders
- Research into the validity and fairness of TBLA-based measures; comparisons between technology-based and paper-based language assessment; usability studies