Data elicitation through language testing: Challenges of test design

Data elicitation through language testing: Challenges of test design

Černá, Monika

The paper discusses the challenges of test design in the context of a research project focusing on the analysis of tertiary students’ spoken production in English. One of the project aims is to create a corpus of learner-spoken English. The participants in the study are Czech first-year students in English language teacher education study programmes at three universities. In order to elicit samples of the students’ oral production, a test of speaking, including a pronunciation subtest,was designed with respect to the research aims and objectives and in accordance with the current trends in the field. The challenges faced by the research team may be divided into three groups – those pertinent to the construction phase of the research instrument, the pilot phase, and the data-collection phase. The paper discusses how the team responded to the perceived challenges.The process of test designing was informed by relevant literature (e.g. Bachman, 1990, Hughes, 2003, Luoma, 2004);the team strove to achieve the highest possible level of ‘test usefulness’, i.e. test qualities including reliability, construct validity, authenticity, interactiveness, impact and practicality, as introduced by Bachman and Palmer (2009). Consequently, the decisions regarding the abilities that the candidates, prospective English teachers, should demonstrate, as well as the decisions about the test format (elicitation techniques, number of tasks, etc.), were made with respect to the proposed ‘test usefulness’. The pilot phase confirmed the usability of the tool to elicit the required data, but also necessitated a few content- and procedure-related modifications. They reflected the results of the analysis of the performances recorded during the trial testing as well as the analysis of feedback questionnaires.Having revised the test, the researchers then implemented it in the data-collection processin three universities in the Czech Republic. The number of studentstested was 176. Keywords: Diagnostic test, speaking, English, tertiary students, research

Article
PeerReviewed

International Burch University

2015

2795