Parvaneh Shayeste-Far; Gholam Reza Kiany; Monireh Norouzi
Abstract
1. IntroductionTesting-based reform has often been used as a lever for introducing desirable changes. This tenet is found in light of the claim that what is targeted to be measured by testing, in particular by tests of higher stakes, would strongly determine and shape what gets taught and learnt at the ...
Read More
1. IntroductionTesting-based reform has often been used as a lever for introducing desirable changes. This tenet is found in light of the claim that what is targeted to be measured by testing, in particular by tests of higher stakes, would strongly determine and shape what gets taught and learnt at the lower stakes. Clear examples are various University Entrance Examinations (UEEs) which have long appeared to have immense importance for the people and institutions involved. In fact, high-stakes tests such as the UEEs are widely perceived to have the potential to shape curricular teaching and learning. Such a test consequential effect, i.e., ‘test washback’, has encouraged decision makers and educators to manage for a test-based change to promote the quality of education in general and the quality of learning and teaching in particular. Reform policies as such have also been increasingly recognized by policymakers, educators, and research community in Iran, during the past decade. Such a recognition and appreciation of reform policies consequently led to reform initiatives that aimed at enhancing the quality of high school learning and teaching through reforming the conventional UEEs system of the country. The outcome was the ‘Act of Student Admission at Universities (ASAU)’ which mandates gradual replacement of the English requirements of the UEEs with high school National English Achievement Tests (NEATs) students have to take over their four high school years. However, the existing evidence shows that success is not necessarily ensured as hoped. Reasons for such a failure were mostly studied after ‘implementation’ or at the ‘completion phase’ of a reformed test but not during its ‘implementation phase’. Such an early evaluation has been mostly neglected. Taking into account the perceptions and perspectives of the English Language Teaching (ELT) practitioners of the country, the present study aimed to explore the effectiveness of the UEEs program change in improving the ELT context of the Ministry of Education (ME). 2. MethodologyA mixed-methods approach was employed to collect both qualitative and quantitative data. For such a purpose, semi-structured interviews with 16 ELT professors and head-teachers followed by questionnaire data from English language teachers were used. Participants were instructors of the Ministry of Science, Research and Technology (MSRT) and ME. Eight university professors and 8 ME teachers, selected through purposive sampling, participated in the qualitative phase, and 406 English teachers (both males and females, with teaching experience of 5 to 30 years), selected through cluster and convenient sampling procedures, participated in different phases of the study. The interview guide was prepared on the basis of the information obtained from the Act, higher-order policy documents of the country, available literature, and the researchers’ experience and knowledge of the two admission systems. The interview guide questions sought interviewees’ views about the consequences of the new tests on the ME’s English Foreign Language (EFL) teaching and learning. The interviews were tape recorded, transcribed, and then carefully content-analyzed.Next, quantitative data were collected through the researcher-made ‘Questionnaire of Change Consequences-Teachers’ version (QCC-Ts)’ that was distributed among 406 English teachers. The interview analyses together with a review of the related literature gave more than one hundred categories and behaviors showing high-stakes test impacts (e.g., teachers’ considerations of practicing the skills measured by high-stakes tests) which later led to the development of a temporary driven model of change impact stated through the QCC-Ts, with a total of 82 items forming three main scales: Teachers’ Planning and Instruction, Learning Objectives, and Fairness in admission. A series of factor analyses, both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were run to substantiate the whole scale. 3. Discussion3.1. Interviews: Professors and Head-TeachersData analysis of qualitative phase resulted into the identification of more than 200 themes that were finally subsumed under 14 categories. The necessity of “Adequacy of the new admission criteria”, “fairness in assessment through the new program”, “Students’ quality and depth of knowledge” as well as considerations about the “quality of the output of the ME”, and “Resurgence of the similar coaching for the UEEs” were the most commonly mentioned areas perceived to be negatively influenced by the NEATs-based program. A relative dissatisfaction was found in the interviewees’ views on the final outcome of the NEATs program in terms of its effects on the quality, quantity, rate, and depth of learning. In teaching area, the participants reported a probability for integrating a balanced teaching and learning of the main skills of the English language in the ELT contexts of high schools. However, the respondents found the consequences positive on “students’ learning activities” and “teachers’ assessment of students’ learning”. That is, the new program will lead to more student engagement in learning activities, and increased regular assessment by teachers. 3.2. Questionnaire Surveys: English TeachersThe results of the factor analysis of the QCC-Ts revealed a significantly good-fitting model for the constructed scales (CMIN/DF=2.79, RMSEA=0.04, CFI=0.981, GFI=0.984, and a small χ2=42). “Classroom activities and time-arrangement”, and “teaching materials” teachers would use in their classes were perceived as the two areas being most strongly driven by the new high-stakes tests (X=4.40, SD=.96; X=4.35, SD=.79 for classroom activities and materials, respectively). For instance, perceived consequences of the NEATs on ‘types and structures of classroom activities’ ‘students’ test-taking skills and strategies’, ‘extra time spent on grammar instruction’, and on ‘particular classroom activities purposefully adopted (e.g., vocab., translation, and structure drills) to increase students’ scores in the NEATs’ were reported by mean scores clustering around 4.5 in the Likert scale that illustrates ‘agreement’ or near ‘complete agreement’ with the NEATs impacts. Inconsistency between the desired policies and the observed practices was found for teachers’ ‘methods of teaching’ and ‘content of teaching’ that came up with the mean values of 4.01 and 3.59, respectively. For instance, two-thirds of the participants said that they change their teaching methods in order to ‘develop students’ NEAT-taking skills’ or ‘increase students’ scores in these tests’. These show how measurement-driven program of the ME obliges EFL teachers to change their adopted methods, employed activities, and applied materials in the direction of meeting testing objectives rather than learning objectives. As to the NEATs consequences on learning-related factors, EFL teachers were almost neutral about the NEATs consequences on learning-related factors, especially with regard to ‘affective consequences’ (2.5
