Washback Effect of TEFL MA Exam on Iranian Lecturers’ Classroom Activities

Introduction

The term washback or backwash generally refers to the potential effect of test on the process of teaching and learning. Several definitions have been used to describe the concept of washback, ranging from simple and straight-forward to very complex (Bailey, 1996). According to Shohamy (1993), “washback effect refers to the impact tests have on teaching and learning” (p. 4). Alderson and Wall (1993) point to the prevalence of the concept that testing influences teaching in the education and applied linguistics literature and call it as ‘backwash’ or ‘washback’. They hint to the power of the tests to determine what happens in the classrooms. Cheng (1997, p. 38) refers to washback as “an active direction and function of intended curriculum change by means of the change of public examinations”. In this study, the effects of TEFL MA University Entrance Examination (TEFL MA UEE) on teaching methodology and lecturers’ activities within the classroom are interpreted as washback.

The importance of a high-stakes test can be understood by the pressure it brings for students, teachers, administrators, parents or general public to perform greatly in it, because it is the test results which act as a criterion upon which vital decisions are made that affect instantly and directly participants’ future (Cheng, 2005). Logically, high-stakes tests cause much stronger washback effect than low-stakes tests, administered on a low population of the participants. According to Cheng (2004, p. 147) “there is convincing evidence to suggest that examinations, especially high-stakes tests, have powerful washback effects on teaching and learning within different educational contexts.”

As a country with a centralized educational system, Iran is not an exception to this issue, and the only opportunity for its graduates to enter higher education is through taking part in a high-stakes test. Entrance examinations, as high-stake tests, in general and MA University Entrance Examination (MA UEE) in particular are obviously of significant importance, because crucial decisions are made just based upon the examinees’ performance on such exams; decisions that have direct determining impacts on the examinees’ future life and their academic career (Razmjoo & Heidari Tabrizi, 2010).

Literature Review

It is stated that “testing is never a neutral process and always has consequences” (Stobart, 2003, p. 140). Tests, peculiarly high-stakes tests, intend to bring about ramifications for the test-takers, teachers, administrators, parents, and policy makers. Regarding the complex nature of this process, the presence of notable variation in the way different researchers have theoretically illustrated this event is not a matter of surprise. Plenteous washback effect enquiries have been made by different researchers in the field of language testing since 1990s (e.g., Alderson & Hamp-Lyons, 1996; Alderson & Wall, 1993; Andrews, 1994; Andrews, Fullilove & Wong, 2002; Birjandi & Taqizadeh, 2015; Cheng, 1999; Muñoz & Álvarez, 2010; Qi, 2005; Ramezaney, 2014; Rezvani & Sayyadi, 2016; Salehi, 2012; Salehi, Yunus & Salehi, 2012; Shohamy, Donitsa-Schmidt, & Ferman, 1996; Spratt, 2005; Stobart, 2003; Watanabe, 1996).

In the context of Iran, Salehi and Yunus (2012) investigated the washback effect of the Entrance Exam of the Universities (EEU) on the Iranian high school English teachers. The findings of their study revealed that the EEU negatively and implicitly influences English teachers “to teach to the content and format of the test” (p. 609). Furthermore, in a qualitative study, Rezvani and Sayyadi (2016) examined the EFL instructors’ insights to explore the potential washback effects of the new Iranian TEFL Ph.D. entrance exam on their teaching methodology, class assessment, and syllabus design. They concluded that despite its influential significance, the new Iranian TEFL Ph.D. entrance exam failed to produce substantial effect on EFL instructors’ teaching methodology. However, it was revealed that the new exam substantially influences the instructors’ syllabus design.

In spite of the importance of high-stakes tests, it seems that studies on washback and its effects have been poorly presented in Iran. This necessity has become the cornerstone of this research; therefore, the following research question was posed in this study:

What kind of washback effects, if any, does TEFL MA UEE exert on Islamic Azad University lecturers’ classroom activities and teaching methodology?

Method

In order to collect the required data and answer the research question, two types of instruments were used. They were (i) a questionnaire as a survey method and (ii) a structured interview protocol as a qualitative method. All the steps taken to design and validate the instruments and the process of data collection will be elaborated in the following.

Research Design for Quantitative and Qualitative Methods

The lecturers’ questionnaire aimed to explore English lecturers’ perceptions of TEFL MA UEE washback effect on their teaching methodology and their activities within the classroom. The interviews were used to check the precision and validity of the information obtained through the quantitative data.

The questionnaire was comprised of two main parts planned and constructed in English (See Appendix A). The first part contained seven sections of lecturers’ personal characteristics related to demographic information including age, gender, degree, major field of study in MA, major field of study in Ph.D., major field of teaching, and years of teaching experience. The second part consisted 35 items entirely designed on a five-point Likert scale of agreement, where 1 = strongly disagree, 2 = disagree, 3 = no opinion, 4 = agree, and 5 = strongly agree. This part primarily concerned with lecturers’ opinions about aspects of teaching methodology and their classroom activities in relation to TEFL MA UEE. The structure and themes of the lecturers’ questionnaire have been illustrated in Appendix B.

In order to check the validity of the questionnaire, the first draft of the questionnaire was given to three experts in TEFL to examine its content and face validity. Based on their suggestions, some modifications were made on the questionnaire. Then, five lecturers at Iranian universities were requested to fill out the modified questionnaire to confirm its reliability (Cronbach’s alpha coefficient was reported to be 0.84).

The questions of the interview protocol were mainly taken from the questionnaires’ items and expressed in new wording (See Appendix C). Each interview question was followed by further probing questions to obtain a more comprehensive understanding of the interviewees’ answers. Similar to validity checking procedure used for the questionnaire, the first draft of the interview protocol along with the content validation form were given to three TEFL experts. The experts’ recommendations were led to some modifications, and the second draft was prepared. Two lecturers who had taken part in the pilot study of the questionnaire were asked to take part in a pilot interview to check the lucidity of the questions, and no ambiguity was found. Then, the final draft of the interview protocol was prepared based on the previous procedures conducted.

Data Collection Procedure

Convenience sampling was used to fulfill the needs of the survey. Sixteen university lecturers who were teaching undergraduate English courses at Islamic Azad Universities of Isfahan and Najafabad were selected. The questionnaires were directly given to the lecturers and all of them were returned. When choosing the participants, the gender was not taken into account as an important factor. Moreover, in order to conduct the interview, all the lecturers who filled out the questionnaire were asked to participate in the interview but 5 lecturers consented to be interviewed individually. They were assured of the confidentiality of the interviews and their answers were only used for this research project. Reporting the interviews, some codes were used in order to protect the participants’ identities. The structured interviews were tape-recorded and all were transcribed directly after each interview session, verbatim, for detailed analysis. The interviews were semi-structured and flexible and sometimes new questions were asked pertaining to the participants’ responses. All the interviews were carried out within a two-week period after the questionnaires were completed.

Results

The quantitative and qualitative research findings of the study gained from a themes-based classified survey questionnaire and an interview protocol are presented in this section.

Lecturers’ Questionnaire

This questionnaire was comprised of 2 main parts. The first part was related to the lecturers’ demographic information and the second part concerned about lecturers’ opinions about aspects of their classroom activities and teaching methods.

Lecturers’ Demographic Information

The summary of the lecturers’ demographic characteristics is illustrated in Table 4.1.

Table 4.1. Demographic Characteristics of the Lecturers

Lecturers’ Perceptions of the Effect of TEFL MA UEE on Their Classroom Activities and Teaching Methodology

This part consisted of 3 main subcategories and 35 items concerning the lecturers’ opinions about their classroom activities and teaching methods entirely designed on a five-point Likert scale of agreement. This main theme was divided into 5 sub-themes of (i) lecturers’ syllabi, (ii) lecturers’ teaching, (iii) score pollution practice, (iv) curriculum alignment aspect, and (v) lecturers’ personality. The obtained results from these sub-themes are reported below.

(i) Lecturers’ Perceptions of the Effect of TEFL MA UEE on Their Syllabi

The lecturers were required to convey their perceptions of the effect of the TEFL MA UEE on their syllabi. The results attained are depicted in Table 2.

Table 2. Lecturers’ Perceptions of the Effect of TEFL MA UEE on Their Syllabi

This category was designed to explore the lecturers’ perceptions of the effect of TEFL MA UEE on their syllabi. The items ‘I try to follow teaching syllabus’ and ‘In devising my teaching syllabus for instruction, I look at relevant sources to ensure that I cover the kind of items that are to be tested in TEFL MA UEE’ got the highest ranks on the mean scores column, followed by ‘I select the books which were the source of the exam in the previous, with the mean scores 4.44, 3.38, and 3.31 respectively. In fact, the majority of the sampled lecturers stated they follow their teaching syllabi knowing that the syllabi have been designed in a way that cover the needs of TEFL MA UEE. Furthermore, half of the respondents agreed that in devising their teaching syllabus for instruction, they look at relevant sources to ensure that they cover the kind of items that are to be tested in TEFL MA UEE. Similarly, more than half of the respondents believed that they select the books which were the source of the exam in the previous years.

On the other hand, more than half of the lecturers rejected the items ‘I overload my syllabus and do my best to cover the whole content needed for TEFL MA UEE’ and ‘I change my syllabus and teaching activities with regard to topics in the previous test of TEFL MA UEE every year’. The findings show that the lecturers are influenced by TEFL MA UEE to some extent, but they just have an eye on this exam and do not put a lot of pressure on the students and themselves to cover whatever is needed for the exam.

(ii) Lecturers’ Perceptions of the Effect of TEFL MA UEE on Their Teaching Methods

In this section, the lecturers were asked about the changes they make in their teaching due to the TEFL MA UEE.

Table 3. Lecturers’ Perceptions of the Effect of TEFL MA UEE on Their Teaching Methods

As it can be seen in Table 3, three items got the highest ratings. The majority of the sampled lecturers indicated that they design their classroom activities according to the students’ needs and abilities. Most of the respondents also indicated that ‘I think my teaching method is helping students to get ready for both final exam and TEFL MA UEE’ and ‘I try to achieve the test objectives throughout teaching’. According to the lecturers’ perceptions, they try to modify their teaching activities to meet the students’ needs, but as they said in the interviews, they make a distinction between students’ real needs and the need to succeed in TEFL MA UEE. Therefore, it can be understood that they do not care about the requirements of the exam.

On the other hand, about two-third of the surveyed lecturers believed that they do not give the students assignments related to TEFL MA UEE indicating that the lecturers do not pay enough attention to the students’ assignments. The reason might refer to the nature of the classroom activities which is usually based on lecturers’ speech and low class participation of the students. Although most of the lecturers declared in item 6 that they try to achieve the test objectives through teaching, more than half of them believed that they do not pass over some teaching methods that are not sufficient in preparing their students for TEFL MA UEE. Moreover, more than half of the respondents expressed that their EFL teaching methods and techniques are not influenced by TEFL MA UEE.

(iii) Lecturers’ Perceptions of the Effect of TEFL MA UEE on Score Pollution Practice

This category was designed aiming to investigate the lecturers’ attitudes towards the effect of the TEFL MA UEE on the score pollution practice. In this section, the lecturers were asked whether they teach test taking strategies, useful tricks for the exam, or explain sample test items.

The items ‘I use the teaching methods and techniques that help my students succeed in the examination’ and ‘I provide my students with the list of important books for TEFL MA UEE preparation’ got the highest ranks on the mean scores column, followed by ‘I advise my students to practice the questions of the previous examinations for relatively better preparations for the examination’ and ‘I review recent past years' knowledge test of TFEL MA UEE every year’. The majority of the respondents believed that they use the teaching methods and techniques that help their students succeed in the examination and they provide their students with the list of important books for TEFL MA UEE preparation. This belief is a clear indication of the lecturers’ attention to TEFL MA UEE.

Table 4. Lecturers’ Perceptions of the Effect of TEFL MA UEE on Score Pollution Practice

Item 11 obtained about two-third of the sampled lecturers’ agreement. Less than two-third of the lecturers also indicated that they teach multiple choice test taking strategies to their students, they encourage their students to learn how to use test-taking strategies, they use TEFL MA UEE items, as examples, while teaching in their classes, and they provide the handouts which contain points covered in TEFL MA UEE for their students, showing that the lecturers prepare the students to meet their needs for passing the exam.

(iv) Lecturers’ Perceptions of the Effect of TEFL MA UEE on Curriculum Alignment Aspects

According to English (2000), “Curriculum alignment is a process to improve the match between the formal instruction that occurs in the school and the classroom and that which any test will measure” (p. 63). In this section, the lecturers were asked about the effect of TEFL MA UEE on curriculum alignment aspects.The obtained results are shown in Table 5.

Table 5. Lecturers’ Perceptions of the Effect of TEFL MA UEE on Curriculum Alignment Aspects

When the respondents were asked whether TEFL MA UEE had an influence on the curriculum alignment aspect, all the items got the mean scores below 3.5, indicating the lecturers’ negative attitude towards these items. Half of the respondents had a positive attitude towards item ‘I increase the pace of my teaching and try to present issues which were covered in the tests during the last years’, however, the same percent of the participants expressed their agreement with the item ‘I think it is important to cover every section of the textbook although some sections are unlikely to be tested in TEFL MA UEE’. On the other hand, item 17 with the mean score of 2.88 indicated the lecturers lack of tendency about necessity of the teaching the contents according to their sequence of importance in TEFL MA UEE. It may justify the neutral effect of the TEFL MA UEE on the lecturers regarding adapting their selected course book to the needs of the exam.

(v) Lecturers’ Perceptions of the Effect of Their Personality on their Teaching Methods and Activities

The respondents were directly asked to express their attitudes towards the effect of their personality on their teaching methods and activities. The obtained results are shown in Table 6.

Table 6. Lecturers’ Perceptions of the Effect of Their Personality on Their Teaching Methods and Activities

When the respondents were asked to express their attitudes towards the effect of their personality on their teaching methods and activities, it was found that the majority of the lecturers believed that they teach whatever they think is important to teach, no matter whether their  students like it or not. More than two-third of the lecturers stated that they teach whatever they think is important to teach, no matter whether it is tested or not as well. Additionally, Item 10 expressing lecturers’ opinion about the effect of their personality on their selection of teaching methods gained more than half of the lecturers’ agreements. On the other hand, the item ‘I try to teach points of TEFL MA UEE of the previous years to avoid feeling of guilt and anger’ followed by the item ‘I feel embarrassed if my students perform less well on TEFL MA UEE’ obtained the lowest mean score signifying the lesser effect of TEFL MA UEE on the lecturers’ teaching methodology and activities compared with the more prominent role of the lecturers’ personality on their teaching activities and methods.

Lecturers’ Interviews

Lecturers’ attitudes towards the effects of TEFL MA UEE on their teaching methodology and classroom activities can be considered as a main aspect in washback studies and the data collected from their contribution in this study is a valuable source for the whole study. The demographic data of the interviewed lecturers are depicted in Table 7.

Table 7.Demographic Data of the Interviewed Lecturers

Note. Codes were used instead of the names.

The interviews with the lecturers as a main part of washback studies presented the valuable information about washback effect of the TEFL MA UEE on teaching methodology and classroom activities. When the lecturers were asked about the general factors influencing their classroom activities and teaching methodology, they highlighted the basic content of the syllabus, relevant teaching according to the needs of the university classes, covering the basic skills needed for a B.A. English student, trying to make everything comprehensible, introducing the books which are the references for M.A. UEE, creating a general picture of the whole content and using methods relevant to the content of the course. The view of F1 illustrated the situation:

I usually try to follow the basic content in the syllabus, which means I don’t alter the syllabus just due to the fact that some questions are included in M.A. exam. I think the syllabus is comprehensive enough to cover all the needs of TEFL MA UEE, and if it seems that there is an incompatibility between the syllabus and the need of MA UEE, it is all because of deficiency in teaching of the whole syllabus in the university classes.

Regarding limitation of time and its effect on her coverage of syllabus she said:

I try to cover all parts of the syllabus, but if there is not enough time to teach all the parts, I will teach those chapters of the book which are necessary for the students to know as the future teachers but not those chapters which are important for MA exam.

She continued the conversation by expressing her attitudes towards teaching test-taking strategies and preparatory classes:

I think it is not my responsibility to teach the students test-taking strategies. There should be some extra classes for students to teach them how to cope with those types of questions. If I were supposed to teach in preparatory classes, I would teach the strategies, techniques, and tricks of test-taking.

F2 spoke in the same way, but her responses were related more to providing comprehensible input for the students, creating a general picture of the whole content, and covering the basic skills needed for a BA English student as the major factors directing her classroom activities and teaching methods. She added:

I follow the syllabus of the course, because most of the time MA exams are based on these syllabi, so I am not trying to change the syllabus to match it with the content of the exam.

She also explained about her teaching method and the principles she was trying to observe while she was teaching:

I try to have a different teaching method not based on the exam but to make everything comprehensible for the students. Most of the time, we do not have classroom activities and the classes are usually teacher-centered.

Concerning teaching test-taking strategies, she had exactly the same view as F1. She mentioned that:

It’s not my responsibility to teach the students test-taking strategies. I think I have to teach in a way that my students feel satisfy after leaving the class.

M1 referred to his lack of interest to be influenced by TEFL MA UEE, and he considered that helping the students to obtain an acceptable level of education suitable for BA and to be qualified MA students as his ultimate goal. He explained that:

Actually TEFL MA UEE does not affect my teaching method because I do not base my teaching method on MA exam. I usually have my own syllabus based on the textbooks that I have. I’m not influenced by MA exam at all.

M1 emphasized that he was not influenced by TEFL MA UEE and any influences that might have affected him were due to the limited popular sources. Moreover, F3 confirmed the neutral effect of TEFL MA UEE on her classroom activities and teaching methodology. She stated:

I do not consider any form of the examination, and I do not think that I am influenced by any types of examinations. I do not care about TEFL MA UEE, so I try to cover the syllabus determined by the ministry, and I think it is a complete syllabus and there is no need to change or add something to it.

She referred to her personality traits as a major factor influencing her teaching. She said:

I cannot divide my personality from my teaching method. It definitely influences my teaching method to a great extent, but I think as a whole, it is my habit to use a method that is exactly related to the content of the course or content of the syllabus not to TEFL MA UEE.

Similarly, she excluded teaching test-taking strategies as a university lecturer’s responsibility when she said:

I do not spend the class time to teach the students test-taking strategies needed for MA exam. I think it is not my responsibility to teach the students test-taking strategies.

In the same way as F1, she considered the students’ future needs as future English teachers as her main concern in her teaching:

Regarding the most important factor in my teaching, first I consider the future needs of the students as BA holders of English teaching major and what they have to know. Sometimes some parts of the course book will be ignored and skipped by, and we will spend more time on those parts which are necessary for their future jobs.

M2, however, had a slightly different opinion regarding the effect of TEFL MA UEE on his teaching method and activities. He referred to the moderate influence of this exam on his teaching. He explained that:

I usually have an eye to what appears on MA exam and prep courses, but that does not mean that I do everything in the class to prepare students for the exam.

Despite the moderate influence of the exam on his teaching, he was also opposed to spend class time teaching test-taking strategies. He mentioned that:

I think it is not my responsibility to teach the students certain strategies for the exam, which is usually true about prep courses. I use to teach such prep classes, and I know about these strategies. So once in a while, I may just tell my students that this is the way you may want to answer MA exam questions but not that I teach them strategies specifically in the class.

The above comments indicate the rather neutral effect of TEFL MA UEE on the lecturers’ classroom activities and teaching methods. There is a common belief among the interviewed lecturers that it is not their responsibility to teach test-taking strategies at the university, and these techniques should be learnt at preparatory classes. Furthermore, they all stated that they would introduce supplementary materials, useful for MA exam, to the students. However, they do not spend the class time teaching them. Majority of the interviewed participants declared that the providing the students with the skills needed for their future job as an English teacher is the main concern of their teaching.

Discussion and Conclusion

Resorting to the power of the high-stakes tests in order to change teaching and learning is commonplace in many parts of the world, especially in countries with centralized educational system (Shohamy et al., 1996), but the findings from this study convey TEFL MA UEE did not induce a high level of washback on the lecturers’ classroom activities and teaching methodology. Spratt (2005) supports this finding as she also found that teaching methods seem to be prone to different types and amount of washback from context to context and teacher to teacher. She added “It varies from no reported washback to considerable washback” (p. 17). She referred to the teacher as the main variable for these differences rather than the exam itself.The finding is also in congruent with previous research by Shih (2009), which concluded that a national English proficiency test failed to create a high level of washback on teaching. When the interviewed teachers were asked about the factors predominant in choosing their classroom activities and teaching method, they referred to some criteria as their priorities including following the basic content of the syllabus which has been developed by the authorities, relevant teaching according to the needs of the university classes, covering the basic skills needed for a BA English student, trying to make everything comprehensible, introducing the books which are the references for TEFL MA UEE, creating a general picture of the whole content, and using methods relevant to the content of the course. They considered the syllabus a comprehensive one, covering all the needs of TEFL MA UEE.

The finding is consistent with Watanabe’s (2000) findings which showed the teachers’ intentional avoidance of referring to test taking techniques, since they assumed that it is actual English skills that would help the students pass the exam. The lecturers further discussed the impossibility of changing the predetermined syllabus and having no other choice but obedience. They confirmed that most of the classes are teacher-centered and give students no opportunity to participate in their own learning and practice and sharpen their language skills.

However, a careful scrutiny of the data obtained from interviews and questionnaires indicates some degrees of exam influence on some teaching activities of a number of lecturers. The majority of the respondents believed that they provide their students with the list of important books for TEFL MA UEE preparation, and it is in line with some studies which support the concept that high-stakes tests might influence some aspects of teaching while not influencing others (Alderson & Wall, 1993; Salehi, 2012), or they might affect some stakeholders in different ways than others (Alderson & Hamp-Lyons, 1996).

There was a contradiction between what lecturers declared to use as their teaching methodology and technique in questionnaire and what they claimed to use in their interviews. In spite of the fact that the majority of the respondents believed that they use teaching methods and techniques that help their students succeed in the examination, participants in the interviews mostly referred to their personality as the factor leading them in adopting their teaching method. It is also concluded by Alderson and Wall (1993) that an exam does not and cannot determine how teachers teach. However, it is against Kellaghan and Greaney’s (1992) claim that examinations have a tendency towards dictating not only what is taught but also how it is taught.

Regarding test-taking strategies, they all stated in the interviews that it is not their responsibility to teach test-taking strategies at the university, and these techniques should be learnt at preparatory classes, and the findings from the same parallel question in the Lecturers’ Questionnaire were supported by the obtained results in this part. Mehrens (1991) took a similar view regarding these classes and described them as courses which are held for training test taking strategies, familiarizing the students with the test, and giving them practice under exam condition. Lecturers believed that it was responsibility of preparatory classes to equip the students with these techniques, and if they were teaching in preparatory classes, they would teach the strategies, techniques, and tricks of test-taking.

In light of what has been discussed above, the results of this research imply that teaching in the classroom could not be merely the consequence of the exams requirements. A variety of factors are involved that should be examined while the washback effect of an exam is under investigation. The findings revealed that, at a micro level, TEFL MA UEE has a fairly neutral washback effect on lecturers. In investigating the participants’ perceptions of the effects of TEFL MA UEE on their process of teaching, there were little signs of influence of the exam on their routines. Regarding the lecturers, covering the syllabus was the most important factor for them, and they felt they were not under any obligations to follow the demands of the exam. They expressed having an eye on the exam as the maximum impact of the exam on their decisions.

The findings of the study might enlighten the policy makers and test developers about how noteworthy and motivating TEFL MA UEE is regarded by lecturers as one of the main parties of interest who may enjoy the immediate influence of the reforms of the exam. Some of the factors that can have a hinder probability regarding the lecturers’ motivation to teach include: lecturers’ attitudes towards TEFL MA UEE, test preparation, and their responsibilities, ineffectiveness of the results of the exam on the lecturers’ positions and its destructive impact on their teaching, and poor communication between test makers and test users. The results of the current study could also be of use to the researchers in the field of washback studies by providing some guidelines and empirical support for this complicated phenomenon.