Students’ and Teachers’ Views on the Portfolio Component of the English Foundation Programme

Introduction

Many educational and professional learning contexts all around the world are incorporating portfolios as a means of monitoring students’ professional development. Nowadays, digital portfolios are taking the place of paper-based ones and they are now most often the preferable option when portfolios are brought to an educational setting for the first time (Lambert & Corrin, 2006; Wade, Abrami & Sclater, 2005). Using a portfolio is a demanding job specifically in exam-oriented cultures and contexts like Sultan Qaboos University (SQU) in Oman. Many researchers have carried out various studies on the portfolio. They have shown that portfolio assessment has had a positive effect on students’ language skills in general (Yurdabakan & Erdogan, 2009; Fahed Al-Serhani, 2007; Burner, 2014; Nicolaidou, 2012). Some have also focused on the perception of students and teachers of the ePortfolios (Deneen, Brown & Carless, 2018; Dougherty & Coelho, 2017). Some other researchers have compared the electronic with the paper-based portfolios (Wesel & Prop, 2009; Driessen, Muijtjens, Van Tartwijk & Van Der Vleuten, 2007).

Nevertheless, there appears to be little focus, if any, on comparing teachers’ and students’ perceptions of portfolio components as a whole in the Gulf Cooperation Council (GCC). The aim of this paper is to shed more light on this issue by exploring teachers’ and students’ views on the perceived benefits of the portfolio component in the English Foundation Programme at SQU. The results of the study could reveal any probable differences, first between the two groups of students based on their medium of the portfolio, and additionally, between teachers and students.

Since students’ and teachers’ views is a crucial factor in evaluating portfolios, the findings could be beneficial to the academic centers where the portfolio is or would be implemented in the curriculum. Hence, the results might seemingly be a  guide for designing portfolios in the future. Academic research has not caught up with the lack of insight into the perceptions of the real users of the portfolio in English language centers in GCC. After reviewing the literature on the subject, research questions and objectives are addressed. Subsequently, each part of the result section is followed by a  discussion.

Review of the Literature

Portfolios were first employed in the field of fine arts where they could show the depth and breadth of the work of an artist’s abilities (Jongsma, 1989). According to Barrett (2006), a portfolio is a collection of work that a learner has gathered, chosen, ordered, reflected on, and presented to indicate his or her understanding and improvement over a period of time. Many researchers have studied portfolios as authentic assessment tools and stated that portfolio assessment had a positive effect on students’ language skills in general (Yurdabakan & Erdogan, 2009; Fahed Al-Serhani, 2007; Burner, 2014; Nicolaidou, 2012). Furthermore, previous studies have confirmed that portfolios are better predictors of students’ performance in an authentic situation to improve students’ higher-order thinking skills (DeFabio, 1993; Jamentz, 1994; Tillema, 1998), empower students to be more actively engaged in the learning process and take control of their own learning (Blake, et al., 1995; Paulson, Paulson & Meyer, 1991; Valeri-Gold, Olson, & Deming, 1991), and improve their learning achievement (Winograd, 1995).

Besides, portfolios are great tools in formative and summative student assessments (Chatham-Carpenter, Seawel, & Raschig, 2009; Klenowski, Askew, & Carnell, 2006; Lam & Lee, 2009). Moreover, portfolios possess integrative learning potential. Therefore, students are able to link experiences and knowledge acquired in the academic context with a variety of other contexts (Acosta & Liu, 2006; Light, Sproule, & Lithgow, 2009; Tosh, Wedmuller, Chen, Light, & Haywood, 2006).

The use of electronic portfolios has gained considerable attention and has become more widespread (Barrett, 2000). Consequently, a lot of studies have scrutinized different aspects of this new version of portfolios. For example, Munday (2017), using a case study approach, embedded ePortfolios into a Master’s Program in an Australian university to investigate the continued use of ePortfolios. She concluded that ePortfolio design requires considerable planning for academic use in higher education. In another study, Chang, Tseng, Liang, and Chen (2013) used e-portfolios to enhance university students’ knowledge management (KM) performance on 88 university students majoring in IT. They found that e-portfolios significantly facilitated KM performance.

Many studies have also investigated the perceptions of teachers or students toward ePortfolios. For instance, Deneen et al., (2018) investigated student technology and assessment perceptions of eportfolios. They realized that the positive attitudes of students towards eportfolio technology and assessment contributed to an increased GPA.  They also showed that a positive attitude towards eportfolios led to positive views about eportfolios as an assessment tool for learning. After examining Arabic-speaking students’ personal opinions in regard to the usefulness of ePortfolios, Dougherty and Coelho (2017) reported students’ positive comments on the impact of ePortfolios on their language learning. They concluded that students saw the ePortfolio as a means to collaborate with their peers and teachers, organize their academic work, and save their academic work for future use.

Moreover, a number of studies have focused on comparing the ePortfolio with its paper-based counterpart. Wesel and Prop (2009) worked on a large-scale study on the first-year students, half of whom used the ePortfolio and the other half used the paper-based portfolio. They reported that the perception of the two groups in terms of support for self-reflection and the usefulness of portfolios does not differ significantly.

Driessen et al., (2007) used an experimental method to compare first-year medical students’ reflective portfolios according to the medium. They utilized a questionnaire for the students and ran interviews with teachers. They concluded that creating an electronic portfolio boosted student motivation and that an ePortfolio is more user-friendly for mentors. They also said that ePortfolio facilitates the acceptance of portfolios by students and teachers.

Wanchid and Charoensuk (2015) investigated the effects of the use of paper-based and weblog-based electronic portfolios on the writing achievement to find out the second-year hotel and tourism students’ attitudes towards the use of the portfolio and to compare the perception of the students based on the medium of the portfolio. It was found that the effects of the use of paper-based portfolios and weblog-based electronic portfolios on the writing achievement were not significantly different. However, they saw the portfolio as a great tool for improving writing ability, evaluating students’ own learning, engaging in their learning both inside and outside the classroom, and gaining critical thinking skills.

Despite the numerous studies carried out on the portfolio, much of the literature research has actually focused on the medium of the portfolio (Wesel & Prop, 2009; Driessen et al, 2007; Wanchid & Charoensuk, 2015). Besides, most of the previous works are context-oriented and consequently, their findings are not generalizable to specific contexts like GCC. Furthermore, none has probed the portfolio in the Foundation Program of a Higher Education center like SQU. The actual users of the portfolio, students and teachers, are seemingly the best ones to evaluate its advantages and disadvantages. Hence, this study aims to investigate whether or not the portfolio is working in students’ and teachers’ ideas in this university. Portfolios are quite new to GCC and therefore require much attention at their early stages of implementation. No previous study has compared perceptions of students and teachers regarding the portfolio in this context.

Research Objectives

Although ePortfolios are being used more frequently in higher education, many educators still prefer to use their paper-based counterparts. As a result, few studies have compared the perceptions of the students and the teachers on the benefits of the portfolio in general and on the forms of portfolios. The present study tries to see what teachers and students actually think about the portfolio element, which is implemented in English courses in higher education in Oman.

Research Questions

Due to the inadequacy of background knowledge on the portfolio component specifically in Oman, the present study tries to answer the following questions:

1-      How do Omani students perceive the benefits of the portfolio component in their Foundation Programme at SQU?

2-      How does the medium of portfolio affect students’ perception?

3-      How do teachers perceive the benefits of the portfolio component in the Foundation Programme at SQU?

4-      Is there any difference between teachers’ perception of portfolio and that of students’?

5-      What are the advantages and disadvantages of each medium of the portfolio in teachers’ ideas?

Components of the Portfolio in the Context

The goals set for the portfolio by SQU are:

1- To help students improve their study skills,

2- To develop learner autonomy in the students,

3- To make them have a stronger sense of ownership of their work.

To reach these goals, SQU has implemented the portfolio in specific English classes which consists of four components.

Academic Planner

In order to be well-organized during the semester, the students use an academic planner daily. In this planner, they have to write important deadlines for assignments, projects, quizzes, and plans for self-study. For the courses which use the ePortfolio, the students use Google Calendar and share their calendar with their teachers every week. For the classes with the paper-based version, the students are given an already-printed calendar they have to fill in every week.

Can-Do Statements

The Can-Do Checklist contains statements that describe skills students develop during the semester. The checklist contains five sections: listening, reading, study skills, writing, and speaking. Students have to complete this list either electronically or paper-based three times during a semester. The students do the task on week 3, week 9, and week 15 to see how well they are progressing in their study by selecting I cannot, I somewhat can, or I can for each statement.

Vocabulary Logs

To stimulate students to work on their vocabulary regularly, they are asked to organize up to 10 vocabulary logs during one academic semester. They are encouraged to put the words that they have in their reading and listening classes. In classes with ePortfolios, they can add the dictionary meaning, part of speech, the source sentence, the phonetic symbols, sound, and a picture for each word, while in paper-based portfolio classes, they have to write the meaning, the source sentence, and part of speech for each entry.

Reflections

Students should reflect on their own learning to promote their critical thinking, monitor, and report on their learning in general, assess their progress toward specific goals, and finally, think about and assess the resources, methods, and strategies they use. They have to do up to 5 weekly reflections, a midterm reflection, and a final one.

Methodology

Study Design and Instrumentation

This study follows the mixed-methods research methodology. As a result, quantitative and qualitative data were collected simultaneously since having both of them extends our understanding of the topic (Creswell, 2002). Random sampling was not done before data collection since the study was conducted at a time of the semester where the teachers had already chosen their medium for doing the portfolio (i.e., paper-based and online versions). Due to the fact that there was no priority between the two sets of quantitative and qualitative data and both were collected at the same time, the researcher followed the convergent parallel design of mixed methods (Creswell, 2002). Methodological triangulation was put to practice by employing questionnaires and sample informant interviews. The student questionnaire consisted of a quantitative section where students express their ideas on a Likert scale. To enrich the exploited data from the questionnaire, an extra comment section complemented this tool.

The quantitative data obtained from the interviews with teachers were recorded in the form of voice and after the interviews, they were transcribed by the researcher.  The manual coding was preferred by the researcher. Using the coding steps proposed by Tesch (1990) and Creswell (2007), the data were coded and were eventually summarized into several themes (Creswell, 2002) based on different theme-making techniques proposed by Ryan and Bernard (2003). For the third and fifth research questions, the qualitative data were utilized directly whereas to answer the 4^th and the main research question, the data from interviews were quantified for the main comparison between teachers and students. Figure 1 illustrates the interrelation between the two sets of data in more detail.

Figure1. The Interrelation between Qual. and Quan. Data

Participants

The participants of the study were the students and the teachers who had portfolio elements in their curriculum in the English language courses in level 450 in the Foundation Programme of the Sciences Department at SQU. Along with the other courses in the Foundation, this course is also supposed to prepare students to enter their colleges after obtaining the required skills in English, math, and IT. The students are either placed at this level through placement test at the beginning of the semester or they are the ones passing previous levels and reaching level 4 which is considered as intermediate English learners. Seventy-eight students participated in the survey, 59 of which were male and 19 of them were female students. Sixty-four of the participants used e-Portfolios in their classes, while 14 were the minority who did the paper-based. Additionally, 9 teachers were interviewed to supplement the results with authentic and qualitative data since having both qualitative and quantitative data ensures a better understanding of the research problem (Creswell, 2002). Three of the interviewees were male teachers and the rest 6 were female colleagues. Seven of these teachers used the electronic platform, while only 2 had decided to utilize the paper medium.

Data Collection and Analysis

To collect students’ ideas about the portfolio element in general and the medium of the portfolio, they were sent the link to the questionnaire online. The questionnaire consisted of two parts. In the first part of the questionnaire, the students shared their ideas for each entry in the questionnaire on a Likert scale containing Strongly Disagree, Disagree, Not Sure, Agree, and Strongly Agree. This section of the questionnaire did not ask portfolio-medium-specific questions. The items included in this section generally reflected the goals set by the curriculum for the level.  In the second section, there were two open-ended questions where the students could write about what they liked the most and the least about the portfolio. The questionnaire was uploaded as a link to Google Docs and the students were asked to complete it during their classes (student questionnaire in Appendix1).

Besides, the teachers in charge of the portfolio from both electronic and paper-based groups were interviewed in person in their offices to find out how they really felt about the portfolio in general and the version of the portfolio they were following in their classes in particular. The questions during the interviews were guided in such a way that accorded with the student questionnaire items. The teachers responsible for the portfolio in each section of level 450 were recorded by a voice recorder after their permission (teachers’ interview questions in Appendix 2).

After gathering the information from both groups of students and teachers, we checked how each group of students perceived the benefits of the portfolio components as a whole and what they thought about each version. To do so, we analyzed the quantitative data from the student questionnaires provided by the Google platform. Hence, the researcher decided to do stratified sampling after collecting the data. However, the size of the two groups of students was significantly different after data collection, with 64 ePortfolio users and 14 paper-based ones.

Therefore, sampling was done only on the ePortfolio group in order to avoid biased interpretations. As a result, a sample size of 16 was selected out of the total 64 for the ePortfolio sample. In order to find the advantages and disadvantages of each form of the portfolio, the researcher probed all the students’ answers to the open-ended questions meticulously to find the common ideas and form some themes according to the word repetition technique (Ryan & Bernard, 2003).

Furthermore, the answers to the open-ended questions for the mentors were also analyzed to extract common themes among them using the Key-Words-in-Context (KWIC) method (Ryan & Bernard, 2003) and to find out their overall perception of the portfolio components in the course and also the advantages and disadvantages of each medium. Finally, the obtained data from two groups, students and teachers, were compared to see if there is any significant difference between their perceptions of portfolio components in the Foundation Programme.

To see whether items of the questionnaire are measuring the same trait or factor, the questionnaire data were analyzed by stratified factor analysis because the aspects were clear, and items belonging to each aspect were also clear (Bryman & Cramer, 2011). Bartlett’s Test of Sphericity and Kaiser–Meyer–Olkin measure of sampling adequacy (KMO) were performed first. The results showed that Bartlett’s test of Sphericity was significant, and KMO was greater than 0.7 (closed to 0.8) which was acceptable (Gravetter &Wallnau, 2008). The results revealed appropriateness for proceeding with factor analysis. The analyses of the results are shown in Table 1.

Table 1. Bartlett’s Test of Sphericity & KMO

Principal Components Analysis (PCA) with varimax rotation of orthogonal rotation was conducted for the factor analysis, so the information among factors would not overlap, and factor loadings were easy to explain (Howell, 2010). According to Hair, Black, Babin, and Anderson (2010), items with factor loadings that were less than 0.4 should be deleted. Therefore, none of the items in the scale had to be deleted since the factor loadings for each item were greater than 0.5. The eigenvalue for the only extracted component was greater than 1 which explained 53% of the variance, implying that the validity of the scale was adequate. The Cronbach’s alpha of the items was .890, indicating good reliability.

Results and Discussion

Students’ Views on the Portfolio Components

The statements in the questionnaire asked the students to express their opinions on a five-point Likert scale. However, to get clearer ideas and to reduce the variance among the responses, we narrowed the Likert scale down to three points consisting of Disagree, Not Sure, and Agree. To understand each individual’s perception of the portfolio, all their responses were summed up. Totals between -20 and -4 were considered as Disagree, the ones between -4 and 4 were seen as Not Sure, and the sums between +4 and +20 were marked as Agree. Table 2 summarizes the results.

Table 2. Perception of Portfolio after Stratified Sampling

As shown in Table 2, 70% of the students in the study agree that the portfolio component was advantageous to them. About 23% of them are not quite sure about the benefits of this component and only about 7% believe that it did not do any good.

As far as the first research question is concerned, according to Table 2, this positive attitude toward the portfolio among the students might be due to the fact that the portfolio is generally argued to be personal. As seen in the table, it is found that most of the students participating in the survey support the portfolio element in the course. The finding is in accord with much of the recent literature (Deneen et al., 2018; Dougherty & Coelho, 2017). The results would suggest that the portfolio element has gained acceptance among the students.

To see if the format of the portfolio has an effect on students’ perception, a chi-square test was run to find out whether there is a significant difference between the two groups of students. SPSS results indicated that there was no significant difference between the perceptions of the two groups in terms of the medium of the portfolio at 5% level (p=.166).

The general concern of the second research question was to find out whether or not the medium of the portfolio affects students’ perception. It was found that students were fond of the portfolio regardless of the medium. The interpretation is that there is no practical difference in doing the portfolio on paper or online. The finding accords with the same view in the literature (Driessen et al., 2007; Wanchid & Charoensuk, 2015; Wesel & Prop, 2009). The implication is that students like the portfolio no matter how they do it.

Furthermore, students’ responses to the 2 open-ended questions for each format were analyzed to understand what they liked the least and the most in the portfolio using the word repetition technique of theme making (Ryan & Bernard, 2003). Fifty students responded to these two questions. Figures 2 and 3 summarize the findings in short.

Figure 2. What did you like the most in the portfolio?

Figure 3. What did you like the least in the portfolio?

According to figures 2 and 3, the results indicated that vocabulary logs were the most interesting component of the portfolio for the students, while reflections were the least. The unexpected part of the findings was the high number of the students who commented ‘nothing’ as the most interesting part of the portfolio. This is likely to be due to a lack of interest in participating in the survey.

To find out teachers’ actual ideas, the researcher scrutinized the transcripts of the interviews with the teachers who were responsible for the portfolio element of the course. The researcher looked for phrases or sentences expressing their idea clearly. Consequently, expressions like ‘I am a big supporter.’ and ‘It is working.’ were considered as Agree, ‘I am not sure.’ as neutral (Not sure), and ‘This is not the true meaning of portfolio.’ or ‘It has lost its meaning.’ were seen as Disagree. Table 3 summarizes teachers’ perceptions of the portfolio.

Table 3. Teachers’ Perceptions of the Portfolio

As shown in the table, out of the 9 interviewees, 6 of them disagree that the portfolio was advantageous to the students, 1 was not sure and 2 believed that it was a useful component in the course.

This finding generally shows that teachers in charge of doing portfolios in their classes mostly have a negative perception of it. According to Table 3, two-third of the teachers do not agree with the usefulness of the portfolio. They say ‘It has lost its true meaning.’, ‘It is not achieving any goals.’ and ‘It is not working.’  The teachers describe portfolio as manual labour, mechanical and repetitive. This unexpected result might be due to the fact that teachers are more concerned with the goals of the portfolio rather than mere grades. The results are in contrast with much of the recent literature (Yastibas & Cepik, 2015; Soruç, 2011; Caldwell, 2007). Nonetheless, the generalization seems quite tentative due to the small sample size in the study. This might be an indicator of a need for further investigation of the portfolio in the context under study. The results imply that teachers in the Foundation Programme at SQU are not quite sure if the portfolio is reaching the aims it meant to.

For the last research question and the main goal of the study and to see if there is a significant difference between students’ perception and that of teachers’, a random sample was taken from the students’ group consisting of 11 participants. After that, based on the frequencies of 3 items in the scale, another chi-square test was run. Table 4 summarizes the results of the test.

Table 4. Comparing Teachers’ and Students’ Perception

It is clear that there is a significant difference between students’ and teachers’ views on the portfolio component of the course at the 5% level (p<.05). This unexpected result is a reflection of the big difference between teachers and students on how they perceive elements distinctively. The interpretation of descriptive statistics shows that while most students like the portfolio, many teachers say it has lost its initial meaning. This part of the results is not surprisingly in contrast with that of Yastibas and Cepik (2015), Soruç (2011), and Caldwell (2007). These findings may raise the possibility of not incorporating teachers’ ideas when designing the portfolio. It implies that teachers and students in level 450 Sciences of the Foundation Programme at SQU have thoroughly opposite ideas considering the portfolio element of the course.

The transcripts were analyzed further to see what teachers thought about the advantages and disadvantages of each format of the portfolio. Table 5 illustrates the findings.

Table 5. Pros and Cons of Each Format of Portfolio

The finding shows that each medium of the portfolio has its own pros and cons. The main superiority of the ePortfolio is its computer-based nature which makes it in line with the current technological world. As a result, teachers find it ‘time-saving’, ‘more enjoyable’, and ‘more convenient’. It can be concluded that it is more user-friendly for teachers. However, the results also indicate that the teachers’ main concern with the ePortfolio is copy-pasting and not actual learning. On the other hand, although teachers find paper-based portfolios time-consuming and old-fashioned, some of them claimed this format has its own convenience. They also added that students were at least writing physically which is good for their writing skills. As far as user-friendliness of portfolios is concerned, the results are similar to those of Driessen et al. (2007) and Wesel and Prop (2009). It is obvious that each medium has its own advantages and disadvantages. This explains why some teachers prefer one medium to the other.

Conclusion

The majority of students participating in the study generally perceived the portfolio component of the course positively. Almost 70% of them believe that it was useful and helpful regardless of the format of the portfolio implemented in their classes. Vocabulary logs were the most favourite section in students’ ideas, whereas reflections were the least interesting. On the other hand, the teachers who participated in the study mostly, 6 out of 9, believe that the portfolio component incorporated in the course has lost its true meaning. In other words, they think that it is not working because doing each part becomes repetitive and mechanical after a while. The results showed there is a significant difference between students’ and teachers’ perception of the portfolio where students agree that it is helping them, while teachers see it as a useless element in the course.

Nevertheless, this study is not without its shortcomings. The first limitation of the study was the number of participants involved in the study, especially the teachers. Secondly, the existing Not Sure among the choices made the interpretation difficult. Also, the sampling and interviews could be carried out in a more participant-paced way. There were also some teachers who could not share their ideas because of their duties and busy schedule.

Despite the numerous studies done on the topic, there are not enough studies done in the Persian Gulf to investigate the portfolio users’ ideas, either students or teachers, about the benefits of this component. The topic can be scrutinized in detail with a bigger number of participants. Studies can focus on the medium of the portfolio to see if there is a difference between paper-based and electronic portfolios on a large scale. Some studies can shed light on any probable connection between portfolio accomplishment and students’ test results. An academic portfolio structure based on a need analysis might be another interesting topic for researchers to work on in the future.