Mitigating respondent fatigue in self-assessment: CEFR-based items for Malaysian undergraduates
DOI:
https://doi.org/10.24200/jonus.vol9iss2pp474-489Abstract
Background and Purpose: The Common European Framework of Reference for Languages (CEFR) is an internationally recognised instrument for describing language skills. It describes language proficiency on a six-level scale, ranging from A1 for beginners to C2 for those who have already mastered a language. The roadmap in the Malaysian Education Plan 2013-2025 sets B2 as the target for Malaysian higher education graduates as this is the level at which they can expect to get a job and function adequately in English, while graduates of English language degree programmes are expected to reach C1 when they graduate. The aim of this study is to test and verify whether it is appropriate to use only the items from B2, C1 and C2 to measure the language proficiency of Malaysian students to avoid respondent fatigue in answering the questionnaire.
Methodology: A proportionate stratified random sampling method was used and four strata were defined: Strata I and II were male and female students in public institutions, Strata III and IV were male and female students in private institutions. The instrument used was a questionnaire containing demographic profiles and eight items from the CEFR Global Self-Assessment Grid. Exploratory factor analysis and reliability analysis were conducted using SPSS.
Findings: The eigenvalue of 7.263 with factor loadings between 0.774 and 0.854 and the Cronbach’s alpha value for language ability of over 0.9 indicate excellent reliability of the selected items. The study found that Malaysian university students were good at understanding and discussing personal information, events, opinions and plans. However, they found it difficult to understand complex and longer texts, to use language flexibly in social and academic settings and to express themselves without constantly asking for feedback. As the current education system focuses too much on memorization, it does not contribute to the development of practical language skills. In order to improve, students should use the CEFR as a self-assessment tool.
Contributions: The findings of the study contribute to the understanding of the optimization of the CEFR framework in Malaysian higher education by reducing respondent fatigue and improving the quality of self-assessment. The study provides empirical evidence of the effectiveness of using CEFR-based items at B2, C1 and C2 levels in reducing fatigue, improving the quality of self-assessment and informing language teaching and learning practices. The findings can assist educators and policy makers in developing more effective language teaching and assessment strategies that promote student learning outcomes and ultimately improve the language education of Malaysian students.
Keywords: CEFR rubric, ESL, self-assessment, perceived language proficiency, answering fatigue.
References
Ahamat, M. I. A., & Abdullah, N. (2019). Self-efficacy and language learning strategies as determinants of students’ English language proficiency. International Journal of Academic Research in Business and Social Sciences, 9(13), 206–217.
Arezoo, K., Behzad, G., Zargham, G., & Reza, G. (2019). Relationship between students’ gender and their use of politeness strategies in the results and discussions section of PhD dissertations. Dinamika Ilmu, 19(1), 75-95.
Aziz, A. A., & Kashinathan, S. (2021). ESL learners’ challenges in speaking English in Malaysian classroom. International Journal of Academic Research in Progressive Education and Development, 10(2), 983–991.
Chiara, S., & Magali, C. (2019). Soft skills to enhance graduate employability: Comparing students and employers’ perceptions. Studies in Higher Education, 1(2), 1-15.
Christopher, A. A. (2016). Overcoming impediments to learning the four language. Advances in Language and Literary Studies, 7(1), 161–167.
David, M. K. (2004). Language policies in a multilingual nation: Focus on Malaysia. In M.K. David (Ed.), Teaching English in second and foreign language setting (pp. 1-15). Peter Lang.
Fareed, M., Ashraf, A., & Bilal, M. (2016). ESL learners’ writing skills: Problems, factors and suggestions. Journal of Education and Social Sciences, 4(2), 81-92.
Foo, B., & Richards, C. (2004). English in Malaysia. RELC journal, 35(2), 229-240.
Gaudart, H. (1987). English language teaching in Malaysia: A historical account. The English Teacher, 16, 17-36.
Gill, S. K. (2005). Language policy in Malaysia: Reversing direction. Language Policy, 4(3), 241-260.
Hair, J. F., Ringle, C., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. Journal of Marketing Theory and Practice, 19(2), 139-151.
Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2-24.
Hashim, A. (2020). Malaysian English. In K. Bolton, W. Botha, & A. Kirkpatrick (Eds.), The handbook of Asian Englishes (pp. 373-398). Wiley Blackwell.
Hassan, I., Azmi, M. N. L., Muhamad, S. N., & Abdullah, A. T. H. (2021). Reading habits and their correlation with reading achievement among ESL learners in selected Malaysian secondary schools. Arab World English Journal, 12(3), 385-399.
Hazlina, A. (2016, September 28). Raising English language proficiency. News Straits Times. https://www.nst.com.my/news/2016/09/176566/raising-english-language- proficiency
Idris, M., & Abdul Raof, A. H. (2017). The CEFR rating scale functioning: An empirical study on self- and peer assessments. Sains Humanika, 9(4-2), 11-17.
Kashinathan, S., & Abdul Aziz, A. (2021). ESL learners’ challenges in speaking English in Malaysian classroom. International Journal of Academic Research in Progressive Education and Development, 10(2), 983-991.
Kitano, K. (2001). Anxiety in the college Japanese language classroom. Modern Language Journal, 85(4), 549-566.
Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607–610.
Lan, Y.-J. (2020). Immersion into virtual reality for language learning. In K.D. Federmeier & H.-W. Huang (Eds.), The psychology of learning and motivation: Adult and second language learning (pp. 1–26). Elsevier Academic Press.
Livingston, K. (2018). Teachers' professional learning within learning systems. European Journal of Teacher Education, 41(4), 415-417.
MacIntyre, P. D., Baker, S. C., Clément, R., & Donovan, L. A. (2002). Sex and age effects on willingness to communicate, anxiety, perceived competence, and L2 motivation among junior high school French immersion students. Language Learning, 52(3), 537-564.
Mohamed Salleh, R. T. A., Mohammad Lotfie, M., Md. Zamin, A. A., Di Biase, B., & Kawaguchi, S. (2020). Towards a new framework of English language learning in Malaysian preschools. Southeast Asia Early Childhood Journal, 9(2), 144-154.
Onwuegbuzie, A. J., Bailey, P., & Daley, C. (1999). Factors associated with foreign language anxiety. Applied Psycholinguistics, 20(2), 217-239.
Sekaran, U., & Bougie, R. (2016). Research methods for business: A skill-building approach (7th ed.). Wiley & Sons.
Takahashi, A. (2009). Self-perception of English ability: Is it related to proficiency and/or class performance? English, Scovel 1978, 1(1), 39–48.
Thirusanku, J., & Yunus, M. M. (2014). Status of English in Malaysia. Asian Social Science, 10(14), 254-260.
Too, W. K. (2017). English language teaching and policies at the tertiary level in Malaysia. In E.S. Park & B. Spolsky (Eds.), English education at the tertiary level in Asia (pp. 109-129). Routledge.
Tradmin, R. (2021). Malaysian fresh graduates’ performance in job interviews: Politeness strategies. International Journal of Academic Research in Business and Social Sciences, 11(9), 975-990.
Vethamani, M. E. (2007). The ebb and flow of English language education in Malaysia. In M.E. Vethamani & R. Perumal (Eds.), Teaching English in Malaysia: A Special 25th MELTA Anniversary Publication (pp. 1-10). Sasbadi Sdn. Bhd.
Yahaya, A., Yahaya, N., Choon Lean, O., Bon, T. A., & Ismail, S. (2011). Factors contributing to proficiency in English as a second language among Chinese students in Johor Bahru. Elixir Online Journal, 41, 5837–5848.
Yetty, Y. (2018). Politeness strategy on social interaction used by Munanese. ELS Journal on Interdisciplinary Studies on Humanities, 1(1), 59-65.
