Mapping the Efficacy of Artificial Intelligence-based Online Proctored Examination (OPE) in Higher Education during COVID-19: Evidence from Assam, India
Abstract
The COVID-19-induced lockdown forced education institutes to use different synchronous and asynchronous assessment techniques. The present study is a case investigation of the Artificial Intelligence-based Online Proctored Examination (OPE) adopted by Nowgong College (Autonomous), Assam, India, during the COVID-19 pandemic. It follows a hybrid approach to understand the OPE mechanisms; to consider challenges faced therein; to assess its efficacy based on students’ perception and satisfaction; and finally, to ascertain the perceived issues and concerns of students. Information from multiple sources was gathered including semi-structured interview with representatives of the exam controlling body, document analysis, observation and a survey of 209 randomly selected students; different parameters of OPE, constructed as per the attributes of Rogers’ Diffusion of Innovation Theory (DOI), were considered. The qualitative data were analysed thematically, while the quantitative data were analysed according to frequency, percentage, mean, sd and one sample t-test. The results revealed that additional costs, lack of compatible devices, low bandwidth in rural areas, technical inability of students are the major challenges in implementing OPE successfully. Students’ perception of OPE was found to be significantly positive and the sampled students were significantly satisfied with their overall experiences in OPE. However, students were found to be significantly negative and reluctant about its future use, as they experienced various technical and financial issues, and were concerned about the environmental and psychological aspects. Thus, the insights from this study yield significant implications for stakeholders, which could assist in ensuring a robust and scalable OPE in the future.
https://doi.org/10.26803/ijlter.21.9.5
Keywords
Full Text:
PDFReferences
Apampa, K. M., Wills, G., & Argles, D. (2011). Towards a blob-based presence verification system in summative e-assessments. International Journal of E-Assessment, 1(1), 1–16. https://ijea.org.uk/index.php/journal/article/view/9
Ashri, D., & Sahoo, B. P. (2021). Open book examination and higher education during COVID-19: Case of University of Delhi. Journal of Educational Technology Systems, 50(1), 73–86. https://doi.org/10.1177/0047239521013783
Attia, M. A. (2014). Postgraduate students’ perceptions toward online assessment: The case of the faculty of education, Umm Al-Qura University. Education for a Knowledge Society in Arabian Gulf Countries (International Perspectives on Education and Society, 24, 151–173. https://doi.org/10.1108/S1479-367920140000024015
Banerjee, A. (2020, June 12). The perils of online education in India. Frontier-An Independent Weekly Since 1968. https://www.frontierweekly.com/views/jun-20/12-6-20 The%20perils%20of%20online%20education%20in%20India.html
Bartley, J. M. (2005). Assessment is as assessment does: A conceptual framework for understanding online assessment and measurement. In S. Howell, & M. Hricko (Eds.), Online Assessment and Measurement: Foundations and Challenges (pp. 1–45). IGI Global. https://doi.org/10.4018/978-1-59140-720-1.ch001
Berkey, D., & Halfond, J. (2015, July 20). Cheating, student authentication and proctoring in online programs. New England Board of Higher Education. http://nebhe.org/journal/cheating-student-authentication-and-proctoring-in-online-programs/
Boitshwarelo, B., Reedy, A. K., & Billany, T. (2017). Envisioning the use of online tests in assessing twenty-first century learning: A literature review. Research and Practice in Technology Enhanced Learning, 12(16), 1–16. https://doi.org/10.1186/s41039-017-0055-7
Bowen, G (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40. https://doi.org/10.3316/QRJ0902027
Brady, A. M. (2005). Assessment of learning with multiple-choice questions. Nurse Education in Practice, 5(4), 238–242. https://doi.org/10.1016/j.nepr.2004.12.005
Cahapay, M. B. (2020). Reshaping assessment practices in a Philippine teacher education institution during the Coronavirus disease 2019 crisis. Pedagogical Research, 5(4), em0079. https://doi.org/10.29333/pr/8535
Corrigan-Gibbs, H., Gupta, N., Northcutt, C., Cutrell, E., & Thies, W. (2015). Deterring cheating in online environments. ACM Transactions on Computer-Human Interaction, 22(6), 1–23. https://doi.org/10.1145/2810239
Crisp, G., Guardia, L., & Hilier, M. (2016). Using e-Assessment to enhance student learning and evidence learning outcomes. International Journal of Educational Technology in Higher Education, 13(18) 1–3. https://doi.org/10.1186/s41239-016-0020-3
Das, S. (2020, August 6). Digital divide biggest scourge in online study. The Pioneer. https://www.dailypioneer.com/2020/state-editions/digital-divide-biggest-scourge-in-online-study.html
De Villiers, R., Scott-Kennel, J., & Larke, R. (2016). Principles of effective e-assessment: A proposed framework. Journal of International Business Education, 11(2016), 65–92. https://neilsonjournals.com/JIBE/abstractjibe11devilliersetal.html
Donovan, J., Mader, C., & Shinsky, J. (2007). Online vs. traditional course evaluation formats: Student perceptions. The Journal of Interactive Online Learning, 6(3), 158–180. https://ncolr.org/issues/jiol/v6/n3/online-vs-traditional-course-evaluation-formats-student-perceptions.html
Drew, H. (2020, April 1). Mass school closures in the wake of the coronavirus are driving a new wave of student surveillance. The Washington Post. https://washingtonpost.com/technology/2020/04/01/online-proctoring-college-exams-coronavirus/
D’Souza, K. A., & Siegfeldt, D. V. (2017). A conceptual framework for detecting cheating in online and take-home exams: Conceptual framework for detecting cheating in exams. Decision Sciences Journal of Innovative Education, 15(4), 370–391. https://doi.org/10.1111/dsji.12140
Elsalem, L, Al-Azzam, N., Jum’ah, A. A., & Obeidat, N. (2021). Remote E-exams during Covid-19 pandemic: A cross-sectional study of students’ preferences and academic dishonesty in faculties of medical sciences. Annals of Medicine and Surgery, 62(February), 326–333. https://doi.org/10.1016/j.amsu.2021.01.054
Gathuri, J. W., Luvanda, A., Matende, S., & Kamundi, S. (2014). Impersonation challenges associated with e-assessment of university students. Journal of Information Engineering and Applications, 4(7), 60–68. https://iiste.org/Journals/index.php/JIEA/article/view/14289
Gilbert, L., Whitelock, D., & Gale, V. (2011). Synthesis report on assessment and feedback with technology enhancement. Electronics and Computer Science. University of Southampton. https://eprints.soton.ac.uk/273221/1/Synthesis%2520report%2520on%2520assessment%2520and%2520feedback%2520Final%2520Report%2520July%25202011.pdf
Gipps, C. V. (2005). What is the role for ICT?based assessment in universities? Studies in Higher Education, 30(2), 171–180. https://doi.org/10.1080/03075070500043176
Hillier, M. (2014). The very idea of e-Exams: student (pre)conceptions. In B. Hegarty, J. McDonald, & S. K. Loke (Eds.), Rhetoric and Reality: Critical perspectives on educational technology, Proceedings ascilite Dunedin (pp. 77–88). https://core.ac.uk/download/pdf/43365108.pdf
Ilgaz, H., & Adan?r, G. A. (2020). Providing Online exams for online learners: Does it really matter for them? Education and Information Technologies, 25(March 2020), 1255–1269. https://doi.org/10.1007/s10639-019-10020-6
IMAI (2019). India Internet 2019. Internet and Mobile Association of India. https://cms.iamai.in/Content/ResearchPapers/d3654bcc-002f-4fc7-ab39-e1fbeb00005d.pdf
Isaias, P., & Issa, T. (2013). E-learning and sustainability in higher education: an international case study. The International Journal of Learning in Higher Education, 20(4), 77–90. https://espace.curtin.edu.au/bitstream/handle/20.500.11937/13152/194083_99461_E-Learning___Sustainability_in_Higher_Education.pdf?sequence=2&isAllowed=y
Jha, A. (2020, August 17). Issues facing online education. Hindustan Times. https://hindustantimes.com/india-news/issues-facing-online-education/story-SaG9rbmlRjRnALWqPspjII.html
Karim, M. N., Kaminsky, S. E., & Behrend, T. S. (2014). Cheating, reactions, and performance in remotely proctored testing: An exploratory experimental study. Journal of Business and Psychology, 29(December), 555–572. https://doi.org/10.1007/s10869-014-9343-z
Khan, R. A, & Jawaid, M. (2020). Technology Enhanced Assessment (TEA) in COVID 19 Pandemic. Pakistan Journal of Medical Sciences, 36(COVID19-S4), COVID19-S108-S110 https://doi.org/10.12669/pjms.36.COVID19-S4.2795
Kharbat, F. F., & Daabes, A. S. A. (2021). E-proctored exams during the COVID-19 pandemic: A close understanding. Education and Information Technologies, 26, 6589–6605. https://doi.org/10.1007/s10639-021-10458-7
Kocdar, S., Karadeniz, A., Peytcheva-Forsyth, R., & Stoeva, V. (2018). Cheating and plagiarism in e-assessment: Students’ perspectives. Open Praxis, 10(3), 221–235. https://doaj.org/article/3e05e6677c834657b3069ed455427e64
Kotler, P., & Keller, K. L. (2006). Marketing Management. Prentice Hall.
Kubiatko, M. (2020). Are ICT being used correctly? Small reflection about correct using of ICT in education. Problems of Education in the 21st Century, 75(1), 4–5. https://doi.org/10.33225/pec/17.75.04
Kundu, A., & Bej, T. (2020). Experiencing e-assessment during COVID-19: An analysis of Indian students’ perception. Higher Education Evaluation and Development, 15(2), 114–134. https://doi.org/10.1108/HEED-03-2021-0032
Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz, A., & Yovkova, B. (2018). Addressing cheating in e-assessment using student authentication and authorship checking systems: Teachers’ perspectives. International Journal for Educational Integrity, 14(2), 1–21. https://doi.org/10.1007/s40979-018-0025-x
Milone, A. S., Cortese, A. M., Balestrieri, R. L., & Pittenger, A. L. (2017). The impact of proctored online exams on the educational experience. Currents in Pharmacy Teaching & Learning, 9(1), 108–114. https://doi.org/10.1016/j.cptl.2016.08.037
National Statistical Office (NSO) (2019, November). Key indicators of household social consumption on education in India, NSS 75th round (July 2017 to June 2018). Ministry of Statistics and Programme Implementation, Govt. of India. https://ruralindiaonline.org/en/library/resource/key-indicators-of-household-social-consumption-on-education-in-india-nss-75th-round-july-2017-june-2018/
Nicol, D. (2007). E-assessment by design: using multiple-choice tests to good effect. Journal of Further and Higher Education, 31 (1), 53–64. https://doi.org/10.1080/03098770601167922
Nie, D., Panfilova, E., Samusenkov, V., & Mikhaylov, A. (2020). E-learning financing models in Russia for sustainable development. Sustainability, 12(11), 2–14. https://doi.org/10.3390/su12114412
Northcutt, C. G., Ho, A. D., & Chuang, I. L. (2016). Detecting and preventing ‘multiple-account’ cheating in massive open online courses. Computers & Education, 100(September), 71–80. https://doi.org/10.1016/j.compedu.2016.04.008
Pedersen, C., White, R., & Smith, D. (2012). Usefulness and reliability of online assessments: A Business Faculty’s experience. International Journal of Organisational Behaviour, 17(3), 33–45. https://eprints.usq.edu.au/22439/5/Pedersen_White_Smith_IJOB_v17n3_PV.pdf
Rahman, A. (2021). Using students’ experience to derive effectiveness of COVID-19-lockdown-induced emergency online learning at undergraduate level: Evidence from Assam, India. Higher Education for the Future, 8(1), 71–89. https://doi.org/10.1177/2347631120980549
Raman, R., B, S., G, V., Vachharajani, H., & Nedungadi, P. (2021). Adoption of online proctored examinations by university students during COVID?19: Innovation diffusion study. Education and Information Technologies, 26, 7339–7358. https://doi.org/10.1007/s10639-021-10581-5
Ridgway, J., Mccusker, S., & Pead, D. (2004). Literature Review of E-Assessment. Futurelab Series Report 10. https://telearn.archives-ouvertes.fr/hal-00190440
Rogers, E. M. (2003). Diffusion of Innovations (5th ed.). Free Press.
Rowe, N. C. (2004). Cheating in online student assessment: Beyond plagiarism. Online Journal of Distance Learning Administration, 7 (2), 1–10. https://westga.edu/~distance/ojdla/summer72/rowe72.html
Sorensen, E. (2013). Implementation and student perceptions of e-assessment in a Chemical Engineering module. European Journal of Engineering Education, 38(2), 172–185. https://doi.org/10.1080/03043797.2012.760533
Stone, D. E., & Zheng, G. (2014). Learning management systems in a changing environment. In V. C. X. Wang (Ed.), Handbook of research on education and technology in a changing society (pp. 756–767). IGI Global. https://doi.org/10.4018/978-1-4666-6046-5
Weiner, J. A., & Hurtz, G. M. (2017). A Comparative study of online remote proctored versus onsite proctored high-stakes exams. Journal of Applied Testing Technology, 18(1), 13–20. https://jattjournal.com/index.php/atp/article/view/113061/79315
Whitelock, D. M., & Brasher, A (2006). Developing a Roadmap for e-Assessment: Which Way Now? In M. Danson (Ed.), Proceedings of the 10th CAA International Computer Assisted Assessment Conference (pp. 487–501). Loughborough, UK: Professional Development. http://oro.open.ac.uk/11950/1/CAA_Conf_06_roadmap_paperfv.pdf
Refbacks
e-ISSN: 1694-2116
p-ISSN: 1694-2493