Conversational agent as a support tool for learning C Language
DOI:
https://doi.org/10.29059/cienciauat.v19i1.1879Keywords:
conversational agent, usability, learning, teachingAbstract
A conversational agent is a software system that emulates communication with a human being through text or voice messages. The objective of this work was to evaluate the functionality and usability (effectiveness, efficiency, satisfaction) of a conversational agent called Codebot, developed at the Faculty of Information Sciences, as a support tool to facilitate the learning of C programming language by students of the Computer Systems Engineering program, as well as their experience and hedonic (social) quality of the tool. The scope of the study was descriptive, with quantitative analysis. The data collection was cross-sectional with convenience samples of different sizes. The System Usability Scale (SUS) was used, which was extended to evaluate three aspects: effectiveness and efficiency of the bot, as well as user satisfaction. The reliability of the instrument was 0.93 Cronbach’s alpha. The behavior of the agent in the final functional test showed 70 % of successful responses and 30 % of unrecognized requests due to two main reasons: 1) the requested information was not in the agent’s knowledge base, 2) poorly formulated or meaningless requests. Desing and implementation of the conversational agent Codebot obtained a robust database to improve learning of the C language and its function was satisfactory. However, is necessary to continue with tests that include a larger number of samples that delve deeper into the characteristics regarding the creation of the conversational agent.
References
Abdullah, A. (2021). Intelligent college enquiry chatbot using tensorflow. International Research Journal of Engineering and Technology, 08(01).
Ait-Baha, T., El-Hajji, M., Es-Saady, Y., & Fadili, H. (2022). Towards highly adaptive Edu-Chatbot. Procedia Computer Science, 198(2022), 397-403. https://doi.org/10.1016/j.procs.2021.12.260
Allouch, M., Azaria, A., & Azoulay, R. (2021). Conversational Agents: Goals, Technologies, Vision and Challenges. Sensors, 21(24), 8448. https://doi.org/10.3390/s21248448
Alonso-Berenguer, I. (2021). Sistema Básico de Habilidades para la Algoritmización Computacional. Revista de Investigación, Formación y Desarrollo: Generando Productividad Institucional, 9(1). https://doi.org/10.34070/rif.v9i1.255
Ascencio-Jordán, E. y Gil-Mateos, J. E. (2023). Enseñanza de la programación: ¿Nueva didáctica o una didáctica diferente? [En línea]. Disponible en: http://scielo.sld.cu/pdf/rces/v42n3/0257-4314-rces-42-03-18.pdf. Fecha de consulta: 30 de abril de 2024.
Astruga, J. A. (2021). Propuesta metodológica para el análisis y diseño de chatbots basados en texto, en Repositorio Documental. Universidad de Valladolid. [En línea]. Disponible en: https://uvadoc.uva.es/handle/10324/50064. Fecha de consulta: 30 de abril de 2024.
Ayala-Cadena, C. O. y Aguilar-Juárez, I. (2023). La enseñanza de la programación mediante software educativo especializado y los agentes conversacionales. Interfases, 17(017), 170-186. https://doi.org/10.26439/interfases2023.n017.6337
Bal, S., Singh, A. K., Prakash, S., Kumar, S., Ghosh, B., Roy, P., & Mandat, L. (2022). An Intelligent Chatbot for Admission System of an Educational Institute and Prediction of User Interest in Taking Admission. Book Applications of Machine Intelligence in Engineering. CRC Press.
Barreto, I. C. H. C., Barros, N. B. S., Theophilo, R. L., Viana, V. F., Silveira, F. R. V., Souza, O., Sousa, F. J. G., Oliveira, A. M. B., & Andrade, L. O. M. (2021). Development and evaluation of the GISSA Mother-Baby ChatBot application in promoting child health. Ciência & Saúde Coletiva, 26(5), 1679-1690. https://doi.org/10.1590/1413-81232021265.04072021
Brooke, J. (1995). SUS: A quick and dirty usability scale. Usability Eval. Ind., 189. [En línea]. Disponible en: https://www.researchgate.net/publication/228593520_SUS_A_quick_and_dirty_usability_scale. Fecha de consulta: 12 de marzo de 2023.
Caldarini, G., Jaf, S., & McGarry, K. A (2022). Literature Survey of Recent Advances in Chatbots. Information, 13, 41. https://doi.org/10.3390/info13010041
Cameron, G., Cameron, D., Megaw, G., Bond, R., Mulvenna, M., O’Neill, S., Armour, Ch., & McTear, M. (2019). Assessing the Usability of a Chatbot for Mental Health Care. In S. Svetlana, S. Bodrunova, O. Olessia-Koltsova, A. Følstad, H. Halpin, P. Kolozaridi, L. Yuldashev, A. Smoliarova, & H. Niedermayer (Eds.), Internet Science. INSCI 2018. Lecture Notes in Computer Science (pp. 121-132). Springer, Cham. https://doi.org/10.1007/978-3-030-17705-8_11
Cheah, C. S. (2020). Factors Contributing to the Difficulties in Teaching and Learning of Computer Programming: A Literature Review. Contemporary Educational Technology, 12(2), ep272. https://doi.org/10.30935/cedtech/8247
Chimba, L. S. (2020). Diseño de un software educativo para el aprendizaje de programación básica en el estudiantado de primer semestre de la carrera de pedagogía en informática en el periodo 2021-2022. Trabajo de titulación. Universidad central de Ecuador. [En línea]. Disponible en: http://www.dspace.uce.edu.ec/handle/25000/27854. Fecha de consulta: 12 de marzo de 2023.
Corral, R. (2001). El concepto de zona de desarrollo próximo: una interpretación. Revista Cubana de Psicología, 18(1), 72-76.
Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal Education Technology Higher Education, 19(57). https://doi.org/10.1186/s41239-022-00362-6
Fernández-Ferrer, M. (2023). Chatbots en educación. Tendencias actuales y desafíos futuros. Barcelona: LMI, en (Colección Transmedia XXI). [En línea]. Disponible en: https://www.lmi-cat.net/sites/default/files/Chatbots_en_Educacion.pdf. Fecha de consulta: 25 de noviembre de 2023.
Følstad, A. & Brandtzaeg, P. B. (2020). Users’ experiences with chatbots: findings from a questionnaire study. Quality and User Experience, 5(3). https://doi.org/10.1007/s41233-020-00033-2
Fornell-Haugeland, I. K., Følstad, A., Taylor, C., & Bjørkli, C. A. (2022). Understanding the user experience of customer service chatbots: An experimental study of chatbot interaction design. International Journal of Human-Computer Studies, 161, 102788. https://doi.org/10.1016/j.ijhcs.2022.102788
Google-Cloud (2022). Documentación Dialogflow. [En línea]. Disponible en: https://cloud.google.com/dialogflow/docs?hl=es-419. Fecha de consulta: 23 de septiembre de 2022.
Gupta, S. & Chen, Y. (2022). Supporting Inclusive Learning Using Chatbots? A Chatbot-Led Interview Study. Journal of Information Systems Education, 33(1), 98-108. https://jise.org/Volume33/n1/JISE2022v33n1pp98-108.html
Hernández-Pérez, M. y Llull-Céspedes, L. A. (2021). Definición de un proceso ingenieril para el desarrollo de un chatbot a partir de buenas prácticas establecidas. Revista cubana de transformación digital, 3(2), 90-109. https://doi.org/10.5281/zenodo.5546033
Hobert, S. (2019). Say Hello to ‘Coding Tutor’! Design and Evaluation of a Chatbot-Based Learning System Supporting Students to Learn to Program. In J. V. Brocke, A. Mitchell, & B. Tan (Eds.), Proceedings of the International Conference on Information Systems (ICIS). Digital learning environment and future is curriculum (pp. 1-17). AIS eLibrary.
Jiménez-Flores, O. J., Jiménez-Flores, J. C., Gutiérrez-Rojas, Y. V. y Jiménez-Flores, V. J. (2018). Métricas de Evaluación para Chatbots, orientadas a optimizar las experiencias de su uso en las redes sociales. Revista Ciencia y Tecnología, 4(Número Especial), 185-191. https://revistas.ujcm.edu.pe/index.php/rctd/article/view/134/118
Jones, E. A., Jimenez, C. A., Ormeño, P. I. y Poblete, N. A. (2022). Metodologías activas para la enseñanza de programación a estudiantes de ingeniería civil informática. Formación Universitaria, 15(3), 53-60. http://dx.doi.org/10.4067/S0718-50062022000300053
Justo-López, A. C., Aguilar-Salinas, W. E., de-las-Fuentes-Lara, M. y Astorga-Vargas, M. A. (2021). Uso de videos educativos en la materia de programación durante la etapa básica de ingeniería. Formación Universitaria, 14(6), 51-64. http://dx.doi.org/10.4067/S0718-50062021000600051
Lalwani, T., Bhalotia, S., Pal, A., Rathod, V., & Bisen, S. (2018). Implementation of a Chatbot System using AI and NLP. International Journal of Innovative Research in Computer Science & Technology (IJIRCST), 6(3). https://doi.org/10.2139/ssrn.3531782
Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children’ interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189(104576). https://doi.org/10.1016/j.compedu.2022.104576
Long, J., Yuan, M. J., & Lee, H. M. (2019). How to program a chatbot – An introductory project and student perceptions. Issues in Informing Science and Information Technology, 16, 1-31. https://doi.org/10.28945/4282
Mad-Daud, S. H., Ibrahim-Teo, N. H., & Mat-Zain, N. H. (2020). E-JAVA Chatbot for Learning Programming Language: A Post-Pandemic Alternative Virtual Tutor. International Journal of Emerging Trends in Engineering Research, 8(7). https://doi.org/10.30534/ijeter/2020/67872020
Man, S. C., Matei, O., Faragau, T., Andreica, L., & Daraba, D. (2023). The Innovative Use of Intelligent Chatbot for Sustainable Health Education Admission Process: Learnt Lessons and Good Practices. Application Science, 13(2415). https://doi.org/10.3390/app13042415
Mateos-Sánchez, M., Melo, A. C., Blanco, L. S., & García, A. M. F. (2022). Chatbot, as Educational and Inclusive Tool for People with Intellectual Disabilities. Sustainability, 14(1520). https://doi.org/10.3390/su14031520
Mora-Blasco, D. (2021). Desarrollo de un chatbot para la evaluación de la usabilidad y la experiencia de usuario de una plataforma web de ayuda a la decisión médica. Trabajo Fin de Grado, Universitat Politècnica de València, España.
Nandeesh, M., Nagesh, S., Amruthkumar, M., Nikhitha, G. J., & Suma, P. (2021). Real-time AI assistant for educational institutions. International Research Journal of Engineering and Technology (IRJET), 08(06). https://www.irjet.net/archives/V8/i6/IRJET-V8I6769.pdf
Neumann, A. T., Arndt, T., Köbi,s L., Meissner, R., Martin, A., de-Lange, P., Pengel, N., Klamma, R., & Wollersheim, H. W. (2021). Chatbots as a Tool to Scale Mentoring Processes: Individually Supporting Self-Study in Higher Education. Frontiers Artificial Intelligence, 4, 668220. https://doi.org/10.3389/frai.2021.668220
Nguyen, T. T., Le, A. D., Hoang, H. T., & Nguyen, T. (2021). NEU-chatbot: Chatbot for admission of National Economics University. Computers and Education: Artificial Intelligence, 2(100036). https://doi.org/10.1016/j.caeai.2021.100036
Okonkwo, Ch. W. & Ade-Ibijola, A. (2022). Revision-Bot: A Chatbot for Studying Past Questions in Introductory Programming. In International Journal of Computer Science, 49(3).
Olausson, O. (2019). Master Thesis: Department of Computing Science, Umea University, Sweden. [En línea]. Disponible en: https://www.diva-portal.org/smash/get/diva2:1359685/FULLTEXT01.pdf. Fecha de consulta: 20 de abril de 2023.
Olmedo-Carpio, A. (2021). Análisis de Necesidades, Diseño y Desarrollo de un Chatbot para la Universidad de Sevilla. Escuela Politécnica de Sevilla, España.
Pawlik, Ł., Płaza, M., Deniziak, S., & Boksa, E. (2022). A method for improving bot effectiveness by recognizing implicit customer intent in contact centre conversations. Speech Communication, 143(2022), 33-45, https://doi.org/10.1016/j.specom.2022.07.003.
Radziwill, N. M. & Benton, M. C. (2017). Evaluating Quality of Chatbots and Intelligent Conversational Agents. ArXiv, abs/1704.04579. https://doi.org/10.48550/arXiv.1704.04579
Ramírez-García, J. J. (2021). Estudio de la experiencia de usuario en los sistemas de gestión del aprendizaje. IE Revista de Investigación Educativa de la REDIECH, 12(e1358). https://doi.org/ie_rie_rediech.v12i0.1358
Ren, R., Castro, J. W., Acuña, S. T., & de-Lara, J. (2019). Usability of Chatbots: A Systematic Mapping Study. Conference: The 31st International Conference on Software Engineering and Knowledge Engineering, 29(11n12), 1673-1702. https://doi.org/10.18293/SEKE2019-029.
Safitri, F., Hardini, T. I., Setiadi, R., & Mutiarsih, Y. (2021). Usability Measurement: Chatbot as a Pedagogical Support for Learning French Grammar. Proceedings of the Fifth International Conference on Language, Literature, Culture, and Education (ICOLLITE 2021), 595. https://doi.org/10.2991/assehr.k.211119.042.
Soares, J. & de-Freitas, L. (2022). C-BOT: Um protótipo de chatterbot para o ensino de programação. En J. Viterbo-Filho (Ed.), Anais do XXXIII Simpósio Brasileiro de Informática na Educação (pp. 1151-1162). Porto Alegre: SBC. https://doi.org/10.5753/sbie.2022.225711
Suganya, T., Ashwin-Kumar, N. S., & Kavijha, R. (2020). Exploring Chatbot Implementation Methodologies used in Customer Support Industry, in International Journal of Recent Technology and Engineering (IJRTE). [En línea]. Disponible en: https://www.ijrte.org/wp-content/uploads/papers/v8i6/F8895038620.pdf. Fecha de consulta: 23 de septiembre de 2022.
Zhang, J., Oh, Y. J., Lange, P., Yu, Z., & Fukuoka, Y. (2020). Artificial Intelligence Chatbot Behavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Activity and a Healthy Diet: Viewpoint. Journal of medical Internet research, 22(9). e22845. https://doi.org/10.2196/22845
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Universidad Autónoma de Tamaulipas
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
