Hobert, Sebastian
How Are You, Chatbot? Evaluating Chatbots in Educational Settings – Results of a Literature Review Proceedings Article
In: Pinkwart, Niels; Konert, Johannes (Hrsg.): DELFI 2019, S. 259–270, Gesellschaft für Informatik e.V., Bonn, 2019, ISBN: 978-3-88579-691-6.
Abstract | Links | BibTeX | Schlagwörter: A, chatbots, evaluation, pedagogical conversational agents, technology-enhanced learning
@inproceedings{Hobert2019,
title = {How Are You, Chatbot? Evaluating Chatbots in Educational Settings – Results of a Literature Review},
author = {Sebastian Hobert},
editor = {Niels Pinkwart and Johannes Konert},
url = {https://dx.doi.org/10.18420/delfi2019_289},
doi = {10.18420/delfi2019_289},
isbn = {978-3-88579-691-6},
year = {2019},
date = {2019-09-16},
booktitle = {DELFI 2019},
pages = {259–270},
publisher = {Gesellschaft für Informatik e.V.},
address = {Bonn},
abstract = {Evaluation studies are essential for determining the utilization of technology-enhanced learning systems. Prior research often focuses on evaluating specific factors like the technology adoption or usability aspects. However, it needs to be questioned if evaluating only specific factors is appropriate in each case. The aim of this research paper is to outline which methods are suited for evaluating technology-enhanced learning systems in interdisciplinary research domains. Specifically, we focus our analysis on pedagogical conversational agents – i.e. learning systems that interact with learners using natural language. For instance, in addition to technology acceptance, further factors like learning success are more important in this case. Based on this assumption, we analyze the current state-of-the-art literature of pedagogical conversational agents to identify evaluation objectives, procedures and measuring instruments. Afterward, we use the results to propose a guideline for evaluations of pedagogical conversational agents.},
keywords = {A, chatbots, evaluation, pedagogical conversational agents, technology-enhanced learning},
pubstate = {published},
tppubtype = {inproceedings}
}
Evaluation studies are essential for determining the utilization of technology-enhanced learning systems. Prior research often focuses on evaluating specific factors like the technology adoption or usability aspects. However, it needs to be questioned if evaluating only specific factors is appropriate in each case. The aim of this research paper is to outline which methods are suited for evaluating technology-enhanced learning systems in interdisciplinary research domains. Specifically, we focus our analysis on pedagogical conversational agents – i.e. learning systems that interact with learners using natural language. For instance, in addition to technology acceptance, further factors like learning success are more important in this case. Based on this assumption, we analyze the current state-of-the-art literature of pedagogical conversational agents to identify evaluation objectives, procedures and measuring instruments. Afterward, we use the results to propose a guideline for evaluations of pedagogical conversational agents.