A Development of an AI Chatbot for TU Library FAQ Using ChatGPT Technology to Enhance Reference Services at Thammasat University Library

Abstract

             This study aims to develop and evaluate the performance of an AI chatbot system, MyGPTs: TU Library FAQ, built on the ChatGPT platform and designed to provide after-hours reference services at Thammasat University Library. The system was developed using an iterative model, focusing on questions related to library hours, basic services, and information resource retrieval. Performance evaluation was conducted in two rounds by experts in library reference services.
       In the initial development phase, a plain-text knowledge base was employed. Testing results indicated that while the system could answer basic questions, it exhibited limitations in data accuracy and comprehensiveness. Subsequently, the knowledge base was redesigned using the Retrieval-Augmented Generation (RAG) approach, with service information organized in tabular format through Airtable. The post-improvement evaluation revealed significant enhancements in system performance and consistency, with average scores ranging from 4.51 to 5.00, placing the system within the excellent range. User satisfaction was rated as good (average scores of 4.00–4.29), particularly in terms of interaction convenience and information accuracy. However, certain limitations remained, such as outdated information on late return fees for Library of Things equipment and expired links to online databases.
       The findings demonstrate the potential of AI chatbots in supporting after-hours library reference services and confirm that the choice of an appropriate knowledge base structure directly impacts system performance. Future development should focus on expanding access channels, implementing vector databases and semantic search systems, and supporting multilingual capabilities to better serve diverse user needs.