Retrieved Augmented Generation (RAG)-Based Academic Chatbot for the Informatics Engineering Study Program, FTI UNISSULA

Abstract

Fast, accurate, and reliable access to academic information remains a major challenge for Informatics Engineering students at the Faculty of Industrial Technology, Sultan Agung Islamic University (UNISSULA). Educational information found in various official documents is often difficult to find quickly, leading to confusion and dependence on external news that may be inaccurate. This research aims to create and implement an academic chatbot that uses Retrieval-Augmented Generation (RAG) technology to provide contextually relevant academic information based on official documents. The research method includes collecting and pre-processing official academic documents, dividing them into smaller parts, generating vectors using the Sentence-BERT model, and storing the vectors in the FAISS database to enable meaningful information retrieval. The RAG mechanism combines relevant document search results with the ability of a large language model (LLM) to generate accurate answers. Evaluation results show that the chatbot successfully answered all questions with a 100% success rate. Tests using the ROUGE-1 and BLEU-4 metrics showed excellent results in handling questions from the FAQ category, as well as demonstrating fairly good meaning relevance for non-FAQ questions, despite differences in answer inference. These findings indicate that the RAG approach successfully improves the accuracy, reliability, and context of chatbot answers in the academic field, as well as assisting in the development of intelligent academic information systems in higher education environments.