Implementation of an Academic Information Chatbot Based on Retrieval-Augmented Generation (RAG) Using LLaMA 3.1
Implementasi Chatbot Informasi Akademik Berbasis Retrieval-Augmented Generation (RAG) Menggunakan LLaMA 3.1
DOI:
https://doi.org/10.21070/ups.9995Keywords:
Academic Chatbot, Retrieval-Augmented Generation, LLaMA 3.1, Groq API, LangGraphAbstract
Providing fast and accurate academic information is a fundamental requirement for universities. However, users often experience difficulties in efficiently searching for specific information on study program websites. This study aims to develop a Retrieval-Augmented Generation (RAG)-based academic information chatbot that is directly integrated with the Informatics Study Program website at Muhammadiyah University of Sidoarjo. The system was built using the LangGraph architecture, ChromaDB vector database, and LLaMA 3.1 model through the Groq API. Testing was conducted using the RAGAS framework to assess the quality of answers and load testing for system performance. The results showed that the system achieved a Context Precision score of 0.86 and a Faithfulness score of 0.77, indicating high relevance and accuracy of answers. In addition, the implementation of the Groq API resulted in an average response latency of 2.11 seconds with a 94% success rate in load testing.
Downloads
References
A. A. Zulfa, T. Ibrahim, and O. Arifudin, “Peran Sistem Informasi Akademik Berbasis Web Dalam Upaya Meningkatkan Efektivitas Dan Efisiensi Pengelolaan Akademik Di Perguruan Tinggi,” J. Tahsinia, vol. 6, no. 1, pp. 115–134, 2025.
S. Naikar and C. Indraji, “AI-Driven Innovation in Information Retrieval: Transforming Library Operations and User Engagement,” Int. Conf. Glob. Perspect. Open Sci. Bridg. Res. Access Innov., no. May, pp. 1–7, 2025, [Online]. Available: http://papers.ssrn.com/abstract_id=5261229
L. Costaner, P. Studi, T. Informatika, F. Ilmu, K. Universitas, and L. Kuning, “Aplikasi Chatbot untuk Layanan Informasi dan Akademik Kampus Berbasis Artificial Intelligence Markup Language ( AIML ),” pp. 291–300, 2020.
F. Purwani et al., “IMPLEMENTASI APLIKASI CHATBOT SEBAGAI MEDIA INFORMASI PADA PENGEMBANGAN SISTEM AKADEMIK UNIVERSITAS ISLAM NEGERI ( UIN ) RADEN FATAH PALEMBANG MENGGUNAKAN ARTIFICIAL INTELLIGENCE MARKUP LANGUAGE,” vol. 1, no. 4, pp. 1–7, 2024.
M. F. Ajiz, M. Faza, S. Ramadan, H. D. Mutia, P. D. Yanuari, and I. Pendahuluan, “Pengembangan Aplikasi Chatbot Informasi Akademik Berbasis Web Menggunakan Metode Artificial Intelligence Markup Language ( AIML ),” vol. 15, no. 2, 2023.
N. Cahyono, “Perancangan Sistem Informasi Penyewaan Alat Outdoor Berbasis Web menggunakan Metode Waterfall pada InOutdoors Rental Sidoarjo,” no. 1, pp. 1–23, 2024.
A. Alvin, R. Robet, and A. Tarigan, “Implementasi chatbot Otomatis Akademik Berbasis Web Menggunakan LLM dan Rule-Based System Studi Kasus : STMIK Time Implementation of Web-Based Academic Automated chatbot Using LLM and Rule-Based System Case Study : STMIK Time,” no. 3, pp. 651–665, 2025, doi: 10.26798/jiko.v9i3.2209.
M. I. T. Khaqiqi and N. H. Harani, “Peningkatan Kinerja Chatbot NLP Asisten : Tinjauan Literatur tentang Metode dan Akurasi dalam Aplikasi Berbasis Percakapan,” vol. 05, no. 01, pp. 50–59, 2024.
D. Apriliani, S. F. Handayani, I. T. Saputra, T. Informatika, and P. H. Bersama, “Implementasi Natural Language Processing ( NLP ) Dalam Pengembangan Aplikasi Chatbot Pada SMK YPE Nusantara Slawi,” vol. 22, no. 4, pp. 1037–1047, 2023.
S. Abedu and A. Abdellatif, “LLM-Based Chatbots for Mining Software Repositories : Challenges and Opportunities,” no. June 2024, 2025, doi: 10.1145/3661167.3661218.
C. Y. Kim, C. P. Lee, and B. Mutlu, “Understanding Large-Language Model (LLM)-powered Human-Robot Interaction,” ACM/IEEE Int. Conf. Human-Robot Interact., no. Llm, pp. 371–380, 2024, doi: 10.1145/3610977.3634966.
S. Vidivelli, M. Ramachandran, and A. Dharunbalaji, “Efficiency-Driven Custom Chatbot Development: Unleashing LangChain, RAG, and Performance-Optimized LLM Fusion,” Comput. Mater. Contin., vol. 80, no. 2, pp. 2423–2442, 2024, doi: 10.32604/cmc.2024.054360.
Milasanti D, “SISTEM CHATBOT BERBASIS LARGE LANGUAGE MODEL (LLM) DANRETRIEVAL AUGMENTED GENERATION (RAG) PADA ARTIKELILMIAH GARUDA KEMDIKBUD,” 2024.
Nu’aeni H, “PENGEMBANGAN CHATBOT UNTUK SISTEM INFORMASIPERPUSTAKAAN DIGITAL (DIGILIB) MENGGUNAKANRETRIEVAL-AUGMENTED GENERATION (RAG),” 2025.
L. R. Hidayat, G. Pasek, S. Wijaya, and R. Dwiyansaputra, “Optimalisasi Layanan Sistem Informasi Mahasiswa dengan Integrasi Telegram : Chatbot Retrieval-Augmented-Generation berbasis Large Language Model (Optimization of Student Information System Services with Telegram Integration : Chatbot Retrieval-Augmented Generation based on Large Language Model),” 2021. [Online]. Available: http://jtika.if.unram.ac.id/index.php/JTIKA/
M. Grusky, “Rogue Scores,” vol. 1, pp. 1914–1934, 2023.
U. States, “Towards Seamless User Query to REST API Conversion,” no. October 2024, 2025, doi: 10.1145/3627673.3680275.
I. Mohr, D. J. Williams, B. Wang, and H. Xiao, “L ATE C HUNKING : C ONTEXTUAL C HUNK E MBED -,” pp. 1–14, 2024.
M. A. Rosid, D. O. Siahaan, and A. Saikhu, “Sarcasm Detection in Indonesian-English Code-Mixed Text Using Multihead Attention-Based Convolutional and Bi-Directional GRU,” no. June, pp. 137063–137079, 2024.
Z. Nussbaum, J. X. Morris, and B. Duderstadt, “Nomic Embed : Training a Reproducible Long Context Text Embedder,” vol. 002, pp. 1–17, 2025.
T. Chatbot, “Implementation of Retrieval-Augmented Generation ( RAG ) and Large Language Models ( LLM ) for a Document and Tabular-Based Chatbot System,” pp. 19–23, 2025.
J. Wang and Z. Duan, “Agent AI with LangGraph: A Modular Framework for Enhancing Machine Translation Using Large Language Models,” 2024, [Online]. Available: http://arxiv.org/abs/2412.03801
S. Es, J. James, L. Espinosa-anke, and S. Schockaert, “RAGA S : Automated Evaluation of Retrieval Augmented Generation,” pp. 150–158, 2024.
Downloads
Additional Files
Posted
License
Copyright (c) 2026 UMSIDA Preprints Server
This work is licensed under a Creative Commons Attribution 4.0 International License.
