Publikasi Jurnal

Daftar artikel yang ditulis oleh Sivitas Akademik Universitas Komputer Indonesia dan telah dipublikasikan di berbagai Jurnal Nasional dan Internasional


2026 2025 2024 2023 2022 2021 2020 2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000

Data Publikasi Jurnal - Tahun 2026

Menampilkan 2 dari 2 data

Klik pada judul artikel untuk melihat detail data

Evaluating Gemini And Gemma Language Models For Indonesian Text-To-Sql Tasks
Author

Galih Hermawan, S.Kom, M.T Dr. Ednawati Rainarli, S.Si., M.Si.

Abstrak
The Text-to-SQL project converts plain human language to SQL to make databases more accessible to everyone, but there are few Indonesian resources; this research provides an Indonesian Text-to-SQL benchmark and tests modern language models. We developed a benchmark of 60 queries across six complexity types on two databases (Sakila and a cooking Resep database) and tested four models - Gemini 2.5 Flash, Gemini 2.5 Flash Lite, Gemma-3-27B-IT, and the locally run Gemma-3n-E2B, with measures of execution success, row-match accuracy, speed (ms), and token usage. The model (Gemini 2.5 Flash) achieved 100% execution success and 95% row-match correctness, with an average response time of 4,707 ms. With a moderate accuracy drop to 75%, Flash Lite reduced the delay to 1,249 ms. With 5,684 ms, Gemma-3-27B-IT achieved 81.7-82% of its accuracy. With a latency of 159,818 ms, the CPU only Gemma-3n-E2B achieved 21.7-22% accuracy. Models hosted on the cloud are highly accurate at interactive delays, although small local models running on the CPU are not. We conclude that Gemini 2.5 Flash is the most reliable overall, while Flash Lite offers a practical speed-accuracy compromise. To support reproducibility and further research impact, we release the benchmark and evaluation pipeline and outline extensions (conversational contexts, retrieval-augmented prompting, quantization, and caching) to improve the viability of lightweight local deployments.
Nama Jurnal

Journal of Engineering Science and Technology
Volume 21 Nomor 2

URL

https://jestec.taylors.edu.my/Special%20Issue%20INCITEST%202025/INCITEST2025_02.pdf

DOI

-