Sistem Pendeteksi Bahasa Isyarat SIBI Menggunakan LSTM Berbasis OpenCV dan MediaPip

Ahmad Sarjianto; Melladia Melladia

doi:10.33884/jif.v14i01.10681

Authors

Ahmad Sarjianto Universitas Nahdlatul Ulama Sumatera Barat
Melladia Melladia Universitas Nahdlatul Ulama Sumatera Barat

DOI:

https://doi.org/10.33884/jif.v14i01.10681

Keywords:

SIBI Sign Language Translation System, LSTM, OpenCV, MediaPip

Abstract

This study aims to develop a Sign Language Translation System which is specifically for the Indonesian Sign Language System (SIBI) based on artificial intelligence (AI) and computer vision which aims to help communication between deaf/mute people and the general public. using the Long Short-Term Memory (LSTM) method, taking important data from Sign Language hand movements and combined with OpenCV and MediaPip. This system is designed with a web-based interface that will display translations in text form in real-time. The testing was conducted on a dataset consisting of SIBI alphabets and basic words, with the highest accuracy reaching 0.85 or 85% for basic words, and 0.45 or 45% for alphabet recognition.In conclusion, this research produced a system capable of automatically translating sign language by utilizing web technology for the interface, and OpenCV, MediaPip, and Long Short-Term Memory (LSTM) for the translation process.This system has great potential to reduce communication barriers between the general public and individuals with hearing or speech impairments, although further development is required to improve its accuracy.

References

I. B. A. Peling, I. M. P. A. Ariawan, and G. B. Subiksa, “Deteksi Bahasa Isyarat Menggunakan Tensorflow Lite dan American Sign Language (ASL),” J. Krisnadana, vol. 3, no. 2, pp. 90–100, 2024, doi: 10.58982/krisnadana.v3i2.534.

Kemenkes, “Buku Saku.Pdf,” 2020. [Online]. Available: https://dinkes.jatimprov.go.id/userimage/dokumen/Buku Saku.pdf

A. Widya Agata, W. S J Saputra, and C. Aji Putra, “Pengenalan Bahasa Isyarat Indonesia (Bisindo) Menggunakan Algoritma Scale Invariant Feature Transform (Sift) Dan Convolutional Neural Network (Cnn),” JATI (Jurnal Mhs. Tek. Inform., vol. 8, no. 1, pp. 1054–1061, 2024, doi: 10.36040/jati.v8i1.8917.

O. D. Nurhayati, D. Eridani, and M. H. Tsalavin, “Sistem Isyarat Bahasa Indonesia (SIBI) Metode Convolutional Neural Network Sequential secara Real Time,” J. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 4, pp. 819–828, 2022, doi: 10.25126/jtiik.2022944787.

N. Anam, “Sistem Deteksi Simbol Pada Sibi (Sistem Isyarat Bahasa Indonesia) Menggunakan Mediapipe Dan Resnet-50,” 2022, [Online]. Available: https://repository.dinamika.ac.id/id/eprint/6259/

I. N. T. A. Putra, K. S. Kartini, Y. K. Suyitno, I. M. Sugiarta, and N. K. E. Puspita, “Penerapan Library Tensorflow, Cvzone, dan Numpy pada Sistem Deteksi Bahasa Isyarat Secara Real Time,” J. Krisnadana, vol. 2, no. 3, pp. 412–423, 2023, doi: 10.58982/krisnadana.v2i3.335.

E. Tikasni, E. Utami, and D. Ariatmanto, “Analisis Akurasi Object Detection Menggunakan Tensorflow Untuk Pengenalan Bahasa Isyarat Tangan Menggunakan Metode SSD,” J. Fasilkom, vol. 14, no. 2, pp. 385–393, 2024, [Online]. Available: https://ejurnal.umri.ac.id/index.php/JIK/article/view/7512

J. T. Santoso, Kecerdasan Buatan & Jaringan Syaraf Buatan, vol. 7, no. 1 SE-Judul Buku. 2021. [Online]. Available: https://penerbit.stekom.ac.id/index.php/yayasanpat/article/view/177

Kusnawati, “Analisa Dan Perancangan Sistem,” Karakteristik Sist., vol. 2, no. 18, pp. 1–10, 2021, [Online]. Available: www.unpam.ac.id

A. Zein et al., “Putra, Jan Wira Gotama. (2020). Pengenalan Pembelajaran Mesin dan Deep Learning.,” J. Stud. Alquran dan Tafsir, vol. 4, no. 1, pp. 29–38, 2020, [Online]. Available: https://jurnal.pranataindonesia.ac.id/index.php/jik/article/download/96/49

A. Rozani, “Penerapan Metode Jaringan Syaraf Tiruan Pada Aplikasi Pengenalan Bahasa Isyarat Abjad Jari,” J. Mhs. Tek. Inform., vol. 1, no. 1, pp. 311–317, 2017.

F. N. Hasanah, Buku Ajar Rekayasa Perangkat Lunak. 2020. doi: 10.21070/2020/978-623-6833-89-6.

I. F. A. Melladia, “Perancancangan Sistem Penanganan Penyakit Tanaman Padi Menggunakan Metode Case Based Reasoning,” vol. 4, no. 1, pp. 50–57, 2024, doi: 10.54259/satesi.v4i1.2948.

M. S. Ummah, kelasifikasisistem isyarat bahasa indonesia(SIBI)mengunkan Computer vision dan Deep Learning, vol. 11, no. 1. 2019. [Online]. Available: http://scioteca.caf.com/bitstream/handle/123456789/1091/RED2017-Eng-8ene.pdf?sequence=12&isAllowed=y%0Ahttp://dx.doi.org/10.1016/j.regsciurbeco.2008.06.005%0Ahttps://www.researchgate.net/publication/305320484_SISTEM_PEMBETUNGAN_TERPUSAT_STRATEGI_MELESTARI

F. Marpaung, F. Aulia, and R. C. Nabila, Computer Vision Dan Pengolahan Citra Digital. 2022. [Online]. Available: www.pustakaaksara.co.id

A. R. Ardiansyah, A. H. Nur’azizan, and R. Fernandis, “Implementasi Deteksi Bahasa Isyarat Tangan Menggunakan OpenCV dan MediaPipe,” Stain. (Seminar Nas. Teknol. Sains), vol. 3, no. 1, pp. 331–337, 2024.

Agus Nugroho, “Deteksi Bahasa Isyarat Bisindo Menggunakan Metode Machine Learning,” J. Process., vol. 18, no. 2, pp. 152–158, 2023, doi: 10.33998/processor.2023.18.2.1380.

A. Ma’arif, “Buku Ajar Pemrograman Lanjut Bahasa Pemrograman Python Oleh : Alfian Ma ’ Arif,” Univ. Ahmad Dahlan, p. 62, 2020, [Online]. Available: http://eprints.uad.ac.id/32743/1/buku python.pdf

I. J. Thira, D. Riana, A. N. Ilhami, B. R. S. Dwinanda, and H. Choerunisya, “Pengenalan Alfabet Sistem Isyarat Bahasa Indonesia (SIBI) Menggunakan Convolutional Neural Network,” J. Algoritm., vol. 20, no. 2, pp. 421–432, 2023, doi: 10.33364/algoritma/v.20-2.1480.

R. Forest, “Hand Sign Recognition of Indonesian Sign Language System ( SIBI ) Using,” vol. 5, no. 158, pp. 258–265, 2025.