Abstract:
Sri Lankan sign language (SSL) is a visualgestural
language used by the deaf community for
communication. Hearing-impaired people cannot effectively
communicate with a normal person due to the difficulty in
understanding sign language. SSL to Sinhala text interpreting
technology using gesture recognition helps to fill up this
communication gap since Sinhala is the majority language used
in Sri Lanka. Hand gesture recognition can be achieved by using
vision-based or sensor-based approaches. Vision-based
approaches are comparatively simple and less costly but sensorbased
approaches are complex. Scale, rotation, occlusion affects
the accuracy of gesture recognition, and keypoints act as better
features to handle them. The research focuses on a combined
approach of convolutional neural network (CNN) and Scale
Invariant Feature Transform (SIFT) to develop a camera-based
low-cost solution to interpret static gestures of SSL into Sinhala
text. The SSL to Sinhala text translation model reached an
accuracy of 86.5% when a dataset of images of 20 static SSL
gestures was used. The classifier showed robustness to scale
variations when the distance to the camera was varied and
uniform color backgrounds were used. Further improvements
can be done for the recognition of dynamic gestures and facial
expressions of SSL.
