AI & Computer Vision Project

Sign Language with Vision

Computer vision system that detects and translates sign language gestures in real time. Uses MediaPipe for hand tracking, OpenCV for video processing and a trained neural network to classify each sign. Built with Python, TensorFlow and Keras.

View featuresTech stack
Reconocimiento de lenguaje de señas con visión artificial
🤟

Real-Time AI

Computer Vision & Deep Learning

The System in Action

Real screenshots of the project running on a standard webcam

Right hand landmarks with detected sign language letters
Hands and signs21 landmarks per hand and the recognized sign on screen: 26 FPS.
MediaPipe face mesh detecting a face in real time
FacesFace mesh fitted to the face frame by frame: 35 FPS.
Real-time object detection: a person and a game controller
ObjectsA person and a game controller detected with bounding boxes: 70 FPS.

Key Features

Key features of the recognition system

✋

Real-Time Hand Detection

MediaPipe tracks up to 21 landmarks per hand in the camera feed with high accuracy and performance.

🧠

Neural Network Classification

Deep learning model trained with TensorFlow/Keras to recognize gesture patterns and classify each sign.

📊

Custom Dataset

Manual collection and labeling of samples for each letter or gesture to build a balanced, representative dataset.

🔤

Letter and Word Translation

Recognizes the fingerspelling alphabet (letters) and common sign language words, showing translated text.

⚡

Streaming Processing

Optimized processing of the video stream for near-instant response without sacrificing accuracy.

🖥️

Visual Interface

Display window that draws the hand skeleton, the detected letter and the sign history on screen.

Tech Stack

Computer vision and machine learning tools

🐍Python
✋MediaPipe Hands
🎥OpenCV
🔢NumPy
📹Camera Streaming

Challenges and Solutions

Technical problems solved during development

1

Robust Hand Detection

Achieving stable hand tracking across different lighting conditions, backgrounds and skin tones.

Solution: Using MediaPipe Hands, which provides an optimized and robust hand detection and landmark model against environment variations.
2

Dataset and Imbalanced Classes

Collecting enough samples of each letter and preventing the model from biasing toward classes with more examples.

Solution: Balanced manual collection and data augmentation (rotations and shifts) to balance and enrich the dataset.
3

Real-Time Latency

Classifying every video frame without the translation feeling slow or choppy.

Solution: Pipeline optimization, frame sampling and a lightweight model that keeps accuracy with low inference latency.
4

Model Generalization

Making the model work well with hands of different users, sizes and angles, not just training data.

Solution: Relative landmark normalization and training with cross-validation to improve generalization ability.

Source Code

Explore the project's full code on GitHub. Includes the training notebook, data capture scripts, dataset and real-time detector.

🧠Deep Learning
✋MediaPipe
🔓Open Source
View Repository on GitHub

Interested in an AI and computer vision project?

I can help you build your machine learning and computer vision solution with the best technologies

ContactSee more projects