TARA – AI-Powered Eye Tracking and Voice Assistant

TARA (Tracking-gaze And Responsive Assistant) is an AI-powered desktop application designed to make computer usage more accessible for people with disabilities that affect their motor control. It provides an alternative way to interact with a computer without depending entirely on a traditional mouse or keyboard.

The system combines eye-gaze estimation, voice recognition, computer vision, and desktop automation to help users interact with on-screen elements. For example, a user can look toward a button and say a command such as “Hey Tara, click Submit.” TARA analyses the user's gaze, processes the spoken command, examines the visible screen, and identifies the most likely target for the requested action.

TARA uses a webcam to estimate where the user is looking. A calibration process helps map eye movements to screen coordinates, while filtering techniques reduce unwanted cursor movement. Its speech-processing system recognises commands in English, Hindi, Marathi, and Telugu, allowing users to interact in supported languages.

The application also uses Optical Character Recognition (OCR) and computer vision to identify text, buttons, links, and other interface elements. Its target-selection system combines gaze information, spoken commands, and detected screen content to determine which element the user may intend to select.

TARA integrates Jev as an optional decision service to help evaluate potential targets, with a local scoring mechanism available as a fallback. Safety controls can request spoken confirmation before risky actions are executed, helping reduce unintended interactions.

The project includes a desktop control panel for setup, camera calibration, voice commands, screen inspection, user profiles, usage statistics, and evaluation. It is designed with a focus on local processing and privacy, although optional external decision services may involve data leaving the device.

TARA is an excellent learning project for students interested in Artificial Intelligence, Computer Vision, Natural Language Processing, Human-Computer Interaction, and assistive technology. It demonstrates how multiple technologies can be integrated into a practical application that addresses a real-world accessibility challenge.