Designing An Eye Monitored System For Wheelchair

Abstract

Statistics show that there are 11,000 new cases of quadriplegia every year in USA. Great people like Stephen Hawking and Max Brito have been suffering from this disease. Our project is an attempt to make lives of the people suffering from this phenomenon simple and by that we mean self-reliant, which will thereby reinculcate their confidence and their happiness. The idea is to create an Eye Monitored System which allows movement of the patient’s wheelchair depending on the eye movements. We know that a person suffering from quadriplegia can partially move his eyes and tilt his head, thus presenting an opportunity for detecting those movements. We have created a prototype where a patient sitting on the Wheel Chair assembly looking directly at the camera, is able to move in a direction just by looking in that direction and can also control lamp, fan etc. with the eye movement. The camera signals are monitored by a MATLAB script, which will then give commands to the motors wired to the Microcontroller over the Serial Interface to move in a particular direction. The system is cost effective and thus can be used by patients spread over a large economy range.

Introduction

Overview

There is an extremely restricted mobility of Patients who cannot use their limbs. Certain wheelchairs such as sip and puff provide an alternative to have movement, but restrict the ability to speak and may be difficult for those with respiratory problems. Our eye controlled wheelchair system has no restriction on the user's other four senses and even sight will not be totally impaired since the camera inbuilt in the laptop will be used for getting the snapshots the Eye. Eye-Monitoring wheelchair control has been a region of study for quite a time now. Many project teams have achieved Eye-Tracking controlled wheelchairs but each having unique benefits and faults. Investigating these designs allowed us to learn and improve our design by avoiding certain design downfalls. Doctors have indicated that about 10 percent of patients find it really difficult to use wheelchair on a daily basis.

Objective

• To help people suffering from Quadriplegia to have a movement on their own will by using wheelchair
• To use technical knowledge to develop wheelchair which user can control by eyes.

Problem definition

To design an eye monitored wheelchair and automation system using the camera to acquire and analysing user intent using head gestures.

Methodology

Camera takes snapshots of the eye and the image is digitally processed in MATLAB by using a specific algorithm in which the pupil region of the eye is detected and based on it’s position, a command is serially sent to the microcontroller where the motor performs different operations based on a decision based system. Motor driver circuit drives motor since output from controller is not enough to drive motors connected to wheelchair. Whenever user wants to control light, microcontroller sends signal to Arduino connected to lamp via Bluetooth module.

Literature Survey

The user stares at a specified location on the screen in order to move in a desired direction. The IR sensor locates the IR reflection and transfers eye location data to the computer. The data transfers from the eye tracker to a laptop. The computer then sends data through LabVIEW into a DAQ (Data Acquisition) unit. Through LabVIEW, we are able to translate the eye data provided through the eye tracker, into output voltages that indicate desired movement directions.

At regular intervals, the camera takes screenshots of the eye, after which the images are sent to a MATLAB script. Hough transform applied to the binary image where the circles in the image are detected and the specific pupil is detected from it. Based on these observations, three ranges for x and y co-ordinates are defined – left, right and top. Two sets of combinations are then specified for each movement.

In this setup, the user’s head should be rigid and straight at all times. Tilting of the head will result in incorrect detection of the eye and the frame is lost without purpose sometimes.

Computerization of Wheel Chair Using PatientIris and Arduino Board, 2017 Face image is given to MATLAB algorithm eliminate background identify only face. From face eyes are detected. Both eyes will be made separated as left eye and right eye scanning of the eye. Will be done to fix whether is closed or open. If it is closed system will send command to stop the chair, if eyes are open system will check the eye ball position whether it is left or right or in centre.

In this assembly patient sitting on wheel chair looks directly at the camera is able to move in a direction just by looking in that direction. Signals from the camera are monitored by MATLAB script which will help in guiding the motors connected to the ATMEL 8051 Microcontroller over the Serial Interface to move in a particular direction.

Camera is mounted on wheelchair in front of the person, for capture the image of eye and tracks the position of eye pupil by using some image processing techniques. According to eye pupil position of user, motor will be move in required direction such as left, right and forward. Ultrasonic sensor is mounted in front of wheelchair for safety to detect static or mobile barriers and stop the wheelchair movement automatically. A central switch is also mounted on wheelchair for emergency purpose and stop to move in require direction if any one call to stop and someone require attention on themselves.

Methodology

A survey indicates that nearly 9-10% of severely disabled patients have difficulties in using powered wheelchair despite of having some training in handling and operating the wheelchair. Our system aims at user friendly product which requires no rigorous training. The transmitter and receiver’s block diagram of proposed system is shown in figure above. Camera takes continuous video frames and samples the input and save it as the screen shots. Each frame is then converted into the black and white frames. For better results, contrast stretching is performed on each frame to make the dark region darker and bright region brighter. To avoid detection errors, the height and width of the user’s eye is calibrated so that the wheelchair won’t have any movement if it detects any other eye measurement except for the user.

Detection of the eye is done by comparing the current position of the eye with the previous position. Thus, the difference in the coordinates will help us to predict the motion in the particular eye. Depending on the position of the pupil, a command is serially sent to the microcontroller where the motor performs different operations based on a decision based system such as move the motor towards right,left, stop etc. . Motor driver circuit drives motor since output from controller is not enough to drive motors connected to wheelchair. A Bluetooth module is also connected to the microcontroller which is used to send commands to the automation unit. The Automation unit consists of Arduino, Bluetooth module, relay, lamp. Whenever user wants to control light, microcontroller sends signal to Arduino connected to lamp via Bluetooth module.

Data Flow

It is first pre-processed to reduced noise and for performing contrast stretching and morphological operation like dilation and erosion. Then features are extracted and position of eye balls is decided. The classified data is fed to the microcontroller to decide the direction of movement of chair.

Detail Design

Button Switch

This is used to switch the module into AT command mode. 5. 1. 4 Motor driver IC (L293D)A motor driver is used to control motors. It act as an interface between microcontroller and the motors. They provide the additional power required to drive the motor since the power from microcontroller is not sufficient. Motor driver IC’s from the L293 series are commonly used such as L293D, L293NE, etc. These ICs control 2 DC motors simultaneously. L293D consist of two H-bridge. H-bridge is the simplest circuit for controlling a low current rated motor.

MATLAB

MATLAB stands for MATrixLABoratory. It is widely used in all areas of applied mathematics, in industries, in education and research at universities. The software is particularly useful for linear algebra since it is built up around vectors and matrices but MATLAB is also a great tool for solving algebraic and differential equations and for numerical integration. MATLAB is useful for signal processing, image processing, optimization, etc. MATLAB can produce pictures in both 2D and 3D. It is also a programming language, and is one of the easiest programming languages for writing mathematical programs.

SVM classifier: Support vector machines (SVMs) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier(although methods such as Platt scaling exist to use SVM in a probabilistic classification setting). An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall. In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces

1. Verify: Compiles and approves your code. It will catch errors in syntax (like missing semicolons or parentheses).
2. Upload: Sends your code to the 101 board.
3. New: This buttons opens up a new code window tab.
4. Open: This button will let you open up an existing sketch.
5. Save: This saves the currently active sketch.
6. Serial Monitor: This will open a window that displays any serial information your 101 board is transmitting. It is very useful for debugging.
7. Sketch Name: This shows the name of the sketch you are currently working on.
8. Code Area: This is the area where you compose the code for your sketch.
9. Message Area: This is where the IDE tells you if there were any errors in your code.
10. Text Console: The text console shows complete error messages. When debugging, the text console is very useful.
11. Board and Serial Port: Shows you what board and the serial port selections.

MPLAB

MPLAB is an integrated development environment used for the development of embedded applications on PIC microcontrollers and is developed by Microchip Technology. MPLAB is designed to work with MPLAB-certified devices such as the MPLAB ICD 3 and MPLAB REAL ICE, for programming and debugging PIC microcontrollers using a personal computer. PICKit programmers are also supported by MPLAB.

Conclusion

Eye monitored wheel chair control will be very useful to the physically handicapped people because of its ease in operation, as with eye movement the wheel chair will be moving in that particular direction only. The design presented allows control of a wheelchair through hardware and software components to create an effective system. This system includes a laptop, a web cam, a microcontroller, an automation unit and electric wheelchair. These pieces comprised the hardware. The software included MATLAB scripts used to process the data and interact with the microcontroller. While not a perfect system the wheelchair now stands able to be controlled solely through eye movement and thus help persons with moderate/severe physical disabilities.