Visually Impaired Individuals: A Camera-Based System Assisting

Abstract

Choosing clothes with complicated patterns and colors may be a difficult task for visually impaired individuals. Automatic clothing patterns and colors can bring independence in their lives. Clothing pattern recognition is a camera-based paradigm system that acknowledges coloring patterns in four classes. They are plaid, striped, patternless, and irregular and this system also identifies eleven colors. The system integrates a camera, raspberry pi, and a speaker or earphone for audio description of clothing colors and patterns. A camera is used for capturing the images of clothes. Choosing clothes with suitable colors and different patterns is a difficult task for visually impaired people. Blind people manage this task with the help of others. So such a system would improve the independence in their lives.

I. Introduction

Based on statistics from the global World Health Organization (WHO), there are about 161 million visually impaired individuals around the world, and thirty-seven million of them can’t see. According to the data provided by World Health Organization (WHO), low vision is outlined as an acuity of 20/200 or less . Close to 100 percent of the male population in Europe suffers from some style of vision deficiency (blindness), the foremost usual case being the inability to differentiate between different colors. Choosing clothes with suitable colors and pattern is a challenging task for these people. They manage this task with the assistance of their family members, or by exploiting plastic Braille labels or differing types of sewing pattern tags on the garments, or by sporting garments with an identical color or with no patterns.. Automatic camera-based pattern recognition may be a difficult task because of several clothing patterns and color styles. Here, we tend to introduce a camera-based system to assist visually impaired individuals to acknowledge patterns and colors. We have used the camera, raspberry pi, and a speaker for designing this system. The camera is used for capturing the image of the clothes and these images are used to determine the color and pattern of the clothes. The output is provided by the speaker.

II. Literature Survey

Selecting clothes with confusing examples and hues may be a testing assignment for visually impaired people. Programmed attire style acknowledgment is in addition a craving analysis issue owing to pivot, scaling, enlightenment, and significantly intensive intra-class style varieties. This approach is used to set up a model that may help in nursing, ongoing item advancement for disabled people. Such a system of recognizing patterns having completely different colors makes their life easier and improves their life quality. The designed paradigm can include most of the patterns like fabric, striped, patternless, irregular, and can acknowledge eight styles of colors.

The system includes a speaker, a computer, and MATLAB. The output of this technique is given by an audio signal. The sound flag is used for this framework. In some systems, the image is captured by the camera and then processed to spot the pattern of the garments that are chosen. The pattern may be classified by the support vector machine formula. The features are extracted by using various descriptors. The statistical features are identified by radon signature descriptor and global features are extracted by wavelet subbands. This gets combined with the local features present to get the required pattern of the clothes. The system uses the CCNY Pattern dataset. This technique may be an efficient method for all the visually impaired people which will help them in identifying various clothing colors and patterns.

Selecting garments with advanced patterns and colors is a difficult task for visually impaired people. People have developed a camera-based advanced system that recognizes patterns of clothes in four classes (plaid, striped, patternless, and irregular) and identify eleven colors. The system integrates a camera, electro-acoustic transducer, a computer, and a Bluetooth earphone for audio description of covering patterns and colors. A camera mounted upon a sunglass is employed to capture pictures. This method is controlled by speech input through the electro-acoustic transducer. To judge the effectiveness of the planned approach, they have used the CCNY Pattern dataset. Their approach achieves ninety-two percent recognition accuracy.

Several essential daily chores can be a tough task for people who are visually impaired. Automatic pattern recognition will bring a lot of required independence in their lives by using a camera-based image system that acknowledges patterns in four main classes (plaid, striped, pattern-less, and irregular) and identifies sixteen colors. The system integrates a camera, a mike, a computer, and earphones for audio description. The technique used by them is edge detection for pattern recognition. To identify the colors they have used the threshold technique.

III. Methodology

The various techniques used in this project are summarized as follows:

A. Open laptop Vision (OpenCV)

Prior information on Python and Numpy array is required before beginning OpenCV. Numpy could be an extremely optimized library for numerical operations. It provides MATLAB-style syntax. The entire Open CV array structures area unit regenerates to-and-from Numpy arrays. Therefore no matter what operations you'll be able to do in Numpy, you will be able to mix it with OpenCV which will increase the variety of weapons in your arsenal. OpenCV could be cross-platform library exploitation that we are able to develop period of time laptop vision applications. It mainly focuses on image process, video capture, and analysis together with options such as face detection and object detection.

B . Convolution Neural Network (CNN)

CNN uses special convolution and pooling operations and performs parameter sharing. To use the CNN technique in the application, there's a desire of Tensor Flow and Keras library for the feature extraction. In our application, we tend to area unit exploitation of 3 layers of CNN.

C .Tensor Flow

Tensor Flow is Associated in Nursing ASCII text file machine learning library for analysis and production. Tensor Flow offers Apis for beginners and consultants to develop for desktop, mobile, web, and cloud. Tensor Flow could be a framework created by Google for making Deep Learning models. It is used in machine learning and deep learning applications. To develop and analysis on fascinating ideas on computing, the Google team created Tensor Flow. Tensor Flow is intended in Python programming language, thus it's thought-about a simple to know framework. Tensor Flow is additionally known as a “Google” product. It includes a range of machine learning and deep learning algorithms. Tensor Flow will train and run deep neural networks for written digit classification, image recognition, word embedding, and the creation of assorted sequence models.

The steps that need the execution and correct dimension of the whole network:

1. Embrace the mandatory modules for Tensor Flow and therefore the knowledge set modules, that area unit required to work out the CNN model.
2. Declare a perform known as run_cnn(), which incorporates varied parameters and improvement variables with the declaration of knowledge placeholders. These improvement variables can declare the coaching pattern.
3. During this step, declare the coaching knowledge placeholders with input parameters. Then reshape the tensor as per the needs.
4. Currently, it’s time to form convolution layers. Let us flatten the output prepared for the totally connected output stage - once 2 layers of stride a pair of pooling with the scale of twenty-eight x twenty-eight, to the dimension of fourteen x fourteen or a minimum of seven x seven x,y coordinates, however with sixty-four output channels. To form the totally connected with 'dense' layer, the new form has to be [-1, 7 x 7 x64].
5. We must always got wind of recording variables. This adds up an outline to store the accuracy of data.

The system mainly integrates:

a) Camera:

• For taking photos of garments we tend to use a USB a pair of 2.0 webcams.
• The consumer goods pattern is classified into 5 sorts like (plaid, striped, patternless, horizontal- vertical, irregular etc.) and completely different colors that are scanned by using a webcam.

b) Raspberry pi:

• The raspberry pi could be a credit-card-sized laptop that plugs into a computer monitor.
• It is employed for computing and process of color patterns.

c) Speaker:

• The output is provided by the speaker.

d) Dataset

• CCNY dataset is employed to store the pattern of clothes which can be used for comparison with the real-time cloth image.

e) SDcard

• It is employed to store the data.

IV. Experimentation and Results

The results were obtained by performing the preprocessing on the real-time image. The grayscale image is obtained by processing the input image for reducing the complexity of the image. For this conversion, we add the three basic colors i.e. RGB, and divide it by 3 to induce the specified grayscale image. A bilateral image is employed for non-linear, edge-preserving, and noise-reducing smoothing of the grayscale image. It replaces the intensity of every pixel with a weighted average of intensity values from nearby pixels. The canny image detection is completed to detect a good range of edges in the image. It is an edge detection operator that uses more than one stage to detect the edge present in an image.

The above four results were obtained after preprocessing another real-time image. The first image is the input image. After preprocessing it grayscale, bilateral, and canny image are obtained. These images will be used for the further process of color and pattern detection. Given below is the figure showing the results of ephocs. This figure shows the accuracy and loss of the system with reference to the epochs. The first figure shows the accuracy with reference to epochs and the second figure shows the loss with reference to epochs.

The above two figure shows the results after processing the given image. Fig. 11 shows the input image of the cloth which is taken for the color and pattern detection. The color detected after processing the image is blue which is shown in Fig 12. This figure shows the detected color.

The above figure shows the pattern which has been detected after processing the given image by using the Convolutional Neural Network algorithm. The Pattern detected is solid.

V.Conclusion

This technique can acknowledge wear patterns and colors to help visually impaired folks in their existence. The projected system uses the detection of colors and patterns. It offers a new approach to the visually impaired person. Here the above figures indicate some detected colors as an audio controller as a speech or sound output. Here we have OPENCV techniques and strategies for our system. Within the long-run work of the system, we include the edge detection and recognition of clothing patterns and colors. For this, the system uses morphological strategies and operations.

VI.References

1. Ms.Tejashri M. Sonawane, Prof. H.T.Patil “Clothing Pattern Recognition Using Features Extraction of Wavelet Coefficient for Visually Impaired People”, International Conference on Trends in Electronics and Informatics (ICOEI),2018.
2. Jarin Joe Rini J, ThilagavathiB,in this paper, “Recognizing clothes patterns and colors for blind people using neural network”, International Conference on Innovations in Information Embedded andCommunication Systems (ICIIECS’15),2015.
3. Xiaodong Yang, Student Member, Shuai Yuan, and YingLiTian, Senior Member, “Assistive Clothing Pattern Recognitionfor Visually Impaired People”,IEEE TRANSACTIONS ON HUMAN- MACHINE SYSTEMS, VOL. 44, NO. 2, APRIL2.
4. Shruti Bharadwaj, Sharath H.K., Praveena M.B., Ajay Shetty, Shivarudraiah B. “Clothing Color and Pattern Recognition for Visually Impaired People”International Journal of Engineering, Management & Sciences (IJEMS)ISSN-2348–3733,Volume-2, Issue-5, May 2015.
5. Suresh Kumar R, “An Implementation of Automatic Clothing Pattern and Color Recognition for Visually Impaired People”, International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 11 (2018)pp.8850-8855.
6. Mrs. Anuradha. S. G, ThogarichetiAshwini, “Clothing Pattern and Color Recognition for Impaired People”, International Journal of Engineering And technology ISSN: 2319- 7242 Volume 5 Issue 4 April 2016, Page No. 16317- 1632.
7. PragatiHegde,Chaithra DS,GeethanjaliS,Nidhi A Bhomkar “Clothing Pattern Recognition For Visually Impaired People”,International journal of current engineering and research,2393- 8374,volume-4,issue.