android launcher for visually impaired · android launcher for visually impaired 1sneha mittal,...

© IJEDR 2019 | Volume 7, Issue 1 | ISSN: 2321-9939

IJEDR1901066 International Journal of Engineering Development and Research (www.ijedr.org) 367

Android Launcher for Visually Impaired 1Sneha Mittal, 2Swati Garg, 3Kanika Mittal

1,2Student, 3Assistant Professor Department of Computer Science, Bhagwan Parshuram Institute of Technology, Delhi, India

_____________________________________________________________________________________________________ Abstract - Smartphone technology and mobile applications have become an indispensable part of our daily life. The sighted people can use such smartphones easily, but due to the inability to read information on the screens, blind people face tremendous difficulties in using smartphones. At present many blind assistive systems have been implemented, but there is no such kind of good system for blind people for using smartphones and tablets. There have been many efforts, but even now, it is not easy for blind people to independently use their smartphones. One major stumbling block is a lack of understanding about how they use touch screens. Many studies have been conducted to compare how blind people and sighted people use touch screen gestures. Moreover, blind people are dependent on other people to open the required applications on their smartphones. In this paper, we propose “VisionAid”- an android launcher intended to help people with visual challenges or blind people to access Android-based devices. This launcher uses TTS(Text-to-Speech) and talkback feature to provide services through voice. It allows the user to listen to music, to use a voice recorder, to send and receive SMS, to receive and make calls, to create and delete alarms, to create and delete contacts, to browse the web with google engine as a starting point. Simple gestures are used such as sliding up and down to browse the menus and sliding right to select an option. Key Words – Android, Accessibility for Blind, Launcher, Visually Impaired Persons, Speech-To-Text, Text-To-Speech, Talkback, Smartphones, Integration of functionalities

I. INTRODUCTION Smartphones are becoming an inevitable part of this globalization era and have very much influence in our daily activities.

The World Health Organization (WHO) has estimated that approximately 1.3 billion people live with some form of distance or near vision impairment. Although technological advancement has been made, touchscreen-based interfaces being the latest, still there is a prominent lack of research on touchscreen-based interfaces accessibility for visually impaired people. The blind still depends on the normal sited person for performing basic day to day activities like a phone call and other mobile functionalities and thus face challenges daily in communicating with the world around them. A lot of information is available on technological advances for visually impaired people but are failing to impose the concept of “Universal Design” as they are not able to give friendly interaction to blind users. Recent android phone applications still do not meet the requirements and challenges faced by the visually impaired people as all the applications available are discrete and visually impaired have to access multiple applications for messaging, internet browsing etc. Thus, the main objective of this paper is to integrate all the basic functionalities required by visually impaired for performing day to day tasks and provide a user interface that is appropriate for them and uses only simple gestures like sliding up and down to browse the menus and sliding right to select an option. Also, it has been observed that individuals with visual impairment are likely to have limited access to information, and thus they also are at a social disadvantage when they are not able to discuss popular topics. Thus, to overcome this an efficient internet browsing feature has been implemented.

II. RELATED WORK In [1] the authors proposed an application in which calls and messages can be made to pre-saved contacts. They made various

functionalities (calling, messaging, knowing the time and battery level) usable by visually challenged users. In [2] the authors proposed an application which listens to the user’s commands and then responds with voice commands by talking. In [3] authors proposed an application to use object detection for blind people and give them vocal information of the object and current location of the blind user. In [4] authors proposed an application to enable the blind to use smartphones with touchscreens to make calls and to send text messages with ease. In [5] the authors present BlindShopping, an RFID and QR-code based mobile solution to enable shopping for blind people, only demanding inexpensive off-the-shelf technology and limiting the deployment effort from the supermarket. In [6] authors describe the VMAIL system architecture for the android platform that can be used by a blind Person to access e-mails easily. The application uses ‘text to speech’ and voice recognizer to facilitate sending, reading, forwarding and replying to emails using an Android smartphone. A secured platform is provided by the application by including an additional feature of Applock for accessing the application, which is designed considering the convenience of a blind person. In [7] authors proposed blind guide that provides the information and guidance to move from one place to another safely using the Android-based smartphone. TTS (Text to Speech) is used in the navigation system to provide navigation through voice for blind people. Also, it uses the Google map API to apply map information. In [8] authors proposed a project to guide blind people with voice navigated GPS using an android phone. The device is used for detecting obstacles and avoiding an



autonomous device where they were using an android device for the information for further obstacle. The design of the obstacle avoidance program requires the integration of many sensors according to their task.

III. MOTIVATION

There are many applications currently available for visually impaired people but very few are successful in providing ease of accessibility to them. There were various factors that motivated us to develop an android launcher-Vision Aid for visually impaired. First, there are many blind assistive systems that have been implemented, but there is no such kind of good system for blind people for using smartphones and tablets that provides user-friendly interfaces and an easy mode of interaction even for visually impaired people. Currently available android applications are not enough capable to address all accessibility issues. It is found that users are facing complications in understanding the gestures as it does not provide sufficient tactile clues to successfully navigate and it becomes hard to remember all the available gestures. Secondly, at present there are many applications that uses TTS(Text-to-Speech) and talkback feature to provide services through voice support but managing the speed of voice is not in their hands. Third, the existing voice-based applications were all discrete and disconnected in their functionality thus leaving behind the user with all applications spread across and thus the key consideration while developing was to integrate all such functionalities like Calling, Messaging, retrieving position, battery level, contacts, to browse the web etc into one. Always a need arises for downloading the application and for this a normal sited person is required, thus there is a need to eliminate this dependency. Fourth, there are many navigation systems for visually impaired people but these applications are not able to help the user with basic mobile phone features like messaging and calling, and if available they always find a difficulty in locating and dialling a number as the keypad is not consistent with the phone. Thus, the main aim is to develop launcher based on Android Smartphone for the visually impaired person overcoming all the mentioned issues.

IV. BASIC TERMINOLOGIES Android Android is developed by Google. It is a mobile operating system and software stack that includes user interface, multimedia message service (MMS), the operating system for portable devices, middleware, and a standard application such as Web Browser, Email Client, SMS. Required application is provided by the Android through the Android Software Development Kit (SDK) to develop APIs and a variety of tools. The Android system uses C / C + + Libraries and Android works on the Linux kernel. Android has been emerged as a whole new mobile development platform, which helps in building on past successes and avoids most of the past failures of other platforms. According to [9], it has been the best-selling OS on smartphones since 2011 and tablets since 2013. It is the platform which is open source, with no charges and up-front fees, and developers enjoy many benefits over other competing platforms. It has following features:

• Complete: A comprehensive approach was taken by developers when they developed the Android platform. They began with a secure operating system and then they on top built a robust software framework that allows for rich application development opportunities.

• Open: It is provided through open source licensing. When applications were developing, developers have

unprecedented access to the handset features.

• Free: Android applications are free to develop. No licensing or royalty fees are there to develop on this platform. Neither membership fees required, nor any testing fees required. There is no requirement of any signing or certification fees. Android applications can be commercialized and distributed in a variety of ways.

Methods Used

1) Text to Speech (TTS): TTS is a method that converts the speech from text. TTS is important for providing voice output as voice feedback to user. TTS is implemented in software where audio capability is required. When user enters voice command, TTS will convert that text into voice format and performs specific action. [10]

2) Speech to Text (STT): Android has an inbuilt feature that is speech-to-text through which user can provide speech input to the software. In the background speech input will be converted to text and perform action in the form of TTS. [11]

V. PROPOSED METHOD Problem Statement

In this project, an Android launcher for visually impaired is proposed. Through this application, a visually impaired person

can use most of the features of their smartphones. A visually impaired person can send and receive messages, make calls, save and view contact, play music, access internet, set alarms, record his voice, open other application on the smartphones, change the setting of the phone, check phone status such as battery level, current time and date, vibration on/off and Wi-Fi connected or not.



Proposed System

Android launcher named VisionAid can be set as a system launcher in Android smartphones and tablets. Setting as a launcher can help visually impaired as they don’t have to take help of any other person to open the application. When the screen opens up, ten icons are visible (Message, Calls, Contacts, Music, Internet, Alarms, Voice Recorder, Application, Settings, and Status). Users can’t click anywhere on the screens as the clicks are disabled. Therefore, they have to slide up and down to browse between the options and slide right to select the item. In the message section, user can check the messages in their inbox, compose a message, send messages and delete messages. Also, they can delete all received messages at once. In the calls section, user can access all the call logs, delete all call logs and make a call to any person. In this, the number is taken from the user by voice command. In contacts sections, we can access all the contacts saved in their smartphone, select a particular contact and call or message that contact. They can also create a new contact or delete any contact in this section. Next comes the music section in which user can play songs that are present in their phones by selecting the artist and the album of that song. Once the song is selected, they can play or stop it easily. Also, they can play the previous and next track of the selected song. User can lower or higher the volumes without using the side buttons in this section.

In the internet section, visually impaired person can either search on google or he can go to bookmarks. If he searches in google then he has to input his search topic by voice command. Once, he selects the search topic, a webpage will get loaded. He can read the page title, page text, and webpage links. He can go to selected links and can also add the current page to bookmark. This launcher maintains its own bookmark list which cannot be merged by any browser’s bookmark list. If a user marks any webpage as a bookmark then he can go to that page anytime later and can delete that bookmark too. In the alarms section, user can set alarm, view alarms and delete alarms. To set an alarm he will have to input a message, select the day, alarm hour and alarm minutes. Once the alarm is set then the user can view that or can delete it. In the voice record section, user can record their voices, can listen to or delete the recorded voices.

Then comes the application section. In this, the user can select any application that is installed in his smartphone and then can launch that application. In the settings section, he can go to system settings and can make changes through it. But there are few settings that are used very frequently such as alarm tone, notification tone, call tone, Wi-Fi settings, profile settings and Bluetooth settings. These features can be changed directly by this launcher without going to system settings. User can also manage the speed of the talkback feature which is reading all the texts on screen on the scale of one to five. The last section that is the status section updates the user about the battery level of the mobile phone, current date and time, current SIM carrier, Wi-Fi connected or not and if connected then to what network it is connected. All the sections mentioned above can be easily accessed by sliding right and up-down.

VI. FLOWCHART Fig.1 shows the flowchart of how the call module works in the launcher.

Fig. 1 Flowchart showing working of call module



Fig. 2 Screenshot of the launcher’s Fig. 3 Screenshot of the launcher’s screen Fig. 4 Screenshot of launcher’s screen Main screen when we enter message module when we enter the calling module

VII. RESULT The launcher was made and Fig. 2 shows the main screen. When the screen opens, it says “Welcome to VisionAid app. Now

you are at the main menu. Swipe up and down to select an item and swipe right to select it”. We can’t touch anywhere on the screen as the touch is disabled. Therefore, we just have to use gestures to use our phone.

Let us discuss how different modules work in the launcher.

Message Fig. 3 shows the screen when we enter the message section from main module. When we enter the inbox, there are option to

either read the messages in our inbox or delete it. We can compose a new message too. While composing a new message first we have to choose an addressee from our saved contacts. Then compose a message by voice-to-text input method and then send it. If we receive a message from an unsaved contact then we can add that number directly to our phone book through this module. There is also an option to delete all the messages at once.

Calls

Fig. 4 shows the screen when we enter the calls section from main module. Here we check our call logs and delete them. If

there is any phone number in user’s call logs which is not saved in his phone book, then they can save that number directly here. They can make a call by inputting the number through voice-to-text method. Making a call is one of the most frequently used function of a phone. Thus, it will be very helpful for people with visual impairment.

Contacts

Fig. 5 shows the screen when we enter the contacts section from main module. Here the visually impaired can access the contacts saved in their smartphones. Once a contact is selected, they can call it, send a message to it or delete it from their phone book. User can also create a new contact for which he has to input the name and their phone number via voice-to-text method. Creating contacts is one the main feature so as to contact close people easily. Thus, this feature helps visually impaired a lot.

Internet

This feature helps visually impaired to access to internet searches easily. But internet connection is necessary for this module. As shown in Fig 6, user has to select the option of “search in Google” which then ask the user to input his search topic by voice-to-text method. User can listen the page title and page text. He can go to web links or add the current page to bookmarks. List of bookmarks are maintained by this launcher. It is not connected to any browser. If a user has saved any bookmarks previously then he can select it and either access it or delete it.



Fig. 5 Screenshot of launcher’s Fig. 6 Screenshot of launcher’s Fig. 7 Screenshot of the launcher’s screen when we enter screen when we enter the screen when we enter the alarm module contact module internet module

Alarm

User can set alarm too. As shown in Fig 7, user has to input a message and select day, hour and minutes to create an alarm. If an alarm is already created, then he can select it from the alarm list. Note that this alarm list is different from the phone’s inbuilt alarm list.

Application

A launcher should be able to open other installed applications too on a smartphone. Therefore, Fig. 8 shows that this

module helps the user to launch the selected application.

Settings

If a user wants to change any settings of his smartphone then he can do so by this option. But there are some settings that that are used very frequently. As shown is Fig. 9, those settings can directly be accessed without going to the system settings. Such settings are screen timeout, alarm tone, notification tone, call tone, Wi-Fi and Bluetooth settings and mobile profile. Also, if user wants to change the speed of talkback feature then he can do that too.

Status

This section tells the user about the current date and time, current SIM carrier, whether our phone is Wi-Fi connected or

not and if connected then to what network it is connected. It basically tells about the status of the smartphone. When the user swipe right at status, the launcher will speak up all the details.

VIII. CONCLUSION The project aims at developing a software for visually impaired so that they can use a smartphone easily and independently.

By far we have successfully implemented the complete project that is allowing blind people to use most of the main features of a smartphone. At present the project is working without any mistake or disparity, that too in a quite rapid and efficient manner.

IX. FUTURE SCOPE Some of the possible ways of extending this project in future are: - it can be optimized so that it takes less space, time and

low specification hardware to run, more features can be added to make it more useful and current drawbacks would be fixed in the later updates.



Fig. 8 Screenshot of launcher’s screen when we enter Fig. 9 Screenshot of the launcher’s screen when we enter Application module Settings module

REFERENCES [1] Sonal R. Pampattiwar and Prof. Anil Z. Chhangani, “Smartphone Accessibility Application for Visually Impaired,” International Journal of Research in Advent Technology Vol.2, No.4, April 2014 [2] Kaur, Parminder & Ganore, Mayuri & Doiphode, Rucha & Garud, Ashwini & Ghuge, Tejaswini. “Be My Eyes: Android App for visually impaired people.,” 10.13140/RG.2.2.12307.48164, (2017). [3] Ketaki B. Tharkude, Aishwarya K. Wayase, Pratiksha S. More and Sonali S. Kothey, “Smart Android Application for Blind People Based on Object Detection,” International Journal of Innovative Research in Computer and Communication Engineering Vol. 4, Issue 4, April 2016 [4] Robest YONG, “VisionTouch Phone for the Blind,” The Malaysian Journal of Medical Sciences, Oct 2013 [5] Diego López-de-Ipiña, Tania Lorido, and Unai López, “Indoor Navigation and Product Recognition for Blind People Assisted Shopping,” Deusto Institute of Technology – DeustoTech., University of Deusto Avda. Universidades 24, 48007 [6] Rahul Anwani, Usha Santuramani, Deeksha Raina and Priya R.L, “VMAIL: Voice Based Email Application,” International Journal of Computer Science and Information Technologies, Vol. 6 (3), 2015 [7] Nandish M S, Mr. Chetan Balaji and Prof. Shantala C, “An Outdoor Navigation with Voice Recognition Security Application for Visually Impaired People,” International Journal of Engineering Trends and Technology (IJETT) Volume 10 Number 10, Apr 2014 [8] Mrunal Pawar, Minal Pawar and Rohit Najawan, “Route Finding Application for Blind People,” International Journal of Engineering Development and Research, Volume 4, Issue 2, 2016 [9] [Online]. Available: https://en.wikipedia.org/wiki/Android_(operating_ system)#cite_note-10 [10] [Online]. Available: https://developer.android.com/reference/android/speech/tts/TextToSpeech [11] [Online]. Available: https://developer.android.com/reference/android/speech/SpeechRecognizer [12] T. Shabana, A. Anam, A. Rafiya, K. Aisha, “Voice based email system for blinds,” International Journal of Advanced Research in Computer and Communication Engineering Vol. 4, Issue 1, January 2015

android launcher for visually impaired · android launcher for visually impaired 1sneha mittal,...

Documents