“Smart Eye for Driver” (SED) is a camera based solution that generates an audio alert when it detects a driver is sleepy or distracted. SED contributes to reducing road crashes, increasing the safety of drivers, passengers and the public, often the innocent victims of road crashes.
Every year millions of crashes occur due to drivers falling asleep and being distracted by a mobile phone or passengers in the car, truck or bus. Some detection devices are available such as physiological sensors based solutions, but drivers find these systems uncomfortable to wear and their detection accuracy is too variable. The costs of other technologies based on steering pattern and lane detection are also costly for use in low-income countries. Hence, SED aims to produce a simple, affordable and user-friendly device.
The SED proof of concept prototype has been developed with driver and government consultation and tested. Feedback from a questionnaire targeting car and bus drivers helped shape the prototype parameters and match driver requirements. Testing for detecting sleepiness and distraction in day and night conditions resulted in an accuracy of 89.71%.
In partnership with iFahja Consulting (the local SME partner in Pakistan), contacts have been established with interested stakeholders and potential distributors. These contacts will shape further design and development of SED in live traffic scenarios followed by technology validation and product commercialisation.
A business plan is foreseen for commercialisation of SED at a price of £100 approximately (without taxes). The first phase will target the Aftermarket, which is perceived as the least risky and a lower cost route. The preferred business model is Business-to-Business (B2B), however, a combination of B2B and Business-to-Consumers (B2C) could be adopted. This approach will be facilitated by using our popular Facebook page “Chill Life” in helping to penetrate the B2C market faster and to generate a positive cash flow.
In parallel, SED technology will be demonstrated to end users in Trade Associations at exhibitions and disseminated to road safety and environmental organisations to seek endorsement for promoting SED benefits. These benefits include local job creation and income generation, but more importantly reduce road crashes thereby saving lives and reducing injuries.
PDF content (text-only)
i Final Report on Smart Eye for Driver (SED) under the programme Transport-Technology Research Innovation for International Development (T-TRIID) A solution to the millions of car crashes that occur due to sleepiness and distraction of drivers Submitted by: Dr. Akhtar Khalil Director, My Smart Remote Email: firstname.lastname@example.org Phone: 0044 208 1234 178 ii Final Report Project Commencement Date: 11th October 2018 Organisation Name: My Smart Remote Ltd. Project Title: Smart Eye for Driver (SED) Reference number: LD005 Contact Person: Dr. Akhtar Khalil Document Control Sheet Issue Status Author(s) Reviewed By IMC Approved By Issue Date 1 Draft Dr Akhtar Khalil Bruce Thompson 22/03/2019 2 Final Dr Akhtar Khalil Graham Fiveash, Nick Jones Bruce Thompson 26/04/2019 iii Executive Summary “Smart Eye for Driver” (SED) is a camera based solution that generates an audio alert when it detects a driver is sleepy or distracted. SED contributes to reducing road crashes, increasing the safety of drivers, passengers and the public, often the innocent victims of road crashes. Every year millions of crashes occur due to drivers falling asleep and being distracted by a mobile phone or passengers in the car, truck or bus. Some detection devices are available such as physiological sensors based solutions, but drivers find these systems uncomfortable to wear and their detection accuracy is too variable. The costs of other technologies based on steering pattern and lane detection are also costly for use in low-income countries. Hence, SED aims to produce a simple, affordable and user-friendly device The SED proof of concept prototype has been developed with driver and government consultation and tested. Feedback from a questionnaire targeting car and bus drivers helped shape the prototype parameters and match driver requirements. Testing for detecting sleepiness and distraction in day and night conditions resulted in an accuracy of 89.71%. In partnership with iFahja Consulting (the local SME partner in Pakistan), contacts have been established with interested stakeholders and potential distributors. These contacts will shape further design and development of SED in live traffic scenarios followed by technology validation and product commercialisation. A business plan is foreseen for commercialisation of SED at a price of £100 approximately (without taxes). The first phase will target the Aftermarket, which is perceived as the least risky and a lower cost route. The preferred business model is Business-to-Business (B2B), however, a combination of B2B and Business-to-Consumers (B2C) could be adopted. This approach will be facilitated by using our popular Facebook page "Chill Life" in helping to penetrate the B2C market faster and to generate a positive cash flow. In parallel, SED technology will be demonstrated to end users in Trade Associations at exhibitions and disseminated to road safety and environmental organisations to seek endorsement for promoting SED benefits. These benefits include local job creation and income generation, but more importantly reduce road crashes thereby saving lives and reducing injuries. iv Contents Executive Summary ................................................................................................................... iii Acronyms .................................................................................................................................... v 1. Smart Eye for Driver – the objectives ................................................................................. 1 2. Smart Eye for Driver – the challenges................................................................................. 1 3. Existing systems and limitations ......................................................................................... 3 4. Smart Eye for Driver - the solution ..................................................................................... 5 5. Smart Eye for Driver – the findings ..................................................................................... 8 6. Benefits to the National Transport System ...................................................................... 10 7. Next Steps ......................................................................................................................... 10 8. Business Model and Strategy ............................................................................................ 11 9. Conclusions ....................................................................................................................... 12 Appendices ........................................................................................................................ 13 Appendix 1 – Project Deliverables .................................................................................... 14 Appendix 2 - Image of the Prototype ............................................................................... 14 Appendix 3 - Screenshots of App ...................................................................................... 15 Appendix 4 - Summary of Questionnaires: ....................................................................... 18 v Acronyms B2B Business to Business B2C Business to Consumers BRT Bus Rapid Transit C&W Communication and Works Department IR Infrared Radiation MPH Miles Per Hour MV Machine Vision NHTSA National Highway Traffic Safety Administration OEM Original Equipment Manufacturer Open CV Open Computer Vision PDA Peshawar Development Authority RTC Road Traffic Crash SED Smart Eye for Driver UK United Kingdom WHO World Health Organisation 1 1. Smart Eye for Driver – the objectives SED aims to investigate the use of technology solutions for reducing road traffic crashes in Pakistan caused by driver sleepiness or distraction. Hence, SED aspires to deliver: the design and development of a device that generates an alert (audio or vibration) for detecting potentially dangerous changes in driver behaviour; an affordable device for motorists and small and medium sized businesses in low-income countries; a user-friendly device that can be mounted on the vehicle dashboard and needs limited data input before use. 2. Smart Eye for Driver – the challenges Studies carried out in Pakistan show that driver sleepiness, fatigue and distraction are the most significant factors contributing to fatal highway crashes.1 According to Mr. Abdul Wadood, Deputy Director, Pakhtunkhwa Highways, road crashes in Pakistan are under reported with only fatal crashes well documented. A literature review of more than 50 publications shows that sleep deprivation and driver distraction are leading causes of road crashes, especially among long-haul truck drivers. Such crashes result in unacceptable levels of fatalities and injuries as evidence of extracts from the following studies show - “Comparison of Fatigue Related Road Traffic Crashes on the National Highways and Motorways in Pakistan”2 , “The sleep of long haul truck drivers”3 , by Edgar Synder & Associates4 , “Teen Distracted Driver Data”5 , Traffic Safety Facts6 , from Centers for Disease Control and Prevention, Jackson and Kennedy and many others: Globally 3,200 lives are lost every day in road crashes; An analysis of 1750 road traffic crashes (RTCs) on the M1 and M2 motorways in Pakistan showed 28% were fatigue-related resulting in 54% of the fatalities. Similarly, on the N-5 national highway, 9% of RTCs were fatigue-related leading to 41% of the serious injuries. Overall, fatigue-related incidents increased by 34.8% between 2003 and 2009 in Pakistan;7 1 K Azam, S. Akbar, S. Noor, H Ullah, S Shah, & M Babar, " Driver Fatigue Related Road Accidents in Pakistan", Journal of Engineering and Applied Sciences, 28 (2), 2009, last retrieved from https://journals.uetjournals.com/index.php/JEAS/article/view/2751 2 K Azam, A Shakoor, R Shah, A Khan, S. Shah, & M. Khalil, “Comparison of Fatigue Related Road Traffic Crashes on the National Highways And Motorways In Pakistan”, Journal Of Engineering And Applied Sciences”, 33(2), last retrieved on 10 January 2019 from: https://journals.uetjournals.com/index.php/JEAS/article/view/2343 3 Mitler, M., Miller, J., Lipsitz, J., Walsh, K., Wylie, C., 1997, "The sleep of long-haul truck drivers", New England Journal of Medicine 337, (11), 755–761. 4 Edgar Synder & Associates, "Texting and Driving Accident Statistics", last retieved from: https://www.edgarsnyder.com/car-accident/cause-of-accident/cell-phone/cell-phone-statistics.html 5 National Highway Traffic Safety Administration, "Teen Distracted Driver Data", 2018, last retrieved on 03 December 2018 from: https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812504 6 National Highway Traffic Safety Administration, "2016 Fatal Motor Vehicle Crashes", 2017, last retrieved on 05 December 2018 from: file:///C:/Users/student/Downloads/812456.pdf 7 Asad Zia, "Early to bed...: Fatigue, a leading cause of road accidents", The Express Tribune, last retrieved on 30 June 2019 from: https://tribune.com.pk/story/570181/early-to-bed-fatigue-a-leading-cause-of-road-accidents/ 2 The Global Status Report on Road Safety 2013 published by the World Health Organisation (WHO) declared road crashes in Pakistan a leading cause of death for young people aged 15-29;8 Every 25 seconds, a person is involved in a road crash, according to World Health Organisation (WHO); The Royal Society for Prevention of Accidents found that more than 50 % of road crashes are caused by driver distraction often resulting in fatal crashes; Drivers in Pakistan consider drink-driving more dangerous than distracted driving, hence they willing use smartphones and tolerate many distractions while driving. Research studies show that texting while driving is six times more likely to cause a crash than when driving drunk. Approximately 421,000 people were injured in crashes involving a distracted driver.9 The United States National Highway Traffic Safety Administration (NHTSA) estimates 660,000 drivers each day use electronic devices while driving; The United States National Safety Council reports that using a cell phone when driving leads to 1.6 million crashes each year, of which 390,000 injuries are caused by texting while driving; A quarter of teenagers respond to at least one text message every time they drive and 20% of teenagers and 10% of parents report having multi-text message conversations while driving; One of every four car crashes in the United States is caused by texting while driving; Answering a text distracts a driver for approximately five seconds, which when driving at 55 mph is tantamount to dangerous driving for the length of a football field; The National Sleep Foundation, Washington, USA, reveals that over 60% of drivers opt to drive while being drowsy; A study on drowsy driving in the United States found that an estimated 1 in 25 adult drivers (aged 18 years or older) report having fallen asleep while driving in the previous 30 days;10, 11 NHSTA estimates that driving while drowsy was responsible for 72,000 crashes, 44,000 injuries, and 800 deaths in 2013 in the United States.12 In addition, these studies reveal how difficult it is to determine the exact moment a driver falls asleep. Hence, developing a device to avoid drivers falling asleep and being distracted is vital to improving driver behaviour and reducing road crashes, fatalities and injuries. Who’s more likely to feel sleepy when driving? Drivers who do not get enough sleep, which includes night guards, cooks and servants who supplement their earnings by taking extra jobs as drivers. They are often overworked and 8 Sukaina Raza Ali, "Traffic rules: On roads oft travelled", Dawn Sunday Magazine, September 20, 2015, last retrieved on 19 December 2018 from: https://www.dawn.com/news/1207837 9 Edgar Synder & Associates, "Texting and Driving Accident Statistics", last retieved from: https://www.edgarsnyder.com/car-accident/cause-of-accident/cell-phone/cell-phone-statistics.html 10 A Wheaton, D Chapman, L Presley-Cantrell, J Croft, D Roehler, "Drowsy Driving — 19 States and the District of Columbia, 2009–2010", Centers for Disease Control and Prevention, 2013, last retrieved on 14 January 2019 from: https://www.cdc.gov/mmwr/pdf/wk/mm6151.pdf 11 A Wheaton, R Shults, D Chapman, E Ford, J Croft, "Drowsy Driving and Risk Behaviors — 10 States and Puerto Rico, 2011–2012", Centers for Disease Control and Prevention, 2014, last retrieved on 18 December 2018 from: https://www.cdc.gov/mmwr/pdf/wk/mm6326.pdf 12 Centers for Disease Control and Prevention, "Drowsy Driving: Asleep at the Wheel", last retrieved on 15 January 2019 from: https://www.cdc.gov/features/dsdrowsydriving/index.html 3 exhausted, which reduces their concentration and the motion of the vehicle further induces sleep;13 Long-haul truck and bus drivers who often have to drive in excess of 12 hours a day to earn a living wage; Shift workers that work night or long shifts; Drivers with untreated sleep disorders; Drivers who use medications that cause drowsiness. Figure 1. Figures showing distracted and sleepy drivers 3. Existing systems and limitations The automotive industry uses various techniques and systems for sleep and distraction detection. The main techniques are based on the driver’s state or the driver’s performance or a combination of both. Most systems are installed in a limited number of modern cars leaving millions of people, especially in low-income countries without additional safety protection. In addition, the high cost of these systems is a barrier to their adoption in low-income countries. These detection systems are installed as either intrusive or non-intrusive: intrusive - the equipment is attached to the driver as a wearable; non-intrusive - the equipment is mounted on the vehicle dashboard or attached to the vehicle driver cabin. 13 Sukaina Raza Ali, "Traffic rules: On roads oft travelled", Dawn Sunday Magazine, September 20, 2015, last retrieved on 19 December 2018 from: https://www.dawn.com/news/1207837 4 Steering pattern and lane detection systems The BOSCH Drowsiness Detection System is an example of a non-intrusive system. It is a vehiclebased approach that continuously monitors driver alertness and performance based on vehicle steering pattern and lane-positioning detection. Driver performance related parameters are measured through an on-board system and compared with previous values to predict any change. Driver performance is monitored continuously and an immediate warning sound is given when any sign of drowsiness or other distraction is detected. Limitations: Vehicles with steering pattern and lane-positioning detection systems are costly and this technology is not common to all makes and models of vehicles, especially in low-income countries. Techniques based on driving pattern such as steering and lane monitoring use a steering movement sensor for detecting the level of drowsiness. When micro-adjustments decrease from the normal level (i.e. between 0.5-5 degree), drowsiness is detected. Such detection is based upon the road geometry, vehicle condition, traffic density and other variables. Regardless of their accuracy in testing, these systems may often fail in operation, proving disastrous and unreliable. Physiological sensor systems Physiological sensor-based methods are intrusive systems that require the equipment to be attached to the driver. An example of the physiological sensor-based technology is the Toyobo Co Ltd. Limitations: Physiological sensors are expensive. Moreover, drivers in general, and specifically in the developing countries, find sensors directly attached to their body uncomfortable. Another Toyobo Co device, Cocomi, is attached to body and remains intact even when the driver is moving. Drivers find these devices difficult to maintain and wear. An alternative system uses a non-intrusive signal capturing device or electrodes attached to the driver’s body through ZigBee and Bluetooth, but these are less accurate in acquiring bio signals, such as galvanic skin response. Signal noise also affects the accuracy of physiological sensor-based systems. The accuracy of these systems is good on high-speed straight roads but at road junctions, traffic signals, stop-and-go traffic scenarios noise interference compromises the performance of an on-board system. Consequently, many false positives are generated that make such devices unacceptable to drivers. One of the major disadvantages of the steering, lane monitoring and physiological sensor-based systems is the false positives (incorrect detection of driver sleepiness or distraction) and false negatives. The accuracy of these systems in a variety of road and traffic conditions is low compared to their accuracy in the lab level tests or tests on straight roads. The alerts/alarms generated by false positives result in drivers ignoring the system. Systems that require data for projections and decision making “Smart Cap, a wearable cap device, is an example of these systems that allow company managers to optimize driver performance and are also used for commercial decision making. Limitations: Smart Cap uses wearables, for example, cap and wrist bands that require drivers to input data before driving, which they are reluctant to do. These systems also result in false positives and false negatives. Moreover, such systems encourage bad practice among drivers in their competition to win a daily job from their manager. Such practices can involve drivers providing false data to avoid the system, predicting sleepiness. Using smartphone camera-based system With the changing technology trends, mobile applications for drowsiness and distraction detection are being developed. The use of smartphone camera was considered but dismissed as messages and 5 calls on such a device distract drivers. Also, when the driver is distracted by the smartphone, the driver will turn to look directly at the camera that will not fail to detect any eye movement or distraction, consequently no alert will sound. Machine Vision in the low-income countries These systems are generally manufactured in developed countries and therefore based on research and data in those countries. Machine vision and image processing systems developed in one region cannot be used in other parts of the world with the same efficiency and accuracy because eye physiology differs between regions and continents. Therefore, the databases of such systems need calibrating for the eyes of different ethnicities 4. Smart Eye for Driver - the solution SED is a camera-based device that generates an audio alert when it detects the driver is becoming sleepy or is distracted, thus contributing to reducing road crashes. SED uses a smartphone app for the initial setup of the camera, which does not require the driver to input any further data and can be mounted on the vehicle dashboard. This approach keeps the production low compared to other systems, maintenance is easy and flexible. SED requires minimal driver training. SED will be “trained” using huge data and users will be provided with options to control sensitivity levels that will help in reducing the false positives and false negatives. Design and development of the prototype SED uses machine vision with an external camera that locates the driver’s eyes and head by computer vision algorithms. Sleepiness is detected when the eyes close for a longer period than during normal blinking. Driver distraction is detected when the driver’s head moves beyond predefined parameters. Detection of sleepiness or distraction generates an audio alarm. The system works in three stages - calibration, face detection, eye detection and drowsiness detection. First, the pre-processing stage captures and analyses camera stream. Second, the camera is configured and sets the home position of the face. The system works on the assumption that in normal driving the driver does not move his head beyond 45o .In the second stage the face detection is performed. Third, the eyes are detected and the control unit determines whether the driver is sleepy. Figure 2 shows the block diagram of the system. 6 Figure 2. Methodology of face detection and eye-state detection system Techniques for Face Detection Human face recognition from video frames can be done using near Infrared (IR) as shown by Dowall, Pavlidis and Bebis14 and Zhu Z and Ji Q15; neural networks as in the study by Perez, Gonza and Medina16 and Rowley, Baluja and Kanade17, skin colour as given by the study in Ji Q18 and Li Y, Qi X & Wang Y19, Gabor filters20 and template matching21, 22, 23 methods. Eye and eye state detection can be 14 Dowall J, Pavlidis I and Bebis G 2001 Face detection in the near-IR spectrum. Image Vis. Comput. 21: 565–578 15 Zhu Z and Ji Q, “Robust real-time eye detection and tracking under variable lighting conditions and various face orientations”, 2005.Comput. Vis. Image Und. 98: 124–154 16 Perez C A, Gonza´lez G D, Medina L E and Galdames F J, “Linear vs. nonlinear neural modeling for 2-D pattern recognition”, 2005, IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 35 (6): 955–964 17 Rowley H, Baluja S and Kanade T, “Neural Network based Face Detection”, 1998, IEEE Trans. Pattern Anal. Mach. Intell. 20 (1): 23–38 18 Jones M and Rehg J, “Statistical color models with application to skin detection”, 2002 Int. J. Comput. Vis. 46: 81–96 19 WangJGand Sung E, “Study on eye gaze estimation”, 2002, IEEE Trans. Syst. Man Cybern. Part B: Cybern. 32 (3): 332–350 20 Chen Y W and Kubo K, “A robust eye detection and tracking technique using Gabor filters”, proceedings of the Third International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2007, vol. 1, pp. 109–112 21 Ji Q, “3D face pose estimation and tracking from a monocular camera” 2002, Image Vis. Comput. 20 (7): 499– 511 7 achieved by employing iris detection24, eyelid movement25, 26 and by measuring the distance between eyelids27. Each technique and algorithm has its advantages and disadvantages. Some algorithms give highly accurate results but are not suitable for real-time applications due to their inability to give the instant feedback SED requries. Others are highly efficient in time but their accuracy is less reliable. Computational time and accuracy are both critical factors when face and eye detection algorithms are used in applications such as sleep detection. SED aims to use a combined approach that computes face and eye parameters simultaneously thereby overcoming the limitations of using two individual algorithms. SED will utilize different stateof-the-art methods for face and eye detection and will fuse the parameters, making it both efficient and accurate. SED is designed to work under variable light conditions enabling it to operate day and night, and in different weather conditions. SED uses a mobile app for the initial configuration of the device and the various interfaces to configure the app are illustrated in the appendices. SED Specification SED consists of a camera, a control unit and a speaker to generate an audio alert. It operates on 5- volt power that can be charged from the vehicle cigarette lighter socket. The smartphone app of SED sets the audio volume and camera position and communicates with the hardware unit via Bluetooth. Assumptions Sleepiness is defined as occurring when the driver’s eyes close for 1.25 seconds or more; Distraction is when the driver’s head moves more than 45o from its default position. Equipment Software: Python 3.5 OpenCV 4.0 Numpy 1.15.4 Hardware: Raspberry Pi 3 and Raspberry Pi Zero 8 MB NoIR Camera IR illuminator Mounting kit 22 Li Y, Qi X and Wang Y, “Eye detection by using fuzzy template matching and feature-parameter-based judgement. Pattern Recogn” 2001, Lett. 22(10): 1111–1124 23 Maio D and Maltoni D, “Real-time face location on grayscale static images”, 2000, Pattern Recogn. 33: 1525– 1539 24 Perez C A, Lazcano V A and Este´vez P A, “Real-time iris detection on coronal-axis-rotated faces, 2007 IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 37 (5): 971–978 25 Liu D, Sun P, Xiao Y and Yin Y, “Drowsiness detection based on eyelid movement”, 2010, proceedings of the Second International Workshop on Education Technology and Computer Science, vol. 2, pp. 49–52 26 Tabrizi P R and Zoroofi RA, “Open/closed eye analysis for drowsiness detection”, 2008, proceedings of the First Workshops on Image Processing Theory, Tools and Applications, pp. 1–7 27 Saradadevi M and Bajaj P, “Driver fatigue detection using mouth and yawning analysis”, 2008, Int. J. Comput. Sci. Netw. Secur. 8: 183–188 8 Raspberry Pie Model B3 is slightly larger than the Raspberry Pi 0+ but if offers many ports (USB port, HDMI and other ports) necessary for prototype development, including direct connections for a keyboard and computer mouse. The prototype uses a micro SD card, Raspberry Pi and external peripherals. The finished product will use an on-board memory and customised Raspberry Pi / Beaglebone. The Linux Operating System and hardware application of the SED system will be installed on the on-board memory. There will be no need to buy any external SD card as the memory will be on-board inside the SED casing. The Raspberry Pi/Beaglebone based SED system will be sold as a complete package with Raspberry Pi, on-board memory, sound voice recorder and playback, camera, infrared for night vision, LED to show power turn on/off, SED’s power on/off button or switch and a structure to hold and fix the device on dashboard or on the rear-view mirror. All components and the Raspberry Pi will be housed in a purpose made casing designed to appeal to potential users. The finished Raspberry Pi/Beaglebone based SED system will be smaller than the prototype and have less ports. The cost of the proposed model of Raspberry Pi (not the Raspberry Pi 3 model used for prototype) in the finished product is under £15. For details on overall SED costs see section 8. Prototype Development In-vehicle and on-road research and tests are necessary to refine the safe period of eye closure that defines driver sleepiness; Machine Vision and Image processing solutions need developing for Asia, which is foreseen in Phase 2 of SED development; Our local research shows that due to the strong emotional attachment with children, drivers respond better to an alert that mimics a child. Therefore, the audio alert buzzer is replaced with a voice message of a child saying “Dad! I need your hug. Come home safe please”. 5. Smart Eye for Driver – the findings Non-probability judgement sampling techniques were used to select the appropriate set of people that satisfy the requirements of data collection and analysis. Dependency for data gathering was based on interviews of targeted groups rather than the general population. SED approached transport organisations and businesses, to identify end user requirements. Based on our market study, literature review and the interviews/meetings we learnt the following: End users have shown interest in a technology-based solution that detects driver sleepiness and distractions; SED must not use intrusive technology or wearable devices; Systems that give a high number of false positives annoy drivers resulting in such systems not being used; Drivers get extremely annoyed if any false positive is reported to their supervisor; Drivers showed interest in the proposed camera-based solution that is configured once and does not require configuration every time, unless an app update change is required; Drivers prefer a solution that detects sleep or distraction in real time; Discussions suggested that using a child’s voice as an alert would increase the vigilance of the driver and the motivation to use SED. 9 Trial Results of Prototype Tests Detecting sleepiness and distractions is the core objective of this project. However, unlike many other research studies, tests were carried out to determine SED accuracy when the driver is fully awake and not distracted. Such testing is important for adapting the system to individual drivers. Based on our survey questionnaire (see appendix 3), results show that drivers are unhappy with devices that show false positives - when the driver is awake but the device detects sleepiness. Therefore, in developing SED, 350 tests were carried out to test the incidence of false positives or false negatives – see table 1 below. S No. Test Total Number False Positives* False Negatives** Accurate Results Accuracy (%) 1 Normal driving position (driver neither sleeping nor distracted) 70 5 NA 65 92.86 2 Sleeping driver (eyes closed) 70 NA 3 67 95.71 3 Distracted driver (driver looking towards right or left) 70 6 2 62 88.57 4 Normal driving position at night 70 11 NA 59 84.29 5 Sleeping driver at night 70 NA 9 61 87.14 Total 350 22 14 314 89.71 Table 1: Result of Tests Performed on the Prototype * False positives: When the driver is in his normal driving position (neither sleeping nor distracted) but SED shows him/her as either sleeping or distracted (incorrect finding). ** False negatives: When the driver is either sleeping or distracted but SED shows him/her as in normal active driving position (neither sleeping nor distracted) (incorrect finding). Limitations and further gaps for improvements SED’s accuracy for the normal driving position was 92.68%. However, during tests it was found that there is an issue with its sensitivity. SED showed false results on some points even when the driver was not sleepy and not distracted. This can be improved by training the device on a large data set and placing a feature for adjusting the sensitivity level. SED’s accuracy on finding driver’s sleepiness was good (95.71%). The accuracy of the test results for distracted drivers was 88.57%. It is expected that the accuracy for the prototype will reduce in the tests in operational environments, because when driving on long straight roads the driver makes considerable movements without being distracted. This issue can be resolved by training the device on large data set for distraction and adjusting the distraction sensitivity level of the device. Accuracy of the device for normal driving at night was 84.29% and accuracy of the device at detecting sleeping driver at night was 87.14%. Video of the working prototype is given on: https://youtu.be/7Ybdq6f11sE List of officials interviewed The key officials are: Muhammad Humayun, Secretary Transport, Khyber Pakhtunkhwa; Kashif, MS Transport, Transport Department, Khyber Pakhtunkhwa; Abdul Waheed Khan, Architect Transport, Transport Department, Khyber Pakhtunkhwa; Engr. Ahmad Nabi Sultan, Managing Director, Pakhtunkhwa Highways; 10 Engr. Abdul Wadood, Deputy Director Vigilance, Pakhtunkhwa Highways; Engr. Fazle Haq, Assistant Director, Pakhtunkhwa Highways; Engr. Asifullah Khan, Assistant Director, Pakhtunkhwa Highways; Imran Naqvi, Manager Training at Daewoo Express (transport service in Pakistan); Itisam, Assistant Director, Peshawar Development Authority, Bus Rapid Transit; Alamzeb, Director Coordination, Bus Rapid Transit (BRT); Amin ur Din Ahmad, Technical Expert, Pesahawar Development Authority; Abdul Qadeer, Deputy Director Roads, Peshawar Development Authority; Fazal Auto Electrician, Ring Road, Peshawar; Nawab Auto Mechanic, University Road, Peshawar; Right Choice Solutions, Peshawar, Pakistan; Distributors and related businesses in Diljan Plaza auto market, Peshawar. 6. Benefits to the National Transport System Smart Eye for Driver offers the following benefits: Reduced road crash fatalities – SED will save lives by reducing the number of road traffic crashes caused by driver sleepiness and distraction; Improved health - Many car crashes result in injuries, disabilities and psychological impacts such as stress, depression, fear and trauma. SED will reduce car crashes, hence reducing injuries, depression, fear and other health related issues; Reduction in financial costs due to road crashes – Vehicles are very expensive in the low income countries such as Pakistan. As a result of the crashes, costly repairs are necessary and vehicle asset values are substantially reduced that erode business margins and increase transport costs. Also, there is considerable additional cost incurred in the treatment of the crash victims, often the innocent public. By reducing the number of accidents, SED will contribute to lowering postcrash health expenditure and vehicle repairs or replacement; Peace of mind –Discussions and interviews with passengers revealed that many passengers are worried about the driver falling asleep, especially on long distance journeys on motorways and national highways. SED offers an additional level of comfort to passengers and drivers that potentially lowers the risks of a road crash; Environmental impact / carbon footprint – Vehicles damaged in crashes are often scrapped that leads to increased production. Producing a new car creates six to 35 tonnes of carbon dioxide emissions depending on the vehicle specification, which is equivalent to the average emissions during a vehicle’s life.28 By reducing road crashes SED will contribute to lowering greenhouse gas emissions. 7. Next Steps Further development includes prototype development for operational environments, testing, physical trials in vehicles and live-demonstrations on roads. Other aspects include the appearance and feel of the product. A critical aspect will be the adaptation of existing vision based solutions and enhanced “training” of the device on large data set to improve its accuracy. A mobile app will be integrated into the device in the next phase allowing the user to configure the device. The user will be able to adjust volume setting, device sensitivity setting and mounting position of the device through the mobile app. The list of the settings for the next step are given in appendix 2. 28 Mike Berners-Lee and Duncan Clark, "What's the carbon footprint of ... a new car?", The Guardian, 2010, last retrieved on 5th February, 2019 from: https://www.theguardian.com/environment/green-livingblog/2010/sep/23/carbon-footprint-new-car 11 The first step of commercialization will involve the design and development of SED. Once SED is accepted by end users, a management system can be designed and integrated within SED that would enable managers to monitor driver behaviour. In addition to the existing audio alert, an optional second alert in the form of a vibration alert on a Smart Watch will also be offered. 8. Business Model and Strategy Users / Market The markets for SED are twofold - Business to Consumers (B2C) and Business to Business (B2B) market. In the B2C market, motorists are our potential customers. The B2B market is divided into two groups: (i) Aftermarket for existing cars and – the customers are: a. Car insurance companies; b. Haulage companies; c. Other car related businesses and motorists; d. Transport departments of organisations and businesses. (ii) Original Equipment Manufacturers (OEM) – this market focuses on new cars. Our preferred business model is B2B, however, in the initial stages of commercialisation a combination of B2B and B2C might be adopted. Based on previous experience of commercialisation, this approach will utilise our Facebook page "Chill Life" and help to penetrate the B2C market faster and to generate a positive cash flow. For auto/car manufacturers, SED will be licensed for manufacturing, and for the Aftermarket customers, SED will be available via distributors and agents already engaged in the automotive market. Other potential producers will include manufacturers of electronic products in Europe, Pakistan, and the UK. Why is the Aftermarket the safest route? The first phase of commercialisation will start with the Aftermarket, which is the safest and the lower cost route. Waiting for car manufacturers to produce a SED system will take time, be restricted to new vehicles leaving a large majority of drivers in low-income countries more exposed to road crashes. Hence, focusing on the Aftermarket provides access to a safe driving device for all motorists. Commercialisation plan The Project Lead (Dr. Akhtar Khalil) is a member of the International Association of Auto Theft Investigators (IAATI) and will head the commercialisation phase. In partnership, with iFahja Consulting (the local SME in Pakistan), contacts in Pakistan have been established with: • The research partners: Who understand the nature of the technology; • Government organisations: Who understand the nature of the problem; • Commercial partners: Who understand the market and who can commercialise the finished product. For efficient exploitation of the SED system, the commercialisation of SED will be phased. The first phase covers product development and initial exploitation in Pakistan. The established contacts at Pakhtunkhwa Highways Authority, Peshawar Development Authority, Transport Department, Haulage companies and related businesses will be used and a presentation will be given to auto businesses and technologists/experts. 12 The second phase (year 2) will extend to the South Asian market followed by expansion to Africa in year 3. The third phase (year 4) foresees expansion to Europe followed by global expansion in year 5. Dissemination strategy Trade Associations throughout Asia and Europe will be used to demonstrate the technology to endusers in a variety of industry sectors, as well as the placement of promotional material and technical articles in industry journals. Similarly, dissemination will include the targeting of environmental lobby groups and road safety organisations and governments to demonstrate the positive environmental impact and safety benefits of SED. The SED website will also play a key role in disseminating the benefits of SED. The website will feature animated demonstrations of the technology, facts and figures on effectiveness and a list of trade fairs where information will be presented. With continuing SED success, the website and other social media will include product reviews, links to publications and technical case studies. The SED team is currently managing a Facebook page with over four million “likes” that will be used as a promotional platform after SED is fully operational. The Project Lead, Dr. Akhtar Khalil has already highlighted the issue of driver sleepiness and distraction in the program, “Dr. Tech” on Pakistan’s National TV Channel (PTV Home program broadcasted in 68 countries). Cost SED will be sold on a for-profit commercial basis at a retail price of £100 approximately (without taxes). For the first 1000 units, the projected production cost would be around £45. This can be further reduced by increasing the volume of production. Distributors will be offered a distributor price linked to the product volume. 9. Conclusions The proof of concept prototype has been developed and tested. The limitations of the existing technologies for detecting sleepiness and distraction of drivers are identified. A survey questionnaire of 170 drivers and interviews with officials helped to identify driver requirements and the barriers in using some existing technologies. SED design matches the local market requirements of the low income countries. Meetings with stakeholders show a high demand. SED will contribute to reducing road crash fatalities and injuries, improve health and peace of mind, create jobs and generate revenue, and reduce transport’s carbon footprint. Phase 2 funding will enable bug fixing, improve accuracy, train SED using large data set, improve device design and develop SED for use in real and operational environments. These design developments will improve the user acceptance and help accelerate entry to the market. 13 Appendices 14 Appendix 1 – Project Deliverables Information gathering and analysis on the requirements of the local market and market study Research on integrating the requirements of end users in the system design Designed user interfaces Developed prototype Preparation of the business model, commercialisation strategy and next steps Report on the project Video of prototype demonstration provided on: https://youtu.be/7Ybdq6f11sE Appendix 2 - Image of the Prototype Figure 3: Image of the hardware prototype 15 Appendix 3 - Screenshots of App Screenshots of the designed interfaces of mobile app are given below: Figure 4: Home screen of SED app Figure 5: The SED device for connection Figure 6: Menu screen of app Figure 7: About screen 16 Figure 8: Settings (main page) Figure 9: Basic settings of volume and SED sensitivity Figure 10: Device setup page Figure 11: Detail page of device setup for mounting position of SED 17 Figure 12: More page for feedback, contacts, sharing and social media 18 Appendix 4 - Summary of Questionnaires: Questionnaires were collected from 170 drivers of haulage companies, government organisations and drivers from the general public. Summary of the survey is given below: 1. Have you ever fallen asleep while driving? Yes More than once No Don't know 97 26 35 12 2. How often are you distracted (off the road) by call, text messaging or any other thing while driving? Very often Sometimes No Don't know 75 73 11 11 3. Are you worried about falling asleep while driving on highways or motorways? Very worried Worried Not Worried Don't know 63 78 21 8 4. Are you worried about being distracted by phone, text messaging or any other thing while driving? Very worried Worried Not Worried Don't know 56 93 14 7 5. How will you feel if you are provided with a solution that alerts you if you are distracted (off the road) or fall asleep while driving? Very safe Safe Slightly Safe Not at all safe 143 23 2 2 6. Will you like a solution for distraction/sleep detection in which you will need to wear something for example paste or wear a sensor and/or carry an extra device every time you drive? Very Like Like Will not like Strongly Dislike 5 12 87 66 7. Will you like to use a solution in which you would need to enter data every time before driving? Very like Like Will not like Strongly Dislike 15 17 60 78 8. Will you like to use a camera based solution for distraction/sleep detection in which camera is set/configured inside car and you do not need to wear anything or enter data every time before driving? 19 Very Like Like Will not like Strongly Dislike 86 67 15 2 9. As a solution to the problem, will you prefer a solution that projects the probability that you will fall asleep OR a solution that works on real time data (alerts the driver when he/she falls asleep)? Strongly prefer projection on probability Alerts based on real system Any of these Don’t know 13 134 9 14 10. If you are provided with a camera based solution for sleep/distraction detection, will you use it? Very likely Likely No Don't know 98 59 9 4 11. Will you use a solution in which your supervisor/manager is sent an alert if you fall asleep or are distracted off the road? Yes No Yes but only if incentives are offered Don't know 34 75 18 43 12. Have you ever found any technology solution for this problem which has been designed based on the specific needs of the Pakistani drivers? Yes Never Not any product for this or other road safety issues Don't know 2 133 18 17 13. Drivers are the most important to ensuring safety of all passengers on board and if drivers fell asleep then safety of all passengers onboard will be compromised. Very Agree Agree Disagree Don't know 111 55 2 2 14. As a passenger which vehicle will you prefer for a long journey? Vehicle with solution for driver’s sleep detection/distraction Vehicle without solution for driver’s sleep detection/distraction 167 3