Real-Time Detection of Human Drowsiness via a Portable Brain-Computer Interface

Show more

1. Introduction

Human drowsiness refers to a physiological state of reduced mental or physical performance resulting from insufficient sleep, long duty periods or irregular work hours. It is attributed to millions of car crashes [1] , 4% - 7% of civil aviation incidents [2] , and 3% of maritime accidents [3] . Detecting the operator’s drowsiness in real-time is a key to prevent such kinds of tragedies.

Conventional optical methods for detecting fatigued driving are unreliably sensitive to lighting conditions [4] and difficult to be extended for drug and alcohol use. Existing studies in sleep science and psychology [5] [6] [7] are not suited to real-time applications in transportation. To classify the drowsiness, tra- ditional machine learning [8] requires a large amount of training data to tune the weights of an artificial neural network. The training data are dependent upon changes due to different test subjects. There is an urgent need to develop a formula (or algorithm) that is relatively invariant with test subjects and does not need a significant amount of training data such that it is ready to be used in real time for any random driver or operator. Another problem of existing methods is the low accuracy of predicting drowsiness [5] [9] - [14] .

The main goal of this paper is to design a new polling algorithm for real-time determination of human drowsiness via an affordable brain-computer interface with the following features:

1) Real-time data acquisition: data sampling window is controlled within one second for computing drowsiness.

2) Real-time data processing: execution time of the algorithm alone is controlled within one millisecond.

3) General algorithm: a general-purposed polling formula is designed and is approximately invariant with test subjects. It does not need the tuning from training data, overcoming a major drawback of existing methods.

4) Affordability: the brain-computer interface is controlled under $200, excluding the cost of computer.

The rest of this paper is organized as follows. In Section 2, the materials and methods related to this study are provided. Next, experimental and computing results are given in Section 3 together with discussion. In Section 4, some conclusions and future work are presented.

2. Materials and Methods

2.1. Materials

In this study, two types of brain-computer interface devices were used:

1) A product from Open BCI (data sampling rate: 250 Hz).

2) A product from Emotiv (data sampling rate: 128 Hz).

Both companies sell products under $200, which are affordable to regular users, as shown in Figure 1.

(a) (b)

Figure 1.Test devices. (a) OpenBCI device; (b) Emotiv device.

In details, the following main materials were used, as shown in Figure 1:

1) Gold-coated electrodes and conductive paste (Open BCI).

2) A data acquisition board (Open BCI).

3) A data wireless receiver (Open BCI).

4) A battery power unity (Open BCI).

5) Integrated electrodes and a data acquisition unit (Emotiv).

6) A data wireless receiver (Emotiv).

7) A bottle of all-purpose saline solution (Emotiv).

2.2. Methods

The outline of our approach is given in Figure 2(b). First, a brain-computer interface device was used to get the brain wave data of test subjects, as illustrated in Figure 2(a). Then, the data was input into the MATLAB software. In MATLAB, a general-purpose polling algorithm was implemented for computing the depth of drowsiness. Last, the computed depth of drowsiness was compared with the inquired depth of drowsiness and with one of the existing methods [5] .

10 - 20 system is an internationally recognized method to specify the location of scalp electrodes for electroencephalography (EEG) test [15] . In this paper, two ear positions (A1 and A2) were used as a reference and a bias position. Based on the previous studies in sleep science, P3 and C4 (or P7 and F4) were chosen as measurement points for obtaining brain waves of test subjects. Five frequency bands (delta, theta, alpha, beta and gamma) of brain waves are defined by Figure 3(b).

(a)(b)

Figure 2. Outline of our approach. (a) A brain-computer interface; (b) Overall approach of our method.

(a)(a)(c)

Figure 3. Brain map, signal processing, and frequency bands. (a) Brain map (source: NR Sign Inc.); (b) Different frequency bands of brain waves; (c) Example of Fourier transform (source: Wikipedia.org).

Brain waves contain much random information that is difficult to be analyzed in time domain (Figure 3(b)). Fourier transform was used to convert the waves to frequency domain (Figure 3(c)) by using Equation (1). It is a tool to break a brain wave into a set of sine or cosine functions with different amplitudes and frequencies:

(1)

・ is a brain wave (time function)

・ is a frequency, ,

・ is the length of sampling window

・ Euler’s formula:

・ represents the amount of frequency in the breakup

In frequency domain, the norm of different frequency bands (delta, theta, alpha, beta and gamma) is defined as biometric markers for drowsiness:

(2)

where is the normalized amplitude of Fourier transform of the brain wave at frequency. The normalization is done by a division from sampling frequency F_{s} (128 or 250 Hz in this paper). Normalized band norms are defined as

(3)

A general-purpose polling formula was designed to predict the human drowsiness:

1) Traditionally, the drowsiness is quantified by a binary variable: Yes or No, which is not accurate enough to describe such a complex process. A real number variable, depth of drowsiness, is proposed to precisely describe the different levels of drowsiness ().

2) In computer graphics [16] , millions of different colors are represented by a linear combination of three basic components: RGB, where R, G, B represent red, green, and blue components, respectively. Inspired by this treatment, the drowsiness is represented by a linear combination of several basic frequency components.

3) In sleep science, different frequency channels are often used to investigate sleep patterns. In signal processing, Fourier transform [17] is an effective tool to convert signals from time domain to frequency domain, and the norm of frequency bands can be used as basic components to quantify human drowsiness. Thus, the norm of frequency bands of brain waves is considered as biometric markers for drowsiness in this paper.

4) Inspired by political voting, a new polling scheme, i.e., judging the drowsiness on the basis of the vote of frequency bands, is designed to express the relation between the depth of drowsiness and the norm of frequency bands:

(4)

, , ,

where N_{δ}, N_{θ}, N_{α}, N_{β}, N_{γ} are normalized band norms for δ, θ, α, β, and γ channels, respectively., , , are the corresponding weights.

The derivation of the above formula is given in Appendix A.

2.3. Testing

For calibrating the sampling rate of the hardware, A stopwatch and the data acquisition software of the devices were used. Table 1 shows the test data of stopwatch and acquisition software of Open BCI. The timing data are quite close to each other.

Table 2 is the time acquisition result of Emotiv device in a period of 0.5 second. It indicates that the sampling rate is 128 Hz and sampling time is quite accurate. The absolute value of voltage does not matter in this study because our formula needs only the normalized band norm, which is invariant from voltage amplitude (Appendix B).

A Human Participant Form was approved by a local IRB in 2015 and each participant gave his/her permission before the experiment. Ten human subjects were measured with names replaced by pseudo identification numbers, with four of them tested only in waking states. The main test steps of this study are:

1) Assembling and calibration of the brain-computer interface devices.

2) Design and development of the biometric algorithm (i.e., a polling scheme) as well as the setup of the brain-computer interface device.

3) Collection of brain wave data by wearing a brain-computer interface device. Measurement last for about 6 - 7 seconds at a sampling frequency (F_{s} = 250 Hz). Drowsy state: a) Test time was chosen around midnight; b) Subjects were asked to click the “Data Collection” button once they felt drowsy. Waking state: a) Test time was chosen during daytime; b) Severe movement was avoided.

4) Inquiry of observed depth of drowsiness from test subjects. Drowsy stage: Extremely sleepy (0.95), Very sleepy (0.8), Moderately sleepy (0.65), Lightly sleepy (0.5).Waking stage: Extremely alert (0.05), Very alert (0.2), Moderately alert (0.35), Lightly alert (0.45).

Table 1. Calibration of sampling rate of the OpenBCI device.

Table 2. Time acquisition result of Emotiv device for a period of 0.5 second.

5) Conversion of brain data to MATLAB data format and execution of the designed polling algorithm for predicting the depth of drowsiness.

6) Analysis and validation of the algorithm.

3. Results and Discussions

3.1. Brain Waves and Normalized Frequency Bands

As shown in Figure 4, plot( ) function in MATLAB was used to draw brain waves in both time and frequency domains. Bar( ) function was used to draw the normalized band norms of five frequency bands (Delta, Theta, Alpha, Beta, and Gamma).

3.2. Impact of Sensor Channels

Our biometric markers for drowsiness are roughly invariant between C4 and P3 channels, as illustrated in Figure 5. This is an important feature that eliminates

Figure 4.Measured brain waves and their Fourir transformation.(a) Waking stage;(b)Drowsy stage.

Figure 5. Influence of sensor channels on the normalized frequency bands.

stringent requirement on the location of electrodes on human scalp. It is difficult to put a sensor at the exact location of a particular channel specified by the 10 - 20 electrode system.

3.3. Data Repetition

The depth of drowsiness varies more severely in waking stage than in drowsy stage. This may be due to the fact that test subjects are more likely influenced from environment during the waking stage. Even with the variation in the waking stage, there is a clear-cut between the two stages, as shown in Figure 6. This figure plots the drowsiness measured five times on a test subject.

3.4. Execution Time

The execution time of the MATLAB code is within 1 millisecond (not including the data acquisition time, which is 1 second for each estimate), as listed in Figure 7. The sum of these two costs supports real-time applications in transportation control.

Figure 6. Fluctuation of depth of drowsiness of a test subject.

Figure 7. Execution time of our algorithm without counting data acquisition and storage.

3.5. Age Effect

The computed depth of drowsiness is also generally invariant from ages of test subjects, as illustrated in Figure 8. This avoids the need of a large amount of training data in existing methods.

3.6. Accuracy Comparison

Figure 9 provides a comparison of our method with one recent study [5] and indicates that our algorithm provides a more accurate clear-cut between drowsy and waking stages. In details, Table 3 shows the results of some tests in this study. The thresholds 0.65 and 0.5 were used to determine the state predictions (w: waking; d: drowsy) in the fourth and sixth columns, respectively. Red color in the table represents the failed cases. Table 4 is a summary of the comparison, indicating 82% and 70% accuracy for our method and the recent method [5] , respectively. Furthermore, the threshold is not sensitive to our method because any value between 0.65 and 0.75 generates a similar accuracy.

4. Conclusions

The following concluding remarks can be drawn from this study:

1) In this paper, we proposed a new concept: depth of drowsiness, which can more precisely describe the drowsiness than existing binary description.

Figure 8. Age effect of our method.

(a)(b)

Figure 9. Accuracy comparison between our method and one recent method [5] . (a) A recent method; (b) Our method.

Table 3. Accuracy comparison between our method and one recent method [5] .

Table 4. Summary of comparing our method with one existing method based on the results of 50 test cases.

2) After many attempts, we found a set of effective biometric markers for drowsiness: normalized band norms. These markers do not change with scaling the voltage amplitude of brain waves, eliminating the need for calibrating the voltage output of the brain-computer interface devices.

3) We designed and implemented a new polling algorithm for computing the depth of drowsiness. The time cost of data acquisition and processing for each estimate is about one second, which is well suited for real-time applications.

4) Test results show that the depth of drowsiness computed by the new method is generally invariant from ages of test subjects and sensor channels (P3 and C4). This eliminates the need of training data required by existing methods. Therefore, our method is better suited to a random driver than existing methods.

5) The cost of the brain-computer interface devices (not including the computer) can be under $200, which is affordable to regular users.

6) In comparison with a recent study [5] , our method increases the success rate of separating drowsy and waking stages from 70% to 82%.

Possible future work may include:

1) Apply the method to a large-scale investigation.

2) Investigate an optimal way to place scalp electrodes.

3) Test the impact of environment such as vehicle vibration.

4) Extend our polling algorithm to include signals from electromyography (EMG) and electrocardiogram (EKG or ECG).

5) Study on alcohol and drug influence, blackout, road rage, and medical emergency.

Appendices

A. Derivation of a Polling Formula

1) First let sleep ® 1.0 and waking ® −1.0

2) Use a linear relation to represent (a variable for the state of brain)

(A1)

where N_{δ} and N_{β} are normalized frequency bands for δ and β channels, as defined in Equation (3). c_{δ} and c_{β} are two coefficients.

3) From prior knowledge in sleep science, δ channel contributes to sleep, while β channel contributes to the waking stage. Then, we have

4) Expand the contributions of θ and α channels:

(A2)

where N_{θ} and N_{α} are defined in Equation (3). Assume c_{θ} = 0.5 and c_{α} = −0.5 for their corresponding contributions to waking and sleep stages, respectively.

5) Consider N_{γ}’s contribution to the waking stage, we rewrite Equation (A2) as

(A3)

6) Map to by letting

(A4)

where and are coefficients. is the depth of drowsiness defined in this paper.

when ®

when ®

By solving the above two equations, we have

and (A5)

Thus, can be expressed as

(A6)

B. Proof of No Need to Calibrate the Biosensor Voltage

Theorem: The computed depth of drowsiness is invariant from scaling the amplitude of brain waves.

Proof:

Given a sequence of N sample, where, , assume that the Fourier transform of the function is expressed as

(A7)

where,. is called the Fourier coefficient. Assume that because of some errors in signal magnification, N samples were multiplied by a scaling factor s, leading to another Fourier coefficient:

(A8)

The substitution of in Equation (2) with and respectively leads to:

(A9)

(A10)

By using in Equation (A9), we have

(A11)

(A12)

By using in Equation (A10), we have

(A13)

Since, where, θ, α, β, γ based on Equations (A11) and (A13), the computed depth of drowsiness in Equation (A8) remains unchanged due to the scaling factor s.

Submit or recommend next manuscript to SCIRP and we will provide best service for you:

Accepting pre-submission inquiries through Email, Facebook, LinkedIn, Twitter, etc.

A wide selection of journals (inclusive of 9 subjects, more than 200 journals)

Providing 24-hour high-quality service

User-friendly online submission system

Fair and swift peer-review system

Efficient typesetting and proofreading procedure

Display of the result of downloads and visits, as well as the number of cited articles

Maximum dissemination of your research work

Submit your manuscript at: http://papersubmission.scirp.org/

Or contact ojapps@scirp.org

References

[1] DOT (2012) Motor Vehicle Crashes: Overview. Traffic Safety Facts Research Notes. U.S. Department of Transportation, National Highway Traffic Safety Administration, Washington DC.

[2] Caldwell, J.A. (2004) Fatigue in Aviation. Travel Medicine and Infectious Disease, 3, 85-96.

[3] Baker, C.C. and Seah, A.K. (2004) Maritime Accidents and Human Performance: the Statistical Trail. MARTECH Conference, Singapore, 22-24 September 2004, 225-239.

[4] Murukesh, C. and Padmanabhan, P. (2015) Drowsiness Detection for Drivers Using Computer Vision. WSEAS Transactions on Information Science and Applications, 12, 43-50.

[5] Yin, Y., Zhu, Y., Xiong, S. and Zhang, J. (2012) Drowsiness DETECTION from EEG Spectrum Analysis. Information in Control Automation and Robotics, Lecture Notes in Electrical Engineering, 133, 753-759.

https://doi.org/10.1007/978-3-642-25992-0_101

[6] Johnson, R.R., Popovic, D.P., Olmstead, R.E., Stikic, M., Levendowski, D.J. and Derka, C. (2011) Drowsiness/Alertness Algorithm Development and Validation Using Synchronized EEG and Cognitive Performance to Individualize a Generalized Model. Biological Psychology, 87, 241-250.

[7] Ogilvie, R.D., Wilkinson, R.T. and Allison, S. (1989) The Detection of Sleep Onset: Behavioral, Physiological, and Subjective Convergence. Sleep, 12, 458-474.

[8] Yegnanarayana, B. (2009) Artificial Neural Network. PHI Learning Pvt. Ltd., Delhi.

[9] Awais, M., Badruddin, N. and Drieberg, M. (2014) A Non-Invasive Approach to Detect Drowsiness in a Monotonous Driving Environment. TENCON 2014, Bangkok, 22-25 October 2014, 1-4.

[10] Brown, T., Johnson, R. and Milavetz, G. (2013) Identifying Periods of Drowsy Driving Using EEG. Annals of Advances in Automotive Medicine, 57, 99-108.

[11] Zhang, A., Bi, J. and Sun, S. (2013) A Method for Drowsiness Detection Based on Tsallis Entropy of EEG. World Congress on Medical Physics and Biomedical Engineering, Beijing, 26-31 May 2013, 505-508.

https://doi.org/10.1007/978-3-642-29305-4_133

[12] Hu, J., Mu, Z. and Wang, P. (2015) Driver Detecting System Based on Brain-Computer Interface. Proceedings of 5th International Conference on Education, Management, Information and Medicine, Shenyang, 24-26 April 2015, 512-515.

[13] Lin, C., Chang, C., Lin, B., Hung, S., Chao, C. and Wang, I. (2010) A Real-Time Wireless Brain-Computer Interface System for Drowsiness Detection. IEEE Transactions on Biomedical Circuits and Systems, 4, 214-222.

https://doi.org/10.1109/TBCAS.2010.2046415

[14] Kaur, R.K. and Kaur, G. (2013) Neural Network Based Drowsiness Detection Using Electroencephalogram. International Journal of Engineering Research & Technology, 2, 2497-2506.

[15] Jasper, H.H. (1958) The Ten-Twenty Electrode System of the International Federation. Electroencephalography and Clinical Neurophysiology, 10, 371-375.

[16] Foley, J.D., van Dam, A., Feiner, S.K. and Hughes, J.F. (1997) Computer Graphics, Principles and Practice. Addison-Wesley Publishing Company, Boston.

[17] Brigham, E.O. and Brigham, E.O. (1974) The Fast Fourier Transform. Prentice-Hall, Englewood Cliffs.