ENG  Vol.12 No.3 , March 2020
Creating a Dataset to Boost Civil Engineering Deep Learning Research and Application
Abstract: With cutting edge deep learning breakthrough, numerous innovations in many fields including civil engineering are stimulated. However, a fundamental issue that civil engineering research community currently facing is lack of a publicly available, free, quality-controlled and human-annotated large dataset that supports and drives civil engineering deep learning research and applications on such as intelligent transportation including connected vehicle, structural health monitoring, and bridge inspection. This paper is a general discussion about demanding needs and construction of a long-anticipated dataset for researchers and engineers in civil engineering and beyond for providing critical training, testing and benchmarking data. The establishment of such a free dataset will remove a major hurdle and boost deep learning research in civil engineering and we hope this work will urge researchers, engineers, government agencies and even computer scientists to work together to start building such datasets. A framework has been developed for the proposed database. Also, some pilot study databases were developed for concrete crack detection, pavement crack detection using normal and infrared thermography, as well as pedestrian and bicyclist detection. A convolution neural network model called Faster RCNN was deployed to check the detection accuracy and a 98% detection accuracy of the proposed datasets was obtained.
Cite this paper: Qurishee, M. , Wu, W. , Atolagbe, B. , Owino, J. , Fomunung, I. and Onyango, M. (2020) Creating a Dataset to Boost Civil Engineering Deep Learning Research and Application. Engineering, 12, 151-165. doi: 10.4236/eng.2020.123013.

[1]   Sato, K., Young, C. and Patterson, D. (2017) An In-Depth Look at Google’s First Tensor Processing Unit (TPU). Google Cloud Big Data and Machine Learning Blog, 12.

[2]   Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K. and Fei-Fei, L. (2009) ImageNet: A Large-Scale Hierarchical Image Database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 20-25 June 2009, 248-255.

[3]   Chollet, F. (2017) Deep Learning with Python. Manning Publications Co., New York.

[4]   Lin, Y.Z., Nie, Z.H. and Ma, H.W. (2017) Structural Damage Detection with Automatic Feature-Extraction through Deep Learning. Computer-Aided Civil and Infrastructure Engineering, 32, 1025-1046.

[5]   Gulgec, N.S., Takáč, M. and Pakzad, S.N. (2017) Structural Damage Detection Using Convolutional Neural Networks. In: Model Validation and Uncertainty Quantification, Volume 3, Springer, New York, 331-337.

[6]   Cha, Y.J., Choi, W. and Büyüköztürk, O. (2017) Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks. Computer-Aided Civil and Infrastructure Engineering, 32, 361-378.

[7]   Qurishee, M.A. (2019) Low-Cost Deep Learning UAV and Raspberry Pi Solution to Real Time Pavement Condition Assessment.

[8]   Cha, Y.J., Choi, W., Suh, G., Mahmoudkhani, S. and Büyüköztürk, O. (2017) Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types. Computer-Aided Civil and Infrastructure Engineering, 33, 731-747.

[9]   Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T. and Omata, H. (2018) Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone. arXiv Preprint arXiv:1801.09454.

[10]   Makantasis, K., Protopapadakis, E., Doulamis, A., Doulamis, N. and Loupos, C. (2015) Deep Convolutional Neural Networks for Efficient Vision Based Tunnel Inspection. 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3-5 September 2015, 335-342.

[11]   Varghese, A., Gubbi, J., Sharma, H. and Balamuralidhar, P. (2017) Power Infrastructure Monitoring and Damage Detection Using Drone Captured Images. 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, 14-19 May 2017, 1681-1687.

[12]   Wang, F., Kerekes, J.P., Xu, Z. and Wang, Y. (2018) Residential Roof Condition Assessment System Using Deep Learning. Journal of Applied Remote Sensing, 12, Article ID: 016040.

[13]   Russakovsky, O., et al. (2015) ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115, 211-252.

[14]   Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) Imagenet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Systems, Springer, New York, 1097-1105.

[15]   Lin, T.-Y., et al. (2014) Microsoft Coco: Common Objects in Context. In: European Conference on Computer Vision, Springer, New York, 740-755.

[16]   Krizhevsky, A., Nair, V. and Hinton, G. (2014) The CIFAR-10 Dataset.

[17]   Torralba, A., Fergus, R. and Freeman, W.T. (2008) 80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 1958-1970.

[18]   Y. LeCun, C. Cortes, and C. Burges (2010) MNIST Handwritten Digit Database. AT & T Labs. Volume 2.

[19]   Knight, W. (2018) The White House Promises to Release Government Data to Fuel the AI Boom.

[20]   Sun, C., Shrivastava, A., Singh, S. and Gupta, A. (2017) Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22-29 October 2017, 843-852.

[21]   Downey, A.S. and Olson, S. (2013) Sharing Clinical Research Data: Workshop Summary. National Academies Press, Washington DC.

[22]   Chollet, F. (2016) Building Powerful Image Classification Models Using Very Little Data. Volume 13.

[23]   Yosinski, J., Clune, J., Bengio, Y. and Lipson, H. (2014) How Transferable Are Features in Deep Neural Networks? In: Advances in Neural Information Processing Systems, Springer, New York, 3320-3328.

[24]   Chourabi, H., et al. (2012) Understanding Smart Cities: An Integrative Framework. 2012 45th Hawaii International Conference on System Science (HICSS), Maui, HI, 4-7 January 2012, 2289-2297.

[25]   Figueiredo, L., Jesus, I., Machado, J.T., Ferreira, J.R. and De Carvalho, J.M. (2001) Towards the Development of Intelligent Transportation Systems. Intelligent Transportation Systems, 2001. Proceedings, Oakland, CA, 25-29 August 2001, 1206-1211.

[26]   Dimitrakopoulos, G. and Demestichas, P. (2010) Intelligent Transportation Systems. IEEE Vehicular Technology Magazine, 5, 77-84.

[27]   Adeli, H. (2008) Smart Structures and Building Automation in the 21st Century. International Symposium on Automation in Construction, 25, 5-10.

[28]   Udd, E. (1993) Fiber Optic Smart Structures. Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Volume 10266.

[29]   Stajano, F., Hoult, N., Wassell, I., Bennett, P., Middleton, C. and Soga, K. (2010) Smart Bridges, Smart Tunnels: Transforming Wireless Sensor Networks from Research Prototypes into Robust Engineering Infrastructure. Ad Hoc Networks, 8, 872-888.

[30]   Lajnef, N., Chatti, K., Chakrabartty, S., Rhimi, M. and Sarkar, P. (2013) Smart Pavement Monitoring System. United States Federal Highway Administration.

[31]   Barbaresso, J., Cordahi, G., Garcia, D., Hill, C., Jendzejec, A. and Wright, K. (2014) USDOT’s Intelligent Transportation Systems (ITS) ITS Strategic Plan 2015-2019.

[32]   Kenney, J.B. (2011) Dedicated Short-Range Communications (DSRC) Standards in the United States. Proceedings of the IEEE, 99, 1162-1182.

[33]   Hartman, K.K. Connected Vehicle Pilots.

[34]   Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016) You Only Look Once: Unified, Real-Time Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779-788.

[35]   Ren, S., He, K., Girshick, R. and Sun, J. (2015) Faster r-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: Advances in Neural Information Processing Systems, Springer, New York, 91-99.

[36]   Farrar, C.R. and Worden, K. (2007) An Introduction to Structural Health Monitoring. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 365, 303-315.

[37]   Kim, S., et al. (2007) Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks. In: Proceedings of the 6th International Conference on Information Processing in Sensor Networks, ACM, New York, 254-263.

[38]   Wu, W., Qurishee, M.A., Owino, J., Fomunung, I., Onyango, M. and Atolagbe, B. (2018) Coupling Deep Learning and UAV for Infrastructure Condition Assessment Automation. 2018 IEEE International Smart Cities Conference (ISC2), Kansas City, MO, 16-19 September 2018, 1-7.

[39]   Polson, N.G. and Sokolov, V.O. (2017) Deep Learning for Short-Term Traffic Flow Prediction. Transportation Research Part C: Emerging Technologies, 79, 1-17.

[40]   Lv, Y., Duan, Y., Kang, W., Li, Z. and Wang, F.-Y. (2015) Traffic Flow Prediction with Big Data: A Deep Learning Approach. IEEE Transactions on Intelligent Transportation Systems, 16, 865-873.

[41]   Mocanu, E., Nguyen, P.H., Gibescu, M. and Kling, W.L. (2016) Deep Learning for Estimating Building Energy Consumption. Sustainable Energy, Grids and Networks, 6, 91-99.

[42]   Li, C., Ding, Z., Zhao, D., Yi, J. and Zhang, G. (2017) Building Energy Consumption Prediction: An Extreme Deep Learning Approach. Energies, 10, 1525.

[43]   NHTS Administration (2017) Federal Motor Vehicle Safety Standards; V2V Communications. Federal Register, 82, 3854-4019.

[44]   Miller, G.A. (1995) WordNet: A Lexical Database for English. Communications of the ACM, 38, 39-41.

[45]   Ryan, T.W., Hartle, R.A., Mann, J.E. and Danovich, L.J. (2006) Bridge Inspector’s Reference Manual. Report No. FHWA NHI, 03-001.

[46]   Russell, B.C., Torralba, A., Murphy, K.P. and Freeman, W.T. (2008) LabelMe: A Database and Web-Based Tool for Image Annotation. International Journal of Computer Vision, 77, 157-173.

[47]   Von Ahn, L. and Dabbish, L. (2004) Labeling Images with a Computer Game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, 319-326.

[48]   Benfield, J.A. and Szlemko, W.J. (2006) Internet-Based Data Collection: Promises and Realities. Journal of Research Practice, 2, 1.

[49]   Kramer, V.H. and Weinberg, D.B. (1974) The Freedom of Information Act. The Georgetown Law Journal, 63, 49.

[50]   Caltrans. Performance Measurement System (PeMS).

[51]   Dot, T. Smartway.

[52]   GDOT. The Georgia Department of Transportation’s Traffic Analysis and Data Application.

[53]   UDO Transportation. ITS Public Data Hub.

[54]   Qurishee, M., Iqbal, I., Islam, M. and Islam, M. (2016) Use of Slag as Coarse Aggregate and Its Effect on Mechanical Properties of Concrete. Proceedings of International Conference on Advances in Civil Engineering, 3, 475-479.

[55]   Al Qurishee, M. (2017) Application of Geosynthetics in Pavement Design.

[56]   Al Qurishee, M. and Fomunung, I. (2000) Smart Materials in Smart Structural Systems.

[57]   Hasnat, A., Qurishee, M., Iqbal, I., Zaman, M. and Wahid, M. (2018) Effectiveness of Using Slag as Coarse Aggregate and Study of Its Impact on Mechanical Properties of Concrete.

[58]   Atolagbe, B. (2019) Automatic Mesh Representation of Urban Environments.

[59]   Islam, M.A. (2018) Intergrading Connected Vehicle Data into the Transportation Performance Measurement Process. The University of Alabama, Birmingham.

[60]   Islam, M.A. (2019) A Literature Review on Freeway Traffic Incidents and Their Impact on Traffic Operations. Journal of Transportation Technologies, 9, 504-516.

[61]   Islam, M.A., Sisiopiku, V.P., Ramadan, O.E. and Hadi, M. (2019) A Framework for Performance-Based Traffic Operations Using Connected Vehicle Data. Simulation (NGSIM), 6, No. 8.

[62]   Al Qurishee, M., Wu, W., Atolagbe, B., El Said, S. and Ghasemi, A. (2019) Non-Destructive Test Application in Civil Infrastructure.

[63]   Al Qurishee, M., Wu, W., Atolagbe, B., El Said, S., Ghasemi, A. and Tareq, S.M. (2008) Wireless Sensor Network and Its Application in Civil Infrastructure.

[64]   Cdot (2020) California Department of Transportation.

[65]   Tdot (2020) SmartWay Traffic.,35.88416963357008,

[66]   Usdot (2020) ITS DataHub.

[67]   Winn, J. (2013) Open Data and the Academy: An Evaluation of CKAN for Research Data Management.

[68]   OK International.