To transport sources such as petroleum products, oil, and water for long-distance in a fast manner, pipelines are majorly preferred due to their reliability and durability    . These pipelines are easy to construct and also it is more economical than any other method of transportation. Since these pipelines carry highly flammable substances, they can be easily damaged by the outside environment and other forces which results in the formation of leak holes  , blockages  , corrosion  , and other damages   to the pipeline. These damages will pave the way for the occurrence of the disaster which is very harmful to mankind and its surrounding environment. In recent years, several underground chemical pipeline accidents in abroad such as Alabama gasoline disaster in 2016, Kentucky gas explosion in 2019, and local accidents like Kaohsiung gas explosion in 2014, Xizhi’s oil spill in 2015, and Yunlin Formosa gas explosion in 2019 grabbed the researcher’s attention to focus more on the inspection of pipelines. Because these accidents are caused majorly by leakage of substances due to the leak holes in the pipeline. Therefore, these pipelines, if not properly maintained, are a high risk for the people  and also it will lead to major losses  to the industry.
For the inspection of pipeline defects where the direct human activities in analyzing those defects are low, many research was focused on an optical inspection system which consists of a Closed-Circuit TV (CCTV) camera fitted on a mobile platform   . Thus the stored images from the videotape are analyzed in offline conditions by an engineer. The major drawbacks in analyzing the raw images acquired by the CCTV are 1) poor resolution of the image due to the inconvenient lighting conditions; 2) higher probability of error during the offline assessment by humans; 3) unable to clearly visualize the interiors of the pipe. If the pipelines to be inspected are more, then this approach towards defect detection is a difficult task and also it will lead to a significant increase in the inspection costs. The factors which will affect the inspection time of the pipe are an exorbitant amount of information  and the number of defects. Hence, for a heavily damaged pipe, the time of inspection is much longer than the inspection of an intact pipe section  . Normally small structural imperfections will initiate pipeline damages. The use of CCTV is not advisable for detecting those imperfections  . In some cases, due to the existent blockages, it is more difficult to deploy the camera  .
In order to overcome these drawbacks, an in-line inspection system called the pipeline robotic inspection system has been proposed in this work. This pipeline robotic inspection will acquire the images of the pipeline by using the camera via a stable light source. By processing those acquired images, the defect location and its properties can be extracted     . For acquiring high-quality images of inspection, a ring-type laser diode is preferred as the light source   . The robot is designed in such a way that it should have continuous movement throughout the pipeline than the walking mechanism in order to reduce the time consumption of defect detection and if the pipeline profile has some curved geometries, the continuous movement will provide the optimum solution than the walking mechanism and it is also a more reliable method for examining small defects with high precision.
In this paper, an in-line inspection system utilizing an interactive image synchronization technique has been proposed for identifying the inner defects in the pipeline. This system is well suited to carry out the inspection when there is no fluid flow in the pipe. Since it employs a high precision laser diode, this system can be recommended in analyzing pipeline that requires a high-quality inspection of defects. To generate inspection images, artificial defects were introduced in an eight-inch PLA pipe printed using the 3-D printing technique. With the help of the NI Vision Assistant software, the robot will continuously acquire images by using the light source and camera. Thus, the acquired images are processed by the processing function parameters in the vision assistant system. As a result, the defect location, properties such as the size of defects, the error percentages were tabulated and compared with the actual defect dimensions.
2. Fault Detection Methodology
2.1. Camera-Based Laser Profiler
For ring-type laser diode, an intensity-based optical inspection system was preferred for the inspection of the defect. As represented in Figure 1(a), the robot is designed in such a way that it consists of two major slots for holding the light source and camera. For this light source, a red-emitting laser diode with a wavelength of 650 nm, working voltage of DC 2.5 - 5 V and the power of 0.5 W is used. On the other hand, to acquire the images of inspection, the camera (SJ4000) with the resolution of 2 MP is directly connected to the computer via
Figure 1. (a) Schematic representation of laser profiler; (b) Prototype of pipeline robot.
the USB cable to achieve a real-time interactive image synchronization. The robot is connected with the personal computer (PC) and by using the NI Vision Assistant system, for each movement the images are acquired and processed using the PC. In the case of corroded pipelines, due to the presence of uneven surfaces along the pipeline, there may be chances for the misalignment of the center axis of the camera  . To overcome this, a spring suspension system is coupled with the supporting wheels that are mounted on the internal surface of the pipe as shown in Figure 1(b).
2.2. Optical Path
The optical path is defined as the path traveled by the light from the laser diode to the pipe wall and back to the camera. In general, the laser light is aligned in such a way that it is coincident with the center axis of the pipe. As in Figure 1(a), the distance between the laser source to the pipe wall (L) and the laser ring illuminated area (A) is obtained by using Equations (1) and (2).
where R is the pipe radius, α1 and α2 are the projection angles. The α1 and α2 are responsible for defining the distance (L) and width of the laser ring projected on the pipe wall. By increasing the α2, the illuminated area (A) also increases accordingly  . The above equations are suitable for the ideal case. In some cases, when the laser light is slanted, the center axis of the pipe is not coincident with the laser diode. Due to this misalignment, the circular laser ring can be changed to an oblique shape. Equations (3) and (4) defines the upper and lower bounds for distance L.
where (x0, y0) represents the coordinates of the laser light position after misalignment and θ is the angle of misalignment with respect to the center axis   . In our setup, there is no coincidence between the pipe center axis and the laser. At the same time, the laser diode is not slanted but the laser diode and the pipe center axis are very close to each other. An adaptive design combined with the respective optics will pave the way for the best inspection results  .
2.3. Image Processing Stages
With the help of LabVIEW NI Vision Assistant software, the acquired images are processed by various processing function parameters (Figures 2(a)-(e)) as follows. First, the acquired raw image is masked for the defined Region of Interest (ROI) for eliminating the unwanted areas in the acquired image. After masking, all the particles in the defined ROI are highlighted using the RGB threshold function. In addition, for obtaining the desired ring profile, the thresholded image is processed using the advanced morphology function and to obtain the smooth edges of the ring profile, the basic morphology function is used. Finally, by using the contour analysis function, defect information such as the size and depth of defects can be obtained.
3.1. Design of Pipeline Robot
The frame of the robotic system is fabricated using 3D printing technology. PLA is preferred as the most suitable filament for the 3D printing process  . Because it exhibits good mechanical properties which are more efficient during the Fused Deposition Modeling (FDM) process than any other polymer    . With the help of Pro/E software, the frame is built according to the dimensions along with the suspension system. The entire robot body is divided into two units. The first unit namely the visual inspection unit consists of the slots for holding the light source and camera. It is connected to the second unit called the power carrier unit via a universal joint. The power carrier unit provides the necessary power for the movement of the robot. It consists of an Arduino UNO, programmed using Sketch software. This program controls the DC motor via the L298N motor control board which provides the axial movement of the robot throughout the pipe. In addition, a rotary encoder (HTR-34-360A-V) is also
Figure 2. Stages of image processing of inspected image: (a) Acquired raw image; (b) Image masking; (c) Color threshold; (d) Advanced morphology; (e) Basic morphology.
used in the power carrier unit to record the moving distance of the carrier in the pipeline. Both units have the spring suspension system and supporting rollers to avoid the misalignment of the devices with the center axis and to provide free movement during the inspection.
Figure 1(b) shows the prototype model of the proposed robot system. It moves axially into the pipeline along with the detecting unit. The light source is projected onto the inner surface by the ring-type laser, allowing for the clear identification of defects. Raw images were captured in the inspection zone consisting of the defect and non-defect areas. Thus, the captured image was processed with the aid of the NI vision assistant system to identify the defect location.
3.2. Types of Defects
Three sets of defects were introduced in an 8-inch diameter PLA pipe as shown in Figure 3. For analyzing the defect characteristics, the PLA pipe is preferred as a substitute for the petrochemical pipeline because it is more convenient to introduce a variety of defects with different parameters and it can be easily fabricated using a 3D printer. The first set consists of an 8 mm diameter hole of various depths as 1, 2 and 4 mm (Figure 3(a)). The second set consists of 5, 7.5 and 10 mm diameter holes of varying depths as 3, 5 and 7 mm as in Figure 3(b). The third set of defects is entirely based on the sources of transportation in the pipeline. If the pipeline transports some chemical substances like petroleum products, after long-term usage there will be a formation of sediment residues in the pipeline. These residues will cause the pipeline wall to rupture due to corrosion. In order to detect these types of sediment residues, three squared sediments of varying sizes with different heights were created in the pipe. Each squared sediment is located at a certain angle to the other squared sediment as shown in Figure 3(c).
4. Results and Discussion
The inner surface of the pipe is irradiated using the ring-type laser. The defects
Figure 3. Types of defects: (a) 8 mm diameter hole at 1, 2 and 4 mm depth; (b) 5, 7.5 and 10 mm diameter at 3, 5 and 7 mm depth; (c) Squared sediments of size 6, 9 and 12 mm at 1.5, 2.5 and 3.5 mm height.
can be easily identified by the changes in the intensity level along the laser ring. In case of small defects (cracks), there will be an increase in the intensity level (bright) at the edges of the defect and for holes, we can observe two forms of intensity changes, an increase in the intensity at the edges of the defect and a drop in intensity at the inside of the defect  .
For analyzing each set of defects and its characteristics, the NI Vision Assistant software is used for acquiring and processing the images. This software consists of predefined processing function parameters which are very helpful in extracting the defect characteristics. In this, the counter analysis function will provide the distance from counter to template plot which describes the defect. The x-axis in this plot will represent the number of pixels along the circumference of the pipe and the y-axis represents the halo offset distance from the counter to the template.
After processing the acquired images, there will be a drop in intensity in the defect portion when compared with the non-defect portion as in Figure 4(b). To compute the size of the defect, the number of pixels in the defect portion needs to be calculated. In an image, each pixel is represented by its coordinates. If we find the start and end coordinates of the pixel in the defect portion  , the number of pixels in the defect portion can be calculated by using Equation (5).
where (x1, y1) and (x2, y2) represent the pixel coordinates at the start and end of the defect.
4.1. Calculation of Pixel Resolution
In an image, the resolution is defined as the number of pixels that the image consists of and it also provides the extent of details that can be extracted from an image. Thus, the image resolution and the physical dimensions of the image can be related to determine the quantity of space that each pixel will represent in the image. Equations (6) and (7) are used to calculate the space contributed by each pixel in an image  . The pixel can be represented using the pixel width (Pw) and pixel height (Ph).
where Iw and Ih are the image width and height, c and r represent the number of pixels in columns and rows of an image. Here, the acquired image consists of 1920 pixels in columns and 1080 pixels in rows, Iw and Ih values are 282 mm and 159 mm respectively. So, the calculated pixel width and pixel height values are 0.1469 and 0.1472 mm/pixel. These sets of values can be used as the conversion parameter for calculating the defect size because, from NI Vision Assistant, the defect size is represented in pixels.
4.2. Fixed Diameter Holes in Various Depths
In the first set, the defects are analyzed based on their position along the circumference of the pipe. For the defect of an 8 mm diameter hole, the required laser ring profiles were obtained (Figure 4(a)) after processing the acquired image with the aid of processing function parameters provided by the NI Vision Assistant software. From the laser ring profile (Figure 4(b)), the defect can be easily visualized because at the defect portion there will be a drop in intensity level than the non-defect portion. For an 8 mm diameter hole of 1 and 2 mm depth, the necessary laser ring profile was detailed in Appendix A. By applying the centroid and caliper function in vision assistant software, the angle of the defect with respect to the circumference of the pipe is displayed in a clockwise manner as in Figure 4(b).
The counter analysis function plays a significant role in calculating the number of pixels in the defect portion. This function provides the distance from counter to template plot which is obtained by comparing the counters. The pixel coordinates at the start and end of the defect portion are extracted from the x-axis of this plot (Figure 5) and the number of pixels in the defect portion is calculated using Equation (5). This pixel can be represented in mm by multiplying with the calculated pixel width. As a result, the size of the defect in mm can be obtained.
The peaks in this plot will represent the halo offset distance (depth of defect) in pixels. From Figures 5(a)-(c), we observe that the peak is slightly increased when the depth of defect is also increased. Since our robot moves in an axial direction throughout the pipe, the camera is unable to capture the actual depth of
Figure 4. For the first set: (a) Laser ring profile of acquired image for 8 mm diameter hole with 4 mm depth; (b) Processed image representing the defect and non-defect portion. The brown circle defines the non-defect region and the yellow circle defines the defect region with a drop in intensity level. The defect is located at an angle of 276.6˚ from the horizontal axis. For the second set; (c) Laser ring profile of acquired image for 10 mm diameter hole with 7 mm depth; (d) Processed image. The defect is located at an angle of 288.2˚ from the horizontal axis.
the defect. Due to this, the obtained peak value which defines the depth of defect is not considered for calculating the actual defect depth. Hence, our research is focused on the calculation of the size of the defect than the depth.
Table 1 shows that for an 8 mm diameter hole of varying depths, the pixels in the defect portion are calculated and converted to their respective size in mm and by using the error formula, the error percentage was computed. From the table, by comparing the actual defect and calculated defect values, our robot system provides a very less percentage of error (less than 5%).
4.3. Different Diameter Holes of Varying Depths
In the second set, three holes of diameter of 5, 7.5, and 10 mm of varying depths
Figure 5. Distance from counter to template plot for 8 mm diameter hole at, (a) 1 mm depth; (b) 2 mm depth; (c) 4 mm depth.
Table 1. Percentage of error for fixed diameter holes in various depths.
along the circumference of the pipe were introduced and the defect analysis is carried out as same as the first set. The laser ring profile (Figure 4(d)), the pixels in the defect portion and the error percentage are calculated and tabulated in Table 2.
From Figure 6, it can be inferred that when the defect size (hole diameter) is increased, the number of pixels in the defect portion is also increased due to the increase in the intense area (Appendix B) occupied by the defect. Table 2 shows that for different diameter holes, the calculated number of pixels in the defect portion is also increased and thus the obtained pixels are converted to their respective size by multiplying with the pixel width and the error percentage was computed. In the second set also, the error percentage was less than 5%.
Figure 6. Distance from counter to template plot for, (a) 5 mm diameter hole at 3 mm depth; (b) 7.5 mm diameter hole at 5 mm depth; (c) 10 mm diameter hole at 7 mm depth.
Table 2. Percentage of error for different diameter holes of varying depths.
4.4. Sediment Residues
The third set of defects represents the sediment residues formed due to the transportation of chemical substances. Three squared sediments were created in different sizes of varying height and those sediments were analyzed in the same way as the first set. In the case of sediments, the defect portion occurs inside of the ellipse (Figure 7(b)) which ensures that our robot system can also measure the height of sediments. Because during the axial movement, the robot can detect the sediment size along with the height.
After using the counter analysis function, the distance from counter to template plot was obtained for the sediments as shown in Figure 7(c). From the x-axis of this plot, it is observed that the number of pixels in the sediment portion increases with an increase in the sediment size which is due to the increase in the intense area occupied by the sediment. The peaks in this plot also increases with an increase in sediment’s height. The respective angle between each sediment is also checked with the existing angle with the help of the caliper function as shown in Figure 7(b). By using Pw and Ph values, the sediment size, height, and error percentage were calculated and presented in Table 3.
Figure 7. (a) Laser ring profile of squared sediments (acquired image); (b) Processed image after applying caliper function for locating angle of defect; (c) Distance from counter to template plot of squared sediments.
Table 3. Percentage of error for squared sediments.
This paper aims to overcome the drawbacks of the raw CCTV footage used for the inspection of pipelines. By using the ring-type laser inline inspection system, we will be able to clearly visualize the interiors of the pipe and the obtained images have better resolution due to the use of laser as a light source. This robot system serves as a real-time interactive image synchronization system for acquiring the inspected images. The processing function parameters provided by the NI Vision Assistant software process the acquired image for calculating the number of pixels in the defect portion. Three sets of defects were analyzed and as a result, this system can accurately measure the size of the defect and in the case of sediment residues, it can measure both the size and height of sediments. Thus, in all sets of defects, the error percentage obtained was less than 5%.
This research was supported by the Ministry of Science and Technology of Taiwan (Grant No. MOST 108-2637-E-027-001).
Appendix A. Laser Ring Profile of Fixed Diameter Holes in Various Depths
Figure A1. For first set: (a), (c) Laser ring profile of acquired raw image for 8 mm diameter hole with 1 and 2 mm depth. (b), (d) Processed image after applying processing function parameters for determining the defect where yellow ring represents the defect position.
Appendix B. Laser Ring Profile of Different Diameter Holes of Varying Depths
Figure B1. For second set: (a), (c) Laser ring profile of acquired raw image for 5 and 7.5 mm diameter hole with 3 and 5 mm depth. (b), (d) Processed image after applying processing function parameters for determining the defect.
 He, X., Iasmin, M., Dean, L.O., Lappi, S.E., Ducoste, J.J. and de los Reyes III, F.L. (2011) Evidence for Fat, Oil, and Grease (FOG) Deposit Formation Mechanisms in Sewer Lines. Environmental Science & Technology, 45, 4385-4391.
 De Belie, N., Monteny, J., Beeldens, A., Vincke, E., Van Gemert, D. and Verstraete, W. (2004) Experimental Research and Prediction of the Effect of Chemical and Biogenic Sulfuric acid on Different Types of Commercially Produced Concrete Sewer Pipes. Cement and Concrete Research, 34, 2223-2236.
 Pohls, O., Bailey, N.G. and May, P.B. (2002) Study of Root Invasion of Sewer Pipes and Potential Ameliorative Techniques. Acta Horticulturae, 643, 113-121.
 Cheng, S.J. (2014) Studies of the Small Leakage in Buried Gas Pipeline under the Condition of Soil Properties. Applied Mechanics and Materials, Trans Tech Publications, 501-504, 2266-2270.
 Realmuto, G.M., Wagner, N. and Bartholow, J. (1991) The Williams Pipeline Disaster: A Controlled Study of a Technological Accident. Journal of Traumatic Stress, 4, 469-479.
 Girgin, S. and Krausmann, E. (2016) Historical Analysis of US Onshore Hazardous liquid Pipeline Accidents Triggered by Natural hazards. Journal of Loss Prevention in the Process Industries, 40, 578-590.
 Hawari, A., Alamin, M., Alkadour, F., Elmasry, M. and Zayed, T. (2018) Automated Defect Detection Tool for Closed Circuit Television (CCTV) Inspected Sewer Pipelines. Automation in Construction, 89, 99-109.
 Haurum, J.B. and Moeslund, T.B. (2020) A Survey on Image-Based Automation of CCTV and SSET Sewer Inspections. Automation in Construction, 111, Article No. 103061.
 Read, G.F. and Nedwell, P. (1996) Repair and Renovation. In: Read, G.F. and Vickridge, I.G., Eds., Sewers, Butterworth-Heinemann, Oxford, 204-232.
 Rayhana, R., Jiao, Y., Zaji, A. and Liu, Z. (2020) Automated Vision Systems for Condition Assessment of Sewer and Water Pipelines. IEEE Transactions on Automation Science and Engineering, 1-18.
 Joint Task Force of the Water Environment, & American Society of Civil Engineers. (1994) Existing Sewer Evaluation and Rehabilitation. ASCE Manuals and Reports on Engineering Practice, No. 62, WPCF Manual of Practice, No. FD-6. American Society of Civil Engineers, New York.
 Kuntze, H.B. and Haffner, H. (1998) Experiences with the Development of a Robot for Smart Multisensoric Pipe Inspection. Proceedings of the 1998 IEEE International Conference on Robotics and Automation, Vol. 2, Leuven, 20 May 1998, 1773-1778.
 Huynh, P., Ross, R., Martchenko, A. and Devlin, J. (2015) Anomaly Inspection in Sewer Pipes Using Stereo Vision. 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, 19-21 October 2015, 60-64.
 Nassiraei, A.A., Kawamura, Y., Ahrary, A., Mikuriya, Y. and Ishii, K. (2007) Concept and Design of a Fully Autonomous Sewer Pipe Inspection Mobile Robot “Kantaro”. Proceedings of 2007 IEEE International Conference on Robotics and Automation, Rome, 10-14 April 2007, 136-143.
 Duran, O., Althoefer, K. and Seneviratne, L.D. (2002) Automated Sewer Pipe Inspection through Image Processing. Proceedings of the 2002 IEEE International Conference on Robotics and Automation, Washington DC, 11-15 May 2002, 2551-2556.
 Duran, O., Althoefer, K. and Seneviratne, L.D. (2007) Automated Pipe Defect Detection and Categorization Using Camera/Laser-Based Profiler and Artificial Neural Network. IEEE Transactions on Automation Science and Engineering, 4, 118-126.
 Safizadeh, M.S. and Azizzadeh, T. (2012) Corrosion Detection of Internal Pipeline Using NDT Optical Inspection System. NDT & E International, 52, 144-148.
 Duran, O., Althoefer, K. and Seneviratne, L.D. (2003) Pipe Inspection Using a Laser- Based Transducer and Automated Analysis Techniques. IEEE/ASME Transactions on Mechatronics, 8, 401-409.
 Liu, W., Zhou, J., Ma, Y., Wang, J. and Xu, J. (2017) Fabrication of PLA filaments and Its Printable Performance. IOP Conference Series: Materials Science and Engineering, 275, Article ID: 012033.
 Farah, S., Anderson, D.G. and Langer, R. (2016) Physical and Mechanical Properties of PLA, and Their Functions in Widespread Applications—A Comprehensive Review. Advanced Drug Delivery Reviews, 107, 367-392.
 Tymrak, B.M., Kreiger, M. and Pearce, J.M. (2014) Mechanical Properties of Components Fabricated with Open-Source 3-D Printers under Realistic Environmental Conditions. Materials & Design, 58, 242-246.
 Eutionnat-Diffo, P.A., Chen, Y., Guan, J., Cayla, A., Campagne, C., Zeng, X. and Nierstrasz, V. (2019) Stress, Strain and Deformation of Poly-Lactic Acid Filament Deposited onto Polyethylene Terephthalate Woven Fabric through 3D Printing Process. Scientific Reports, 9, Article No. 14333.
 Tamura, Y., Kanai, I., Yamada, K. and Lim, H.O. (2016) Development of Pipe Inspection Robot Using Ring-Type Laser. 2016 16th International Conference on Control, Automation and Systems (ICCAS), Gyeongju, 16-19 October 2016, 211-214.