FNS  Vol.12 No.3 , March 2021
Developing a Machine Vision System Equipped with UV Light to Predict Fish Freshness Based on Fish-Surface Color
Abstract: This study assessed the feasibility of developing a machine vision system equipped with ultraviolet (UV) light, using changes in fish-surface color to predict aerobic plate count (APC, a standard freshness indicator) during storage. The APC values were tested and images of the fish surface were taken when fish were stored at room temperature. Then, images’ color-space conversion among RGB, HSV, and L*a*b* color spaces was carried out and analyzed. The results revealed that a* and b* values from the UV-light image decreased linearly during storage. A further regression analysis of these two parameters with APC value demonstrated a good exponential relationship between the a* value and the APC value (R2 = 0.97), followed by the b* (R2 = 0.85). Therefore, our results suggest that the change in color of the fish surface under UV light can be used to assess fish freshness during storage.

1. Introduction

Fish is a leading source of high-quality protein and plays an important role in human nutrition. In recent decades, the harvesting and consumption of fish has continued to increase [1]. In addition to its contribution to human health, the freshness of fish is crucial to consumer acceptance of fish as a food source as well as to the credibility of the international fishery trade [2]. However, after the fish dies, quality degrades quickly owing to pre-harvest and post-mortem factors, with a loss of acceptable appearance, taste, and nutritional quality. The fish can even become toxic. Therefore, the ability to determine freshness is critical to the research and development of the fishing industry.

Normally, consumers utilize sensory methods (color, smell, or touch) to assess fish freshness [3]. Reliable indicators of the sanitary quality and safety of fish [4], include chemical methods to determine the total bacteria count, the total volatile basic nitrogen (TVB-N), and thiobarbituric acid reactive substances (TBARS). However, the conventional sensory, chemical, and microbiological methods are imprecise, time consuming, expensive, destructive, and labor-intensive; a fast, inexpensive, and simple method is needed. In recent years, some fast, non-destructive methods for assessing fish freshness have received attention, including near-infrared spectroscopy [5], front-face fluorescence spectroscopy [6], hyperspectral imaging [7], and image technologies [8]. Among these, the application of machine vision technology coupled with image-processing techniques to ascertain fish quality is a growing area [9] [10] [11] [12], in response to the demand for simple, low-cost, and rapid real-time techniques. Color is considered the most important sensory attribute of fish quality, which changes during storage as freshness deteriorates. It has been employed as an indicator to accept or reject fish products [13] [14]. Machine vision is a promising technique that is currently being investigated for food-color measurement. Acquiring and analyzing images of changes in color of fish in storage provide obvious advantages over conventional methods, by providing a detailed characterization of color uniformity at a pixel-based level of the entire body surface, allowing the quantification of the change in surface color [15]. The most popular color-space models used in food computer vision to detect changes in color are the color model of RGB (red, green, and blue), HSV (hue, saturation, and value), and L*a*b* (lightness from black (0) to white (100), redness from green (−) to red (+), and yellowness from blue (−) to yellow (+)) [16].

In previous studies, we have found that fish surface (main part is scales) was rich in organic protein contents, which contain plenty of tyrosine residuals and show strong fluorescence at excitation wavelength around 365 nm. These fluorescent compounds change regularly due to complex processes of enzymatic autolysis, oxidation, and microbial activities, thus the corresponding fluorescence signals were suggested to be used to assess fish freshness during storage [17]. Based on these findings, we propose the fluorescence signal on a fish’s surface as an effective way to assess fish freshness. Therefore, in this study, we used ultraviolet (UV) light at a wavelength of 365 nm to excite the fish-surface fluorescence signals to obtain fluorescence images. We analyzed the RGB, HSV, and L*a*b* color-space models of UV light image to determine the optimal color parameters to assess fish freshness.

2. Materials and Methods

2.1. Sample Preparation

33 crucian carp (Carassiusauratus) with an average fork length of 20 cm and weight of 250 g were purchased from a local supermarket (Chengdu, China) on September 20, 2019. The fish were transported to the laboratory of the institute of urban agriculture, Chinese academy of agricultural sciences alive and immediately slaughtered by cutting the neck bone. They were stored in a container of room temperature (22˚C ± 2˚C) and 70% humidity for 24 hours. We divided the samples into two groups: the first group of 6 for scanning with UV light at 365 nm. The second group of 27 was stored for the detection of aerobic plate count (APC; a standard freshness evaluation parameter) every 3 hours since the beginning of storage.

2.2. Aerobic Plate Count

Monitoring the count of micro-organisms has been suggested to measure fish freshness [18], in this study we used aerobic plate count (APC) in fish meat as the standard indicator of fish freshness. We detected the APC according to the method described by China National Standard GB 4789.2 (2016). 25 g meat from the dorsal part of the fish samples was weighed aseptically and mixed with 225 mL sterile saline solution, and next homogenized with a Blender (JYL-A100, Joyoung Co., Ltd., China) for 120 s at 10,000 rpm. The resulting meat homogenate was serially diluted in sterile saline solution. Three appropriate portions (1 mL) of serial dilutions were all repeated twice spreading in the Petri dishes, and then mixed well with plate count agar (plate count agar: 5.0 g tryptone, 2.5 g yeast extract, 1.0 g glucose and 15.0 g agar add to distilled 1000 mL water, then adjusted pH at 7.0 ± 0.2, sterilized at 121˚C for 15 min). After the agar solidification, dishes were incubated at 36˚C ± 1˚C for 48 ± 2 h to detect the APC of the samples. The bacterial counts were recorded as colony-forming units (CFU/g). Aerobic plate counts were performed at fish which were stored for 0 h, 3 h, 6 h, 9 h, 12 h, 15 h, 18 h, 21 h, 24 h at room temperature.

2.3. Machine Vision System

We used a machine vision system as referred to in Xu and Sun (2017) [19]. It is shown and described in Figure 1.

This machine vision system includes three main components: a high-resolution camera (Canon 80D digital camera equipped with a Canon EF 18 - 135 nm macro lens (Canon, Tokyo, Japan), with a CPL filter); A UV illumination system (Rsee Corp., Guangdong, China): the light source with excitation wavelength was of 365 nm, by which the fish surface could be strongly excited [17], the lighting was activated only when an image was being taken, a computer (Dell Computer Corp., USA) with a remote capture software installed (version Canon, Japan) to acquire and analyze the images. All images were taken in a dark box to avoid interference from other light sources. For calibrating the camera, the white balance adjustment was manually applied. All images were taken with the same camera specifications (exposure time: 1 s, ISO: 400, f-value: 8, resolution: 1280 × 1024, image type: PNG, no flash). After setting the best illumination, images of fish during 24-hour storage were automatically taken every 10 min by the digital camera and saved in the computer.

Figure 1. Machine vision system.

2.4. Image Treatment

Fish images were treated as follows. First, the region of interest (i.e., the whole surface including head and tail) was selected automatically by using an image-processing program written in Python 3. Second, the low-frequency background noise was removed, the intensity of the individual particle images was normalized, the geometric distortions and gray level were corrected, and blurring and reflections were removed from the images. Finally, the color parameters (R value, G value, B value, H value, S value, V value, L* value, a* value, and b* value) in the region of interest were calculated by converting RGB to different color formats [20].

3. Results and Discussion

3.1. APC Value Variation in Fish

Microbial evaluation of crucian carp samples during storage over a 24 h period at room temperature is shown in Figure 2. The initial microbial load of crucian carp dorsal muscle samples was 12 CFU/g and increased slowly to 405 CFU/g after 15 h storage. It increased sharply after 18 h (2410 CFU/g) of storage onwards, reaching 5655 CFU/g after 24 h in storage. The rapid increase in the APC value began at 18 h storage. This indicates that it is to be recommended that fish, when stored longer than 18 h at room temperature, be well cooked, or it should be stated that they are not fit for human consumption. This result agrees with K value (which is extracted as adenosine triphosphate (ATP) concentration and its breakdown products) as an indicator of freshness to evaluate the various fish types [21]. Furthermore, the direct and exponential relationship (R2 = 0.97) between storage time and APC value in crucian carp indicates that the APC value is an effective indicator for expressing freshness.

3.2. Imaging Crucian Carp by Machine Vision

Images of crucian carp reveal changes in color under UV light from 0 h to 24 h of storage at room temperature (Figure 3). The fish surface initially displayed weak fluorescence and increased with longer storage. The fish surface is covered mainly by scales, which contain 41% - 81% organic protein, including 24% ichthylepidin and 76% collagen. These proteins, which are mostly fluorescence,

Figure 2. APC value change in crucian carp when stored at room temperature (room temperature).

Figure 3. The surface of crucian carp changes under UV light, stored at room temperature.

could be excited by UV light and express the obvious fluorescence signal [22]. Moreover, the mucus (and epidermis), which is a complex jelly-like fluid formed predominantly by macromolecules and made mainly of proteins [23], covers the surface of the fish, to some extent, to the fluorescence signal on the fish surface. Therefore, the fish surface exhibited fluorescence under UV light. During storage, the micro-organisms are expected to multiply rapidly and consume the proteins into tyrosine in the mucus, which will significantly increase total fluorescence intensity levels [17]. As a result, fluorescence in the image under UV light increased as the time in storage increased.

3.3. Color Changes on the Fish Surface as Represented by Three Color Models

Figure 4 shows the changes in color parameter on the surface of crucian carp under UV light during storage at room temperature as represented by three color models: RGB, HSV, and L*a*b*. In the RGB color model, R, G, and B in the UV-light image decreased during the first 6 h (B decreased first at 12 h), and then increased until the end of storage. The result of concomitant proportional change in the values of R, G, and B may change the lightness of the fish surface [24]. The HSV color space is like HSI, which is closely related to the physiology of the human eye [25], and the three components (H, S, and V) are relatively

Figure 4. Variation in the color parameters (R, G, H, S, V, B, L*, a*, b*) on the surface of crucian carp under UV light during storage at room temperature.

independent. In the HSV color space, H is the color portion of the model, extracted as a number from 0˚ to 360˚. The H value was range in 100˚ to 180˚ (green to cyan) from UV-light image. The S value describes the amount of gray in a color. The V value works in conjunction with saturation and describes the brightness or intensity of the color. In the UV-light image, the S and V values decreased initially until the third hour, after which they increased, with value ranges of 0.15 to 0.3 and 0.16 to 0.23, respectively. The L*a*b* color space is a global color model in which colors are assigned numeric values across different channels. In the UV-light image, L* fluctuated during storage. The a* value decreased during storage under UV light. The b* value decreased in the UV-light image. Possibly, the changes in the L*, a*, and b* values on the fish surface in the UV images were caused by chemical and biochemical reactions owing to exposure to air during long-term storage, causing the darker color and lower reflectance of the fish surface [26] [27].

Furthermore, In the UV-light images, the color parameters R, G, B, H, S, V, and L decreased during the first storage period then increased as storage time increased. Values a* (R2 = 0.93) and b* (R2 = 0.95) decreased linearly with increased storage time and exhibited a good correlation with storage time. These results indicate that changes in fish-surface color with different lighting methods, in parallel with storage time, can be used as indices for the evaluation of the freshness of crucian carp.

3.4. Different Color Parameters of the Fish Surface for APC Prediction in Crucian Carp

The a* and b* values of the UV-light image, change regularly with the length of storage, indicating that fish-surface color could reliably indicate changes in fish freshness. We plotted the selected color parameters against the standard fish freshness indicator of the APC value, as shown in Figure 5. The APC value in

Figure 5. Different color parameters in the UV-light images for APC value prediction: regression model for predicting the APC value, based on the a* value (left) and b* value (right); a good exponential relationship is obtained between the APC, the a* (R2 = 0.97), and the b* values (R2 = 0.85), respectively.

muscle decreased exponentially with the a* value (R2 = 0.97) and b* value (R2 = 0.85) in the UV-light image. The APC value and a* value from the UV-light image have a strong exponential relationship, indicating that the a* value from the UV-light image can be used to accurately predict fish freshness. This suggests great potential as a fast and simple method for assessing fish freshness.

4. Conclusion

In conclusion, we developed a machine vision system for predicting the APC value in crucian carp using changes in fish-surface color during storage at room temperature. Images under UV light were preprocessed, after which color-parameter conversion was performed automatically by an image-analysis algorithm. In the UV-light images, color parameters R, G, B, H, S, V, and L decreased in the first period, then increased as storage time increased; a* and b* values decreased with increased storage time. Regression models of the a* and b*value demonstrated excellent prediction results with a high correlation coefficient (R2 = 0.97 and R2 = 0.85, respectively) for determining the APC value. Compared to the traditional chemistry method of detecting the APC which is regarded as complicated operation, time-consuming, high cost for assessing fish freshness [28], a machine vision system, equipped with UV light and using surface-color parameters, demonstrates great potential for developing an accurate, real-time, online method for predicting whole fish freshness during storage.


Financial support was provided by the Science and Technology program of Chengdu (NO. 2018YF0501197SN) and the Agricultural Science and Technology Innovation Program (ASTIP-2020-001). We are grateful to Nie Shen for help to gather experimental data. The manuscript benefited greatly from the constructive comments of Dr. Shiigi Tomoo (Department of Ocean Mechanical Engineering, National Fisheries University, Japan).

Cite this paper: Liao, Q. , Wei, C. , Li, Y. , Guo, L. and Ouyang, H. (2021) Developing a Machine Vision System Equipped with UV Light to Predict Fish Freshness Based on Fish-Surface Color. Food and Nutrition Sciences, 12, 239-248. doi: 10.4236/fns.2021.123019.

[1]   Dowlati, M., Guardia, M.D.L., Dowlati, M. and Mohtasebi, S.S. (2012) Application of Machine-Vision Techniques to Fish-Quality Assessment. TrAC-Trends in Analytical Chemistry, 40, 168-179.

[2]   Dowlati, M., Mohtasebi, S.S., Omid, M., Razavi, S.H., Jamzad, M. and Guardia, M.D.L. (2013) Freshness Assessment of Gilthead Sea Bream (Sparus aurata) by Machine Vision Based on Gill and Eye Color Changes. Journal of Food Engineering, 119, 277-287.

[3]   Ghaly, A.E., Dave, D., Budge, S. and Brooks, M.S. (2010) Fish Spoilage Mechanisms and Preservation Techniques: Review. American Journal of Applied Sciences, 7, 859-877.

[4]   Bernardo, Y.A.A., Rosario, D.K.A., Delgado, I.F. and Conte-Junior, C.A. (2020) Fish Quality Index Method: Principles, Weaknesses, Validation, and Alternatives—A Review. Comprehensive Reviews in Food Science and Food Safety, 19, 2657-2676.

[5]   Huang, X., Xu, H., Wu, L., Huang, D., Yao, L. and Han, F. (2016) A Data Fusion Detection Method for Fish Freshness Based on Machine Vision and Near-Infrared Spectroscopy. Analytical Methods, 8, 2929-2935.

[6]   Dufour, é., Frencia, J.P. and Kane, E. (2003) Development of a Rapid Method Based on Front-Face Fluorescence Spectroscopy for the Monitoring of Fish Freshness. Food Research International, 36, 415-423.

[7]   Cheng, J.H., Da, W.S., Xin, A.Z. and Hong, B.P. (2014) Non-Destructive and Rapid Determination of TVB-N Content for Freshness Evaluation of Grass Carp (Ctenopharyngodon idella) by Hyperspectral Imaging. Innovative Food Science & Emerging Technologies, 21, 179-187.

[8]   Dutta, M.K., Issac, A., Minhas, N. and Sarkar, B. (2016) Image Processing Based Method to Assess Fish Quality and Freshness. Journal of Food Engineering, 177, 50-58.

[9]   Issac, A., Dutta, M.K. and Sarkar, B. (2017) Computer Vision Based Method for Quality and Freshness Check for Fish from Segmented Gills. Computers and Electronics in Agriculture, 139, 10-21.

[10]   Xu, J.L. and Sun, D.W. (2018) Machine Vision Detection of Salmon Muscle Gaping Using Convolutional Neural Network Features. Food Analytical Methods, 11, 34-47.

[11]   Taheri-Garavand, A., Fatahi, S., Omid, M. and Makino, Y. (2019) Meat Quality Evaluation Based on Computer Vision Technique: A Review. Meat Science, 156, 183-195.

[12]   Quevedo, R. and Aguilera, J.M. (2010) Machine Vision and Stereoscopy for Estimating Firmness in the Salmon (Salmon salar) Fillets. Food and Bioprocess Technology, 3, 561-567.

[13]   Wu, D. and Sun, D.W. (2012) Colour Measurements by Computer Vision for Food Quality Control—A Review. Trends in Food Science and Technology, 29, 5-20.

[14]   Rocculi, P., Cevoli, C., Tappi, S., Genovese, J., Urbinati, E., Picone, G., Fabbri, A., Capozzi, F. and Rosa, M.D. (2019) Freshness Assessment of European Hake (Merluccius merluccius) through the Evaluation of Eye Chromatic and Morphological Characteristics. Food Research International, 115, 234-240.

[15]   Korel, F., Luzuriaga, D.A. and Balaban, M.Ö. (2010) Objective Quality Assessment of Raw Tilapia (Oreochromis niloticus) Fillets Using Electronic Nose and Machine Vision. Journal of Food Science, 66, 1018-1024.

[16]   Katherine, L., Mery, D., Pedreschi, F. and Jorge, L. (2006) Color Measurement in L*a*b* Units from RGB Digital Images. Food Research International, 39, 1084-1091.

[17]   Liao, Q., Suzuki, T., Kohno, Y., Dimas, A.R., Kuramoto, M. and Kondo, N. (2017) Monitoring Red Sea Bream Scale Fluorescence as a Freshness Indicator. Fishes, 2, 10.

[18]   Lerke, P. and Farber, L. (1969) Direct Bacterial Count as a Rapid Freshness Test for Fish Fillets. Journal of Applied Microbiology, 17, 197-201.

[19]   Xu, J.L. and Sun, D.W. (2017) Identification of Freezer Burn on Frozen Salmon Surface Using Hyperspectral Imaging and Computer Vision Combined with Machine Learning Algorithm. International Journal of Refrigeration, 74, 151-164.

[20]   Ce, S., Qian, J., Han, S., Fan, B., Yang, X. and Wu, X. (2018) Developing a Machine Vision System for Simultaneous Prediction of Freshness Indicators based on Tilapia (Oreochromis niloticus) Pupil and Gill Color during Storage at 4ÖC. Food Chemistry, 243, 134-140.

[21]   Liao, Q., Suzuki, T., Shirataki, Y., Kuramoto, M. and Kondo, N. (2018) Freshness Related Fluorescent Compound Changes in Japanese Dace Fish (Tribolodon hakonensis) Eye Fluid during Storage. Engineering in Agriculture, Environment and Food, 11, 95-100.

[22]   Masood, Z., Yasmeen, R., Haider, M.S. and Tarar, O.M. (2015) Evaluations of Crude Protein and Amino Acid Contents from the Scales of Four Mullet Species (Mugilidae) Collected from Karachi Fish Harbor, Pakistan. Indian Journal of Geo-Marine Sciences, 44, 724-731.

[23]   ángeles, E.M. (2012) An Overview of the Immunological Defenses in Fish Skin. International Scholarly Research Notices, 2012, Article ID: 853470.

[24]   Chmiel, M., Słowiński, M., Dasiewicz, K. and Florowski, T. (2015) Use of Computer Vision System (CVS) for Detection of PSE Pork Meat Obtained from m. semimembranosus. LWT—Food Science and Technology, 65, 532-536.

[25]   Dehrouyeh, M.H., Omid, M., Ahmadi, H., Mohtasebi, S.S. and Jamzad, M. (2010) Grading and Quality Inspection of Defected Eggs Using Machine Vision. International Journal of Advanced Science and Technology, 16, 43-50.

[26]   Nuray, E. and Özkan, Ö. (2008) Quality Assessment of Whole and Gutted Sardines (Sardina pilchardus) Stored in Ice. International Journal of Food Science & Technology, 43, 1549-1559.

[27]   Hong, H., Luo, Y., Zhou, Z. and Shen, H. (2012) Effects of Low Concentration of Salt and Sucrose on the Quality of Bighead Carp (Aristichthys nobilis) Fillets Stored at 4°C. Food Chemistry, 133, 102-107.

[28]   Prabhakar, P.K., Vatsa, S., Srivastav, P.P. and Pathak, S.S. (2020) A Comprehensive Review on Freshness of Fish and Assessment: Analytical Methods and Recent Innovations. Food Research International, 133, Article No.109157.