Defect Detection Algorithm of Patterned Fabrics Based on Convolutional Neural Network

XU Yang(徐 洋)， FEI Libin(費(fèi)利斌)， YU Zhiqi(余智祺)， SHENG Xiaowei(盛曉偉)

College of Mechanical Engineering， Donghua University， Shanghai 201620， China

Abstract: The background pattern of patterned fabrics is complex， which has a great interference in the extraction of defect features. Traditional machine vision algorithms rely on artificially designed features， which are greatly affected by background patterns and are difficult to effectively extract flaw features. Therefore， a convolutional neural network(CNN) with automatic feature extraction is proposed. On the basis of the two-stage detection model Faster R-CNN， Resnet-50 is used as the backbone network， and the problem of flaws with extreme aspect ratio is solved by improving the initialization algorithm of the prior frame aspect ratio， and the improved multi-scale model is designed to improve detection of small defects. The cascade R-CNN is introduced to improve the accuracy of defect detection， and the online hard example mining (OHEM) algorithm is used to strengthen the learning of hard samples to reduce the interference of complex backgrounds on the defect detection of patterned fabrics， and construct the focal loss as a loss function to reduce the impact of sample imbalance. In order to verify the effectiveness of the improved algorithm， a defect detection comparison experiment was set up. The experimental results show that the accuracy of the defect detection algorithm of patterned fabrics in this paper can reach 95.7%， and it can accurately locate the defect location and meet the actual needs of the factory.

Key words: patterned fabrics； defect detection； convolutional neural network(CNN)； multi-scale model； cascade network

Introduction

At present， most textile companies adopt manual methods in the process of fabric defect detection in China. In recent years， some textile companies have begun to develop automatic defect detection systems based on machine vision. However， most defect detection systems in the textile industry have the problems of low detection accuracy and poor universality. At the same time， the algorithm relies on the extraction of artificially designed features， which is limited by the traditional image processing technology， and is only suitable for defect detection on simple plain fabrics. For example， Xuetal.[1]used non-local mean filter to suppress noise， and optimized parameters by using the average of inter-class maximum variance to achieve the effect of defect segmentation. Wenetal.[2]designed an adaptive wavelet filter， which used the different characteristics of the flaw feature and the background response for defect detection. But in the actual production of the factory， most of the cloth are patterned fabrics with complex background patterns. However， due to the interference of complex background patterns， traditional image processing algorithms cannot effectively extract the defect features of the patterned fabrics， and it is difficult to achieve effective detection results.

Lietal.[3]used an improved Gaussian mixture model to extract the characteristics of printed fabrics， and then used a template matching method to detect fabric defects. Renetal.[4]proposed a method based on the combination of golden image subtraction and Fourier transform displacement theorem curve matching algorithm to realize the detection of wrong pattern defects in patterned fabrics. Both of two methods can detect defects on patterned fabrics with complex backgrounds， but have greater limitations on the detected images. The collection position and size of the image to be detected need to correspond to the template image， which is not suitable for the actual situation of the factory. The development of deep learning made convolutional neural network(CNN) algorithms widely used in various fields， and also brought new breakthroughs to the defect detection of patterned fabrics[5-7]. Jingetal.[8]decomposed the texture fabric image into partial patches， and performed network training on the marked patches， and finally used the training model to slide on the image to detect defects. This method can be used to detect defects in fabrics such as regular stripes and grids. Xie and Wu[9]proposed a detection method based on refinedet， and improved the head structure based on the full convolutional channel attention block and the bottom-up path augmentation transfer connection block， which could effectively detect the defects of the patterned fabrics.

In order to solve the problem of color cloth defect detection， a patterned fabrics defect detection method based on an improved CNN is proposed in this paper. On the basis of the two-stage model Faster R-CNN[10]， Resnet-50[11]is used as the backbone network， and the cascade R-CNN is introduced to improve the accuracy of defect detection. We improve the initialization algorithm of anchor’s aspect ratio to deal with the wide distribution of defect aspect ratios， design a multi-scale model to improve the detection of small defects， and use the online hard example mining(OHEM)[12]algorithms to strengthen the learning for the difficult samples and reduce the interference of the detection of defect in complex backgrounds. Finally， we do experiments on defect images and compare them with mainstream two-stage network models to verify the effectiveness of the model.

1 Defect Detection System of Patterned Fabrics

As shown in Fig. 1， the defect detection system of patterned fabrics is mainly composed of linear cameras， the strip light source， the computer， the encoder， and the detection platform. The camera is a Dalsa color line scan camera with an image resolution of 4 096 pixels × 1 pixel， and the sampling frequency is determined by the pulse signal sent by the encoder. The installation position of the encoder is in contact with the surface of the cloth， and the cloth is moved to drive the encoder to rotate， so that the encoder sends a pulse signal to control the camera capture card for image acquisition. The height of the camera， which can be adjusted according to the width of the cloth， is set to 1 meter in this inspection platform. The brightness of the strip light source can be adjusted by the output voltage according to the color of the cloth and the effect of the collected image. The output voltage in this system is set to 40V. The computer is configured with Intel(R) Xeon(R) CPU E5-2650 v2 and NVIDIA GeForce GTX 1080Ti GPU. The development language of the detection algorithm uses Python3.6.

Fig. 1 Defect detection system of patterned fabrics

2 Design of Algorithm for Defect Detection of Patterned Fabrics

The defect detection algorithm for patterned fabrics mainly includes five parts： design of multi-scale model， improved initialization algorithm of anchor aspect ratio， cascade network， introduction of OHEM algorithm， and setting of focal loss. The model of defect detection algorithm is shown in Fig. 2.

Fig. 2 Model of defect detection algorithm

2.1 Design of a multi-scale model

In traditional object detection models， follow-up operations are usually only performed on the last feature map of the CNN， but the feature map is obtained through multiple downsampling of the original image， and small defects on the feature map will lose a lot of effective information. Therefore， it is necessary to design a multi-scale model to improve the detection effect of small defects. Feature pyramid network (FPN)[13]uses three methods of bottom-up， top-down， and horizontal connection to combine the last four stages of Resnet-50 and the feature map obtained by down-sampling the last stage to perform feature fusion. It combines the semantic information in deep layer into the shallow layer which has more location information to improve the detection of small targets. Because there is a good correlation between the upper and lower layer feature maps， the upper layer has better classification-related semantic information， and the lower layer will retain more positioning-related information. Therefore, an improved multi-scale model is proposed.

As shown in Fig. 3， C2， C3， C4， and C5 are the feature maps of the last residual module in the last four stages of Resnet-50. First， reduce the number of channels through a 1×1 convolution， so that the five feature maps obtain the same number of channels 256. P6 is obtained by downsampling from P5 and then perform feature fusion operations on the feature maps of each layer. Taking the F4 layer as an example， the P5 layer upsamples by two times first， and then a 3×3 convolution with a step size of 1 is used. Use ordinary convolution on the P4 layer for convolution. Convolution with a step size of 2 is performed on the P3 layer. Finally, add up the three feature maps of the same size obtained above to obtain the F4 layer. Its deepest layer P6 and shallowest layer P2 only perform feature fusion with its adjacent layers. In addition， Resnet-50 will destroy the semantic information around the flaw to a certain extent while extracting the characteristics of the flaw. The global information contained in the original image can provide surrounding semantic information for the defect feature and has a guiding significance for the location of the defect. Therefore， the global context is used as a region of interest (RoI) for feature extraction to get a feature map. Then the feature map combined with F2， F3， F4， F5 and F6 to obtain a new feature map and perform subsequent predictions. Subsequent predictions are based on the size of the RoI obtained by the region proposal network(RPN) to select feature layers with appropriate scales for prediction respectively. The calculation formula of feature layer selection is as follows：

(1)

where，wandhare the width and height of Anchor， andFkcorresponds to feature map F2 to F6. RoI of small area selects shallow feature map， while RoI of large area selects deep feature map.

Fig. 3 Improved multi-scale model

2.2 Improved initialization algorithm of anchor’s aspect ratio

The two-stage detection model needs to provide some anchors first， and then screen and modify. It is easier to detect on the basis of anchor than to match the defect from scratch. In Faster R-CNN， in order to adapt to the size of different objects， the initial aspect ratio is {0.5， 1， 2}. However， in the actual situation of defect detection in colorful fabrics， some defects like seam heads and the seam head prints are long and narrow， so the original aspect ratio cannot be well adapted to such defects. Therefore， theK-means[14]clustering analysis method is used to cluster the aspect ratio of the flawed samples. Finally， the aspect ratio of anchor is set to {0.04， 0.10， 0.25， 0.50， 1， 2， 4， 10， 25} by analyzing the clustering results.

2.3 Cascade R-CNN

In the two-stage detection model， after the RoI is obtained， a threshold is selected to determine whether the RoI is a positive sample or a negative sample. The intersection over union(IoU) calculation formula between anchorAand labelM：

(2)

In the Faster R-CNN network， there will be a problem in the selection of the threshold. If the threshold is high， although the selected RoI is closer to the real object， there will be fewer RoIs that meet the threshold condition， which makes the positive and negative samples more imbalanced， leading to training overfitting. If the threshold is low， although the number of positive samples increases， the false detections will increase， resulting in increasing classification errors.

Two principles are followed in the selection of the threshold. When the IoU between input RoI and label is near the set threshold， the detection effect will be better than based on other thresholds； after the regression， the IoU of the detection boxes will be improved. Based on these two principles， the effect of the detector is optimized by setting cascade R-CNN. The cascade R-CNN connects three detectors. Because the accuracy of the detection boxes will be improved after the regression is passed， the positioning of the detection boxes will be more accurate after the regression result of the previous detector is input into the next detector. Among them， the IoU threshold of the three detectors should be gradually increased. The structure of the cascaded network is shown in Fig. 4. H represents part of the RCNN network， C represents the classification network， and B represents the regression network.

Fig. 4 Cascade network

2.4 Introduction of the OHEM algorithm

In the process of detecting the flaws of colorful fabrics， there is the problem of imbalance samples. Hard samples refer to RoI that is not clearly classified. Its border is in the transition area between the foreground and the background. In the current situation， it refers to the border of the RoI in the transition between the defect and the background pattern. These RoIs are more difficult to detect than the border of the RoIs boxed on the background which make the recognition ability of the model decrease. Therefore， the OHEM algorithm is introduced to strengthen the learning of hard samples， thereby improving the detection performance of the network. The part A calculates the loss value for all RoIs forward， and selects the fixed number of RoIs with a large loss as the hard samples， and then inputs the hard samples into the network B to calculate the loss forward and update the parameters.

Fig. 5 Algorithm principle of OHEM

2.5 Setting of the focal loss

During neural network training， the stability of parameter optimization depends on the choice of loss function. With regarding to the defect detection problem of patterned fabrics， because the actual number of defects on the patterned fabrics is often less， the number of positive samples is far more than the number of negative samples during the training process， which leads to a problem of imbalance between different categories and makes the model cannot learn the feature expression of defect samples well. Therefore， the focal loss[15]is used in order to alleviate the problem of imbalance between positive and negative samples.

F(pt)=-αt(1-pt)γlnpt，

(3)

(4)

(5)

where，αtis the weight coefficient of positive and negative samples， generally 0.25， which can be used to increase the weight of negative sample loss；γis a parameter used to adjust the weights of difficult and easy samples， generally 2，pis the prediction confidence of the sample， andyrepresents the label value of the sample. By adding two parameter constraints in the loss function， the imbalance of positive and negative samples， as well as difficult and easy samples， can be automatically adjusted during the network training process.

3 Defect Detection Experiment of Patterned Fabrics

In order to verify the detection effect of the improved algorithm， a detection experiment was carried out on the defect pictures of the patterned fabrics. Take the five common defects of patterned fabrics， which are stains， seam， holes， seam mark and fluff. The input image size is 512 pixels×512 pixels， among which there are a total of 1 740 images in the training set and a total of 253 images in the test set for detection.

Table 1 Samples distribution of the data

3.1 Comparison of defect detection effects of two models

The trained model to perform defect detection experiments is used on the test set of patterned fabrics. On the basis of the two-stage model Faster R-CNN， Resnet-50 is used as the backbone network and FPN is introduced. The fused model is used as a comparative detection model. The detection effects of the two models are displayed in Fig. 6. The top of the predicted bounding box in Fig. 6 is the type of defect， where the labels 1， 2， 3， 4， and 5 respectively represent stain， seam， hole， seam mark and fluff， the score represents the probability that the model predicts this type of defect. The higher the score level， the better the model’s extraction of defect features and the more accurate the defect classification. The results show that compared with the comparison model， the model in this paper is more accurate in the recognition and the predicted bounding box of defects， with higher confidence and better defect detection effect.

Fig. 6 Comparison of the defect detection results of two models

3.2 Detection accuracy and speed

In the defect detection experiment of the patterned fabrics，PmAis used as the evaluation standard. Assuming that defects are dichotomized and divided according to positive and negative samples， confusion matrix can be obtained， as shown in Table 2.

Table 2 Confusion matrix for detecting fabric defects

Accuracy ratePrepresents the proportion of samples predicted by the model to be defective that are actually defective， and its calculation formula is

(6)

The recall rateRrepresents the proportion of samples which defective samples can be predicted to be defective， and its calculation formula is

(7)

In order to combine the accuracy of defects and recall rate，PAis used to represent the average accuracy of the same category of defects in different recall rates， and its calculation formula is

(8)

In view of the unbalanced characteristics of defect categories，PmAis selected as the final evaluation standard of the model in combination with eachPAvalue of all types of defects， and its formula is

(9)

where，cis the number of categories of defects. According to the evaluation criteria ofPmA， as long as the final evaluation value of the model obtained from training is high enough， the detection effect of the model will be better.

Statistics of the detection results of the two models on the test set are shown in Table 3.

Table 3 Contrast results of different models

The results show that the average accuracy of the flaw detection of the model in this paper is 95.7%， which is better than the comparison model by 5.0%， and thePmAvalue is also improved 4.5%. At the same time， this model has a better effect on the recognition and the classification of defects， and the accuracy of positioning is also higher. In addition， while the model is improved， the average time for defect detection does not increase much. On average， each image only takes an extra 0.007 s. Taking the cloth with a width of 1.5 m as an example， the detection speed of the model is about 0.5 m/s， which can meet the actual needs of factory.

4 Conclusions

In this paper， we use a defect detection algorithm for patterned fabrics based on CNN. The algorithm has a good detection effect and can replace manual detection. The specific conclusions are as follows.

(1) By using the detection algorithm based on CNN， it breaks through the detection range of traditional machine vision algorithm， and can effectively detect the defects of patterned fabrics with complex background.

(2) Through the use ofK-means to improve the initialization algorithm of anchor’s aspect ratio and the use of improved multi-scale model that integrates the global context and upper and lower layer features， the detection algorithm is more suitable for the defect detection of patterned fabrics， and the detection effect is improved.

(3) The cascaded network is used to improve the location and accuracy of defects. At the same time， the OHEM algorithm is introduced and the focal loss is set to strengthen the learning of hard samples and improve the accuracy of the detection algorithm.