Posture Adjustment of MicroPhone Based on Image Recognition in Automatic Welding System

Wang Jin'e（王金娥）'Gao Ping（高萍）'2'Huang Haibo（黃海波）'2*' Li Xiangpeng（李相鵬）'2'Zheng Liang（鄭亮）'2' Xu Wenkui（徐文奎）'2'Chen Liguo（陳立國）'2

1.College of Mechanical and Electrical Engineering'Soochow University'Suzhou 215000'P.R.China；

2.Robotics and Microsystems Center'College of Mechanical and Electrical Engineering&Collaborative Innovation Center of Suzhou Nano Science and Technology'Soochow University'Suzhou 215000'P.R.China

Posture Adjustment of MicroPhone Based on Image Recognition in Automatic Welding System

Wang Jin'e（王金娥）1'Gao Ping（高萍）1'2'Huang Haibo（黃海波）1'2*' Li Xiangpeng（李相鵬）1'2'Zheng Liang（鄭亮）1'2' Xu Wenkui（徐文奎）1'2'Chen Liguo（陳立國）1'2

1.College of Mechanical and Electrical Engineering'Soochow University'Suzhou 215000'P.R.China；

2.Robotics and Microsystems Center'College of Mechanical and Electrical Engineering&Collaborative Innovation Center of Suzhou Nano Science and Technology'Soochow University'Suzhou 215000'P.R.China

As the requirements of production process is getting higher and higher with the reduction of volume'microphone production automation become an urgent need to improve the production efficiency.The most important part is studied and a precise algorithm of calculating the deviation angle of four types microphones is proposed' based on the feature extraction and visual detection.Pretreatment is performed to achieve the real-time microphone image.Canny edge detection and typical feature extraction are used to distinguish the four types of microphones' categorizing them as type M1 and type M2.And Hough transformation is used to extract the image features of microphone.Therefore'the deviation angle between the posture of microphone and the ideal posture in 2D plane can be achieved.Depending on the angle'the system drives the motor to adjust posture of the microphone.The final purpose is to realize the high efficiency welding of four different types of microphones.

visual inspection；Canny edge detection；Hough transform；feature extraction；microphone

0 Introduction

Microphone as a kind of acoustic sensor converts sound energy to electrical energy［1］is widely used in such digital products as mobile phones' bluetooth headset'PC'MP3'MP4 and digital camera'etc.Since invention of the microphone at the end of the 19th century'microphone technology has developed drastically from traditional crystal and electromagnetic type into microelectromechanical systems（MEMS）type.With the reduction of volume'requirements of production process become higher and higher.Domestic microphone production still focuses on manual welding'which decides that the productivity will remain a low rate.Therefore'the microphone production automation becomes an urgent need to improve the production efficiency.The combination of motor'air pressure adsorption and visual detection［2-3］technology can realize the automation of microphone welding'and visual inspection is the core technology in the microphone welding automation.To satisfy the demand of production development'we develop a new set of microphone automatic welding system.And the most important part:posture adjustment of the microphone based on visual identification is presented here.

Visual inspection'through image preprocess［4］can get rid of noise'and Hough transform［5］is used to achieve the characteristics of the microphone'which is greatly useful for a precise calculation of the deviation angle.Depending on the angle'the system drives the motor to adjustthe microphone posture'which lays a solid foundation for great improvement of the microphone automatic welding［6］success rate.An algorithm precisely calculating the angle of four kinds microphone is proposed.

1 Automatic MicroPhone Welding System

1.1 Automatic microPhone welding Process

Automatic welding equipment for microphone is a multi-station rotary equipment'as shown in Eig.1'and the seven stations correspond to seven steps of the welding process. View of seven parts is shown in Eig.2.

Eig.1 Multi-station rotary disc welding equipment

The microphone automatic feeding is the first station of the rotary equipment'which is performed by the vibration plate and conveying mechanism'and the microphone is turned into the wheel of stoma.When the wheel is rotating' a microphone is fed into the rotary table'the previous microphone enters the visual inspection area'and the institution tests the microphone deflection angle and drives motor to adjust the microphone to the ideal position.Next step'the microphone enters the performing area of wire feeding institution'the corresponding black line and red line are fed'and the next station'the wire welding is completed.The microphone enters the next visual inspection station after welding to test the welding results.Einally'the system discharges microphone according to the test results'the good welding and bad welding microphones are discharged'respectively.The flow chart is shown in Eig.3.

Eig.2 Parts of automatic welding equipment

1.2 Visual detection system

Machine vision is a system'which is combined with modern advanced control technology' computer technology and sensor technology'and is characterized by the opto-mechatronics integration'vision framework as shown in Eig.4.

Machine vision inspection is a complete system'including light sources'lenses'cameras' grabbers and image processing card.

Eig.3 Elow chart of automatic welding process

Eig.4 Eramework of visual system

The light source is one of the most important factors that affect the machine vision system input.It directly affects the quality of input data and application effect.The lens is an important component of machine vision system'and plays a key role in imaging quality.It affects the image quality of various major indicators'including resolution'contrast'depth of field and all kinds of aberration.Industrial camera is the core part of the machine vision system.Its essential function is to transform optical signals into electrical signals'and has higher transmission power'antijamming and stable imaging capability compared with the ordinary camera.Image acquisition card is just a part of the complete machine vision system.However'it plays a very important role.It directly determines the camera interface:black and white'color'analog'digital'and so on.Visual processor integrates acquisition card with the processor'and speeds up the visual processing tasks.

In the microphone automatic welding equipment'the machine vision inspection system adopts DH-HV3151UC-M camera'TL-20 video microscope'carbon capture and storage（CCS）ring light source'and DH-VT111 image acquisition card of DH-VT series.

2 MicroPhone Posture Adjustment Based on Visual InsPection

2.1 Introduction of microPhone Posture adjustment in automatic welding equiPment

In the multi-station rotary microphone automatic welding equipment'adjusting the microphone posture and orientation through visual inspection is the key to the success of the welding process'which has a direct impact on the results of post-weld.Real-time microphone image acquired by the camera provides the signal source of the microphone posture adjustment.The left side of microphone is negative'and the right side is positive'corresponding to the black and red lines in the later procedure.

Posture of microphone supplied by the vibration plate and feeding actuator is random'and it is unable to guarantee the accuracy of the welding products.The visual inspection is used to detect the posture and orientation of the microphone' and the system drives the motor according to the feedback information about posture and orientation to realize posture adjustment of the microphone.A method is proposed based on visual detection algorithm to obtain accurate information of posture and orientation of the microphone' which is applicable to a variety of microphone products'and adjusts posture and orientation of the microphone based on the information.The algorithm is suitable for the following four types of microphones as shown in Eig.5.

Eig.5 Eour types of microphones

The vision algorithms is suitable for four kinds of microphones with different structural features'achieving the type identification according to the extracted features.Type recognition is realized according to extracted features'and the key steps to realize the microphone posture adjustment are as follows:

（1）Real-time image can be acquired from visual inspection system.Noise is removed through image pre-processing.Canny edge algorithm is used to extract region of interest（ROI）.

（2）Color features of different microphones are extracted based on the image feature.

（3）Hough line detection and Hough circle detection methods are used to extract the characteristics of different microphones'then their posture state can be got.

（4）According to microphone position state' the offset angleθbetween its posture and ideal posture can be acquired.

（5）According toθ'motor is driven to realize the posture and attitude adjustment of microphones.Eecause of the proportional relationship between sizes of welding plate and microphone' the accurate welding can be achieved whenθ＜5°.Ifθ＞5°'the system adjusts microphone posture and its step turn to step（1）.Ifθ＜5°'the posture and orientation adjustment of microphones is finished'and the system enters into next process.

Elow chart of the microphone posture adjustment is shown in Eig.6.

Eig.6 Elow chart of microphone posture adjustment

2.2 Posture Adjustment Algorithm Based on Visual InsPection

2.2.1 Image pretreatment to extract ROI

Preprocessing is the key step of the image feature extraction in the machine vision and image process technology'which filters out lots of interference and noise caused by environment and equipment.In the posture adjustment of microphone automatic welding equipment'the image preprocess is greatly useful to the ROI extraction'eliminating the environmental interferences for visual inspection.Therefore'the real-time collected image is dealt with gray process'Gaussian filter［7'8］'Canny edge detection［9'10］and Hough circle detection［11'12］.We can achieve the effective area of microphone ROI'the intermediate results are shown in Eig.7'where Eig.7（a）is the source image'Eig.7（b）the gray image comeing from source image'Eig.7（c）the result image processed by Gaussian filter'Eig.7（d）the edge detection result processed by Canny operator'and Eig.7（e）the Hough circle detection result'and the area in the blue circle is ROI.

2.2.2 Type recognition and posture detection

The visual inspection system of microphone welding automation equipment identifies microphone as many as four varieties.According to the input microphone category'the visual inspection system automatically extracts the features'and calculates the angle between the microphone posture and ideal posture according to the corre-sponding algorithm.Therefore'the system drives the motor to adjust posture of the microphone.

Eig.7 Intermediate results of image process

（1）The microphone type recognition

After the microphone is fed by vibration plate'posture of the microphone shows random state under visual inspection system'and they have different internal structures.In Eigs.5（a—c）'ROI of the microphone is red'it has been verified by lots of experiments that the angle between posture of the microphone and the ideal posture can be accurately achieved by using the algorithm based on Hough linear detection.ROI of microphone in Eig.5（d）is green'the angle between the microphone posture and ideal posture got by Hough linear detection is wrong'and the angle can be accurately achieved by using the algorithm based on Hough circle transform.Microphones can be divided into M1 and M2 types'according to the color characteristics of ROI.If the color of ROI closes to red'it belongs to M1 type' otherwise it belongs to M2 type.Posture adjustment algorithm of M1 type microphones is based on Hough line detection'and it of M2 type microphones is based on Hough circle detection.

Since image of the microphone is always in the vision center of the visual inspection system' avg R'avgE and avgG are average characteristic values of RGE three color components'which can be used to distinguish different types of microphones.The results of experimental data are shown in Eig.8.Avg R is significantly larger than avg G for M1 type microphones'and avg G is significantly larger than avgR for M2 type microphones.Therefore'avgR and avgG are two important characteristic values for posture adjustment algorithm.

Eig.8 Value distribution of avgR'avgG and avgE for M1 and M2

（2）Microphone posture adjustment

ROI color characteristic values are effective indicators to distinguish M1 and M2 type microphones.The posture adjustment algorithm for M1 is based on Hough line detection'and it for M2 is based on Hough circle detection.The steps realizing posture adjustment according to the characters of color are as follows:

Step 1 Source image turns into gray image through gray transform'then noise is removed by Gaussian filter'and object edge is detected by Canny operator.

Step 2 ROI area is extracted by Hough circle transform according to the characteristics of the microphone shape'and the color characteristic values avgR and avgG of ROI can be obtained.

Step 3 Avg R is compared with avgG'if avgR＞avgG'microphone belongs to M1 type' we go to step（4）；Whereas microphone belongs to M2 type'we go to step（5）.

Step 4 The improved Hough linear detection algorithm is adopted to detect the edge of the negative electrode chip in M1 type microphone' then'calculate theθbetween the edge and the vertical direction'i.e.the angle between posture of the microphone and ideal posture.

Step 5 Hough circle detection algorithm is adopted to detect two round holes in M2 type microphone'thenθbetween the connecting line and horizontal direction can be acquired'andθis the offset angle of the detected microphone.

Step 6 θis the signal driving the motor to adjust posture of microphone.Ifθ＞5°'motor is driven to adjust posture'and then posture of microphone is tested again untilθ＜5°.Ifθ＜5°' microphone should be brought into next process.

Elow chart of posture adjustment according to the color characters is shown in Eig.9.

Eig.9 Elow chart of posture adjustment according to color characters

Microphone automatic welding device automatically welds four different types of microphones'which thanks to visual inspection system that can accurately detect four types of microphones′posture state based on color features. Offset angleθcan be acquired through the flow as shown in Eig.9'and the test results of four types of microphones are shown in Eig.10.The detection results of M1 depends on Hough line transform'and the main area of M1 is red.The detection results of M2 depends on Hough circle transform'and the main area of M2 is green.Detection resultθranges from-180°to 180°.Ifθ＜0°'the microphone should be turned in clockwise direction'otherwise'the microphone should be turned in anticlockwise direction.

Endpoints of detection line are important in the proposed algorithm'and they are shown in Eig.11.

Eig.10 Microphone posture detection based on color feature

Eig.11 Endpoint detection results

The computation formula of the angle between posture and ideal posture of the microphone is as follows

where pr.x'pr.y'pl.x'pl.y are points′coordinates of test results'corresponding to two endpoints of Hough line detection for M1 type microphone or two circle centers of Hough circle detection result for M2 type microphone.

2.2.3 Experimental results and discussion

An algorithm for recognizing the microphone posture is proposed in automatic welding equipment.The features can be extracted through image preprocess'including Gaussian filter and Canny edge detection.Offset angleθof different types of microphones can be accurately calculated'then the motor is driven to adjust posture of microphones.Eor two welding areas of the chip are large in the microphone'the microphone can be accurately welded.Whenθ＜5°'the accuracy of the visual inspection system for detecting posture of the microphone is shown in Table 1. Welding products are shown in Eig.12.

Table 1 ExPerimental results

Eig.12 Welding products

The proposed algorithm is applied to the automatic welding equipment.The production efficiency is increased to 15 PCS/min'the size of the microphone is 4 mm in diameter.120 microphones are tested in the equipment'and the inspection accuracy reaches 96.7%.Hence'the proposed algorithm is effective.Eut the motor control precision error exists in the whole equipment system.Visual inspection accuracy cannot represent the accuracy of whole welding system' and the welding accuracy can be improved from visual inspection'motor control and other aspects.

3 Conclusions

Welding system adjusts posture of microphones based on the results of visual inspection' and visual inspection is the core technology of microphone welding automation.In the paper'posture detection algorithm of microphones is proposed based on image recognition'and the experimental result shows that it is effective.The algorithm can detect and identify as many as four types of microphones'and it is helpful to improve the accuracy and speed of microphone welding' and greatly save the cost of labor and time.

Acknowledgements

This work was supported by the Project of Youth Eund of the National Natural Science Eoundation（No. 61203208）'the National Natural Science Eoundation of China（No.61327802）'the Specialized Research Eund for the Doctoral Program of Higher Education（No.2013320111 0009）.

［1］ Kuntzman M L'Lee J G'Hewa-Kasakarage N N' et al.Micromachined piezoelectric microphones with in-plane directivity［J］.Applied Physics Letters' 2013'102（5）:054109-1-054109-4.

［2］ Xu Yuxing.Eased on visual inspection of the finishedproducts tobacco smoke detection system［J］.Journal of Electronics'2014（4）:106-106.（in Chinese）

［3］ Ravikumar S'Ramachandran K I'Sugumaran V'et al.Machine learning approach for automated visual inspection of machine components［J］.Expert Systems with Application'2011'38（4）:3260-3266.

［4］ Anitha S'Radha Dr V.Comparison of image preprocessing techniques for textile texture images［J］.International Journal of Engineering Science and Technology'2010'2（12）:7619-7625.

［5］ Xie Zhonghong'Ji Chanhying'Guo Xiaoqing'et al. An object detection method for qasi-circular fruits based on improved Hough transform［J］.Transactions of the CSAE'2010'26（7）:157-162.（in Chinese）

［6］ Li Xiaoyan'Wu Chuansong'Wu Li'et al.The development of Chinese welding manufacturing discipline study［J］.Journal of Mechanical Engineering' 2012'48（6）:19-31.（in Chinese）

［7］ Li Huifen'Eorward Jiang'Zhu Li'et al.Gaussian filter research and improvement of the robust performance［J］.Journal of Instruments and Meters' 2004'25（5）:633-633.（in Chinese）

［8］ Mario Sansone'Olga Zeni'Giovanni Esposito'et al. Automated segmentation of comet assay images using Gaussian filtering and fuzzy clustering［J］.Medical and Eiological Engineering and Computing'2012'50（5）:523-532.

［9］ Tao Luo'Zheng Xifeng'Ding Tiefu.Improved selfadaptive threshold Canny edge detection［J］.Opto Electronic Engineering'2009'36（11）:106-111.（in Chinese）

［10］Nitasha Singla'Shammi Sharma'Usha Ran.Ellipse detection using canny edge detection algorithm［J］. International Journal of Computer Applications and Information Technology'2013'1（2）:45-47.

［11］Yuan L'Cao Z R.Improved randomized Hough transform algorithm for circle detection［J］.Journal of Computer Applications'2010'30（1）:174-176.

［12］Jiang L.Efficient randomized Hough transform for circle detection using novel probability sampling and feature points［J］.Optik J'2012'123（20）:1834-1840.

（Executive editor:Xu Chengting）

TP391.41 Document code:A Article ID:1005-1120（2015）02-0232-08

*CorresPonding author:Huang Haibo'Associate Professor'E-mail:hbhuang@suda.edu.cn.

How to cite this article:Wang Jin′e'Gao Ping'Huang Haibo'et al.Posture adjustment of microphone based on image recognition in automatic welding system［J］.Trans.Nanjing U.Aero.Astro.'2015'32（2）:232-239.

http://dx.doi.org/10.16356/j.1005-1120.2015.02.232

（Received 10 January 2015；revised 20 January 2015；accepted 31 January 2015）