Full Text:   <3164>

Summary:  <1712>

CLC number: TP391.4

On-line Access: 2014-08-06

Received: 2013-12-29

Revision Accepted: 2014-03-03

Crosschecked: 2014-07-16

Cited: 2

Clicked: 5569

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE C 2014 Vol.15 No.8 P.593-606


Development of a monocular vision system for robotic drilling

Author(s):  Wei-dong Zhu, Biao Mei, Guo-rui Yan, Ying-lin Ke

Affiliation(s):  Department of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   wdzhu@zju.edu.cn

Key Words:  Vision system, Robotic drilling, Error measurement, Elliptical contour extraction, Hand-eye calibration

Share this article to: More |Next Article >>>

Wei-dong Zhu, Biao Mei, Guo-rui Yan, Ying-lin Ke. Development of a monocular vision system for robotic drilling[J]. Journal of Zhejiang University Science C, 2014, 15(8): 593-606.

@article{title="Development of a monocular vision system for robotic drilling",
author="Wei-dong Zhu, Biao Mei, Guo-rui Yan, Ying-lin Ke",
journal="Journal of Zhejiang University Science C",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Development of a monocular vision system for robotic drilling
%A Wei-dong Zhu
%A Biao Mei
%A Guo-rui Yan
%A Ying-lin Ke
%J Journal of Zhejiang University SCIENCE C
%V 15
%N 8
%P 593-606
%@ 1869-1951
%D 2014
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.C1300379

T1 - Development of a monocular vision system for robotic drilling
A1 - Wei-dong Zhu
A1 - Biao Mei
A1 - Guo-rui Yan
A1 - Ying-lin Ke
J0 - Journal of Zhejiang University Science C
VL - 15
IS - 8
SP - 593
EP - 606
%@ 1869-1951
Y1 - 2014
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.C1300379

robotic drilling for aerospace structures demands a high positioning accuracy of the robot, which is usually achieved through error measurement and compensation. In this paper, we report the development of a practical monocular vision system for measurement of the relative error between the drill tool center point (TCP) and the reference hole. First, the principle of relative error measurement with the vision system is explained, followed by a detailed discussion on the hardware components, software components, and system integration. The elliptical contour extraction algorithm is presented for accurate and robust reference hole detection. System calibration is of key importance to the measurement accuracy of a vision system. A new method is proposed for the simultaneous calibration of camera internal parameters and hand-eye relationship with a dedicated calibration board. Extensive measurement experiments have been performed on a robotic drilling system. Experimental results show that the measurement accuracy of the developed vision system is higher than 0.15 mm, which meets the requirement of robotic drilling for aircraft structures.



Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Armingol, J.M., Otamendi, J., de la Escalera, A., et al., 2003. Statistical pattern modeling in vision-based quality control systems. J. Intell. Robot. Syst., 37(3):321-336.

[2]Bilen, H., Hocaoglu, M., Unel, M., et al., 2012. Developing robust vision modules for microsystems applications. Mach. Vis. Appl., 23(1):25-42.

[3]Bone, G.M., Capson, D., 2003. Vision-guided fixtureless assembly of automotive components. Robot. Comput.-Integr. Manuf., 19(1-2):79-87.

[4]Bradski, G., Kaehler, A., 2008. Learning OpenCV: Computer Vision with the OpenCV Library. O’reilly, Sebastopol, CA.

[5]Chen, S., Zhang, Y., Qiu, T., et al., 2003. Robotic welding systems with vision-sensing and self-learning neuron control of arc welding dynamic process. J. Intell. Robot. Syst., 36(2):191-208.

[6]DeVlieg, R., Sitton, K., Feikert, E., et al., 2002. ONCE (ONe-sided Cell End effector) Robotic Drilling System. SAE Technical Paper 2002-01-2626.

[7]Dornaika, F., Horaud, R., 1998. Simultaneous robot-world and hand-eye calibration. IEEE Trans. Robot. Autom., 14(4):617-622.

[8]Eberli, D., Scaramuzza, D., Weiss, S., et al., 2010. Vision based position control for MAVs using one single circular landmark. J. Intell. Robot. Syst., 61(1-4):495-512.

[9]Fitzgibbon, A., Pilu, M., Fisher, R.B., 1999. Direct least square fitting of ellipses. IEEE Trans. Pattern Anal. Mach. Intell., 21(5):476-480.

[10]Forsyth, D.A., Ponce, J., 2011. Computer Vision: a Modern Approach (2nd Ed.). Prentice-Hall, Englewood Cliffs, New Jersey.

[11]Ibarguren, A., Martínez-Otzeta, J.M., Maurtua, I., 2014. Particle filtering for industrial 6DOF visual servoing. J. Intell. Robot. Syst., 74(3-4):689-696.

[12]Lee, B., Tarng, Y., 2001. Surface roughness inspection by computer vision in turning operations. Int. J. Mach. Tool. Manuf., 41(9):1251-1263.

[13]Malassiotis, S., Strintzis, M.G., 2003. Stereo vision system for precision dimensional inspection of 3D holes. Mach. Vis. Appl., 15(2):101-113.

[14]Malti, A., 2012. Hand–eye calibration with epipolar constraints: application to endoscopy. Robot. Auton. Syst., 61(2):161-169.

[15]Motta, J.M.C.S., de Carvalho, G.C., McMaster, R., 2001. Robot calibration using a 3D vision-based measurement system with a single camera. Robot. Comput.-Integr. Manuf., 17(6):487-497.

[16]Neto, H.V., Nehmzow, U., 2007. Real-time automated visual inspection using mobile robots. J. Intell. Robot. Syst., 49(3):293-307.

[17]Olsson, T., Haage, M., Kihlman, H., et al., 2010. Cost-efficient drilling using industrial robots with high-bandwidth force feedback. Robot. Comput.-Integr. Manuf., 26(1):24-38.

[18]Otsu, N., 1979. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern., 9(1):62-66.

[19]Pachidis, T.P., Lygouras, J.N., 2007. Vision-based path generation method for a robot-based arc welding system. J. Intell. Robot. Syst., 48(3):307-331.

[20]Simonvsky, M., 2011. Ellipse Detection Using 1D Hough Transform. Available from http://www.mathworks.com/matlabcentral/fileexchange/33970-ellipse-detection-using-1d-hough-transform.

[21]Strobl, K.H., Hirzinger, G., 2008. More accurate camera and hand-eye calibrations with unknown grid pattern dimensions. IEEE Int. Conf. on Robotics and Automation, p.1398-1405.

[22]Strobl, K.H., Sepp, W., Hirzinger, G., 2009. On the issue of camera calibration with narrow angular field of view. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p.309-315.

[23]Suzuki, S., 1985. Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process., 30(1):32-46.

[24]Thompson, P., Hartmann, J., Feikert, E., et al., 2005. Flex Track for Use in Production. SAE Technical Paper 2005-01-3318.

[25]Tsai, R., 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom., 3(4):323-344.

[26]Webb, P., Eastwood, S., Jayaweera, N., et al., 2005. Automated aerostructure assembly. Ind. Robot., 32(5):383-387.

[27]Williams, D.J., Shah, M., 1992. A fast algorithm for active contours and curvature estimation. CVGIP Image Understand., 55(1):14-26.

[28]Zhan, Q., Wang, X., 2012. Hand–eye calibration and positioning for a robot drilling system. Int. J. Adv. Manuf. Technol., 61(5-8):691-701.

[29]Zhang, Z., 2000. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell., 22(11):1330-1334.

[30]Zhao, Z., Weng, Y., 2013. A flexible method combining camera calibration and hand–eye calibration. Robotica, 31(5):747-756.

[31]Zhou, Y., Nelson, B.J., Vikramaditya, B., 2000. Integrating optical force sensing with visual servoing for microassembly. J. Intell. Robot. Syst., 28(3):259-276.

[32]Zhu, W., Qu, W., Cao, L., et al., 2013. An off-line programming system for robotic drilling in aerospace manufacturing. Int. J. Adv. Manuf. Technol., 68(9-12):2535-2545.

[33]Zou, X., Zou, H., Lu, J., 2012. Virtual manipulator-based binocular stereo vision positioning system and errors modelling. Mach. Vis. Appl., 23(1):43-63.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE