Accuracy Analysis on Bundle Adjusrmenr of Remore Sensing Images Based on Dual Quarernion

Sheng Qinghong（盛慶紅），F(xiàn)ei Lijia（費(fèi)利佳），Liu Jianfeng（柳建鋒），Chen Shuwen（陳姝文），Wang Huinan（王惠南）

College of Astronautics，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China

（Received 29 October 2013；revised 23 March 2014；accepted 11 April 2014）

Accuracy Analysis on Bundle Adjusrmenr of Remore Sensing Images Based on Dual Quarernion

Sheng Qinghong（盛慶紅）*，F(xiàn)ei Lijia（費(fèi)利佳），Liu Jianfeng（柳建鋒），Chen Shuwen（陳姝文），Wang Huinan（王惠南）

College of Astronautics，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China

（Received 29 October 2013；revised 23 March 2014；accepted 11 April 2014）

Absrracr：A bundle adjustment method of remote sensing images based on dual quaternion is presented，which conducted the uniform disposal corresponding location and attitude of sequence images by the dual quaternion.The constraint relationship of image itself and sequence images is constructed to compensate the systematic errors.The feasibility of this method used in bundle adjustment is theoretically tested by the analysis of the structural characteristics of error equation and normal equation based on dual quaternion.Different distributions of control points and stepwise regression analysis are introduced into the experiment for RC30 image.The results show that the adjustment accuracy can achieve 0.2 m in plane and 1 m in elevation.As a result，this method provides a new technique for geometric location problem of remote sensing images.

photogrammetry；bundle adjustment；geometric correction；dual quaternion；geometric imaging model

0 Inrroducrion

Bundle adjustment establishes the adjustment mathematical model based on the observation values of the original image，and it can best consider and correct the effects of system error. With the development of photogrammetric technology and computer science，this most rigorous adjustment method has been widely applied［1］. Nowadays，low-altitude platform including unmanned aircraft and airship has been gradually applied in the aerial photogrammetry［2，3］.Due to the instability of this kind of platform，the rotation angle of the obtained image is generally more than three degree，which makes it difficult to carry out the traditional aerial triangulation.Especially in the bundle adjustment，which is based on the principle of least squares，the non-convergence caused by the dependence of initial values ultimately leads to the incorrect results.

The existing adjustment calculation methods include bundle adjustment based on Euler angles，bundle adjustment based on matrix factorization［4］，fast algorithm for bundle adjustment［5］and bundle adjustment based on unit quaternion［6］.The first three have a very strong dependence on the initial values，and the calculation process includes tedious trigonometric calculations，which leads to a large amount of calculation.Although bundle adjustment based on unit quaternion uses quaternion［7，8］to represent the angular elements，which reduces the dependence on initial values and the amount of adjustment calculation，it only solves the rotation transformation between the coordinates without solving the translation transformation at the same time. As a result，dual quaternion is introduced into the adjustment calculations.

Dual quaternion was put forward by Clifford in the study of spinor algebra to introduce the dual unit，and then it was improved by Study［9］. Compared with other available methods descri-bing a spiral motion，dual quaternion is proved to be the most simple and most effective one［9］. Charles theorem showed that，any general rigid body motion could be equivalent to a spiral motion，and dual quaternion took both rotation and translation into consideration，which was the most simple geometric algebra tool to describe the spiral movement［10］.In recent years，dual quaternion has been widely applied in the areas of robot kinematics［11］，spacecraft rendezvous and docking［12］，moving target attitude determination［13］and muti-rigid-body system［14］.Sheng Qinghong et al.verified the correctness and universality of dual quaternion imaging geometric model in positioning［15］.

1 Dual Quarernion

1.1 Quarernion and dual quarernion

Quaternions are four terms of real numbers（qrqxqyqz），which include a three-term vector with components qx，qyand qz.

Quaternions are usually represented in the form

where qrand q are the real and vector parts，respectively，and i，j and k the unit vectors associated with the axes of a Cartesian coordinate system.

Dual quaternions are mathematical entities whose four components are dual numbers.Therefore，assuming thatεis a dual unit which satisfies ε2=ε3=0，then dual quaternions can be expressed as follows

1.2 Coordinare rransformarion by dual quarernion

Dual quaternion is a powerful tool in coordinate transformation.As seen in Fig.1，coordinate system O revolves an angle ofθaround special vector l，and makes a translation of d along l at the same time，then it becomes a new coordinate system N.

Fig.1 Coordinate transformation based on dual quaternion

From Fig.1，if dual vector l and dual angleθ are given，the traditional complex coordinate system transformation becomes quite simple using dual quaternion curve.Define the following equation

2 Bundle Adjusrmenr Based on Dual Quarernion

Bundle adjustment can use photogrammetric analytical method to determine the detector＇s orientation of all the images in the region at the shooting time.Its principle is directly based on the beam of each image.Taking the coordinates of the image point as observed values，through the translation and rotation of each beam in three dimensions，the conjugated lines best intersect in the object space.And make it into a specified coordinate system so as to calculate the position and orientation of each image at the shooting time. Therefore，bundle adjustment is theoretically the most rigorous method.Using the rigorous imaging geometric model based on dual quaternion to proceed bundle adjustment，its principle is to describe the position and orientation of the time series images uniformly by dual quaternion，thus bundle adjustment compensates for the systematic errors of the imaging geometric model.

2.1 Bundle adjusrmenr model

Bundle adjustment based on dual quaternionis shown in Fig.2.

Fig.2 Bundle adjustment based on dual quaternion

As can be seen in Fig.2，O is the intersection of the two lights O-S1，O-S2.The elements of exterior orientation of any image can be used to describe coordinate transformation.Coordinate system Si-xiyizirevolves an angle ofθaround special vector l and then becomes a new coordinate system Si-wiuivi，and coordinate system Si-wiuivimakes a translation of d along l at the same time，then it becomes a new coordinate system OXYZ.

As a consequence，using the product rule of quaternion，one can gain the relationship between dual quaternion and rotation/translation matrix. Using quaternion to describe the elements of exterior orientation，the exterior orientation parameters（EOPs）of image is（q0qxqyqzr0rxryrz）. The collinear equation of any images is given in Ref.［13］.

2.2 Marhemarical framework

Applying Taylor principle to expand the collinear equation，one can get the linearization error equation

When unit quaternion is used to describe the EOPs of the images in the bundle adjustment，there are two condition equations for every image

The two equations also should be linearized，and the linearization equation is given by

The above equations can be written as follow Putting Eqs.（4，7）together，one can get the total error equation of the bundle block adjustment

where t is the vector of EOPs，X the vector of the ground points coordinates，and L the observation residue error vector of the image points.A，B and C are the corresponding coefficient matrix.

In this section，we take the RC30 image data as an example to illustrate the structural characteristics of the error equation and normal equation.

As can be seen in Fig.3，the RC30 image data is made up of two air strips and each air strip is composed of 3 images.According to the principle of control point distribution in Fig.3，matrix of the error equation coefficient can be formed in Fig.4.Point 2155 is obtained in images 2 and 3，so this point can list two error equations.And non-zero elements occupied two black blocks，one is related to image 2 and the other is related to image 3.For coordinate corrections，the two black blocks join together into a vertical rectan-gle.Point 6156 is the intersection of six beams，and it can list six error equations in two air strips.There are six black blocks in Fig.4，and they are related to images 1，2，3 and 4，5，6，respectively.

Fig.3 Regional distribution of RC30 image

Fig.4 Matrix of error equation coefficients

Arranging the control points according to Fig.3，the matrix of the normal equation coefficients is shown in Fig.5.

2.行業(yè)指導(dǎo)價(jià)格（包括現(xiàn)場(chǎng)施工安裝指導(dǎo)費(fèi)用的出廠價(jià)）：18 200元/t（PE原材料參考價(jià)按12 500元/t計(jì)）。

Fig.5 Matrix of normal equation coefficients

It is a symmetric sparse matrix.To each image，ATA is a 8×8 matrix，and to an unknown point，BTB is a 3×3 matrix，and ATB is a 8×3 matrix.If there are N air strips，and there are n images in each air strip，ATA of the whole area is a 8n N×8n N matrix.If there are n＇unknown points in the whole area，the corresponding BTB is a 3n＇×3n＇matrix.With Eq.（5）added to the adjustment model，the matrix of the normal equation coefficients increases an edge part D，which is associated with t，and its order is 8×2.Fig.5 shows that the bigger the 8×8 black block on the main diagonal is，the smaller the 3×3 black block on the main diagonal is.The non-main diagonal reflects the content of the unknown points in each image，and each air strip involves three rows of unknown points.Therefore，there are three ladder-like black blocks in Fig.5.There are a row of common points between two air strips，and on this line of points there are two ladder-like black blocks respectively associated with two adjacent air strips.

3 Resulrs and Analysis

In least square adjustment，in addition to the unknowns solved，one can also obtain the weight reciprocals of certain derived values.Using the diagonal elements of the weight reciprocals，we can get the weight coefficient of point i as Qxixi，Qyiyiand Qzizi，and the corresponding standard error is as follows

whereσ0is the mean square error of unit weight in the parameter adjustment with constraints.

In our experiments，the mean accuracy of a block is obtained through computing the mean value of variance of all points.

Next，experiment with RC30 data is presented to indicate the reliability of the bundle adjustment model based on dual quaternion in improving the positioning accuracy of the detector.Thedetailed parameters of RC30 data can be seen in Table 1.

In addition，the overlap of these two images exceeds 60%.The control points are measured from 1：15 000 topographic maps.

Table 1 Derailed paramerers of RC30 dara

The initial value of the position and posture has been given as a known value.

It can be seen from Table 2 that the actual precision of the proposed bundle adjustment based on unit dual quaternion is similar to the theoreti-cal precision，which verifies the reliability of the algorithm.From the six experiments，we find that there is an obvious regularity in the residue error of the checkpoint.

Table 2 Accuracy of bundle adjusrmenr for differenr programs of conrrol poinr disrriburion

According to the principle of multivariate regression analysis，the regression analysis can easily determine the relationship between a set of variables and another random variable.In the case of the residue error of the checkpoint，the control points on the edge and inside constitute a set of variables，and the residue error is the random variable.

Therefore，if you can get some significant results between the control point distribution and the residue error of the checkpoint by regression analysis，the residue error of the checkpoint obeys a certain distribution.According to Table 2，take the general polynomial as the equation of the regression analysis

where di（i=x，y an d z plane）are four different kinds of residue errors，bj（j=0—4）the five regression coefficients of the regression analysis，and 1，n，m，n2，m2the independent variables（Corresponding to the regression coefficients）.n is the number of marginal control points and m the number of central control points.

Table 3 lists the results of the regression analysis，where Rmis the residual standard deviation.In the experiment，significance level a= 0.05.

Table 3 Resulrs of regression analysis for four random variables

It can be seen from Tables 2，3 that：

（1）The residual standard deviations of the regression analysis for four different kinds of residue errors are quite small，which proves the correctness of the regression analysis model.

（2）The residual standard deviation of the elevation is bigger than the other three，and the reason may be that this image is taken from a mountain area，which the terrain is quite complicated.Thus to obtain higher precision in elevation in complicated terrain，one needs more control points.

（3）Compared with the coefficients of variable n and m，n2and m2，we can find that：b1＞b2，b3＞b4.Therefore，a conclusion can be drawn：the amount of marginal control points affects more than the internal control points on the precision，that is to say，the weakest part of the precision for bundle adjustment is in the margin of the area.

（4）Compared with the control point distribution programs 1，2，we can find that，under the circumstances that the marginal control points exist，the lack of the central control point will reduce the precision in plane and in elevation.And because the experimental image is taken from a mountain area，where the terrain is quite complicated，the central control point affects more the precision in elevation than that in plane.

（5）The proposed method only needs to set four marginal control points and one central control point in the image，and a high precision in plane and in elevation can be achieved，so it has a great advantage over other traditional adjustment methods.

（6）The experimental results show that the accuracy of the adjustment based on unit dual quaternion is as much as that of the conventional adjustment based on Euler，and the requirement of the number and distribution of ground control points is almost the same as the Euler method.

4 Conclusions

Bundle adjustment of remote sensing image based on unit dual quaternion is a completely new exploration in the field of digital photogrammetry and it has unique advantages.Its accuracy can reach 0.2 m in plane and 1 m in elevation.Com-pared with the traditional bundle adjustment based on Euler angle，the method based on unit dual quaternion hardly depends on initial values，and calculates without tedious trigonometric operation.Therefore，it has a faster convergence of the iteration and it can be applied to bundle adjustment of close-range images and bundle adjustment of images generated from unstable aerial platform.However，due to the difficulty of obtaining the experimental data and other reasons，the related experiments are not enough.As a result，the technical details of bundle adjustment based on unit dual quaternion need further validation.

Under the circumstances that the central control point exists，the control points should be laid on the four corners of the region as much as possible.Therefore，it can play a greater role in the control of the accuracy.If the layout of the control points is reasonable，the amount of control points can be reduced under the premise that the accuracy in plane and in elevation basically unchanged，which greatly reduces the work amount of obtaining the control points.In the traditional aerial triangulation，due to the complexity of the terrain and the quality of the aerial photographs，the quality and quantity of the control points in each image cannot be guaranteed. As a result，the bundle adjustment based on unit dual quaternion has a good prospect in the application of aerial triangulation.

Acknowledgemenrs

This work was supported by the National Natural Science Foundations of China（Nos.41101441，60974107，41471381）and the Foundation of Graduate Innovation Center in NUAA（No.kfjj130133）.

［1］ Zhang Jianqing，Pan Li，Wang Shugen.Photogrammetry［M］.2nd edition.Wuhan：Wuhan University Press，2009：87-90.

［2］ Yao Keming，Liu Yanbin，Lu Yuping，et al.Fast scene matching for exploration unmanned aerial vehicle on mars［J］.Journal of Nanjing University of Aeronautics&Astronautics，2011，43（6）：810-815.（in Chinese）

［3］ Zhang Chao，Wang Daobo，F(xiàn)arooq M.Real-time tracking for fast moving object on complex background［J］.Transactions of Nanjing University of Aeronautics and Astronautics，2010，27（4）：311-315.

［4］ Du Zhiquan，Zheng Shunyi.A fast algorithm for bundle adjustment［J］.Geospatial Information，2007，5（1）：78-80.（in Chinese）

［5］ Hu Jiancai，Liu Xianyong，Qiu Zhiqiang，et al.Fast algorithm for bundle adjustment based on close-range photogrammetry［J］.Tool Engineering，2010，44（9）：85-90.（in Chinese）

［6］ Ji Shunping，Wu Zhenli.The application and practice of dual quaternion method in aerial triangulation［J］. Science of Surveying and Mapping，2010，35（1）：311-316.（in Chinese）

［7］ Duan Zhiyong，Yuan Xin.GPS-based attitude determination based on quaternion algorithm［J］.Journal of Nanjing University of Aeronautics&Astronautics，1999，31（5）：492-496.（in Chinese）

［8］ Lin Xueyuan，Liu Jianye，Liu Hong.An improved rotation vector attitude algorithm for laser strapdown inertial navigation system［J］.Transactions of Nanjing University of Aeronautics and Astronautics，2003，20（1）：47-52.

［9］ Brodsky V，Shoham M.Dual numbers representation of rigid body dynamics［J］.Mechanism and Machine Theory，1999，34（5）：693-718.

［10］Martinez J M R，Duffy J.The principle of transference：History，statement and proof［J］.Mechanisms and Machine Theory，1993，26（1）：165-177.

［11］Wang X K，Yu C B，Lin Z Y.A dual quaternion solution to attitude and position control for Rigid-Body coordination［J］.IEEE Transaction on Robotics，2012，28（5）：1162-1170.

［12］Ding Shangwen，Wang Huinan，Liu Haiying，et al. Algorithm of vision measure for relative position and pose of RVD spacecraft based on dual-quaternion［J］.Journal of Astronautics，2009，30（6）：2145-2150.（in Chinese）

［13］Li Jing，Wang Huinan，Liu Haiying.Dual quaternion curve interpolation algorithm for formation satellites［J］.Transactions of Nanjing University of Aeronautics and Astronautics，2012，29（1）：90-95.

［14］Schlanbusch R，Kristiansen R，Nicklassom P J.On choosing quaternion equilibrium point in attitude stabilization［C］∥International Conference on Aerospace.Montana，Alberta：University of Montana，2010：410-417.

［15］Ji Ting，Sheng Qinghong，Wang Huinan，et al.Dual quaternion of space resection with single-image［J］. Journal of Image and Graphics，2012，17（4）：494-503.（in Chinese）

（Executive Editor：Xu Chengting）

P23Documenr code：AArricle ID：1005-1120（2015）05-0523-07

*Corresponding aurhor：Sheng Qinghong，Associate Professor，E-mail：qhsheng@nuaa.edu.cn.

How ro cire rhis arricle：Sheng Qinghong，F(xiàn)ei Lijia，Liu Jianfeng，et al.Accuracy analysis on bundle adjustment of remote sensing images based on dual quaternion［J］.Trans.Nanjing U.Aero.Astro.，2015，32（5）：523-529.

http：//dx.doi.org/10.16356/j.1005-1120.2015.05.523