Remote Sensing Image Registration based on a Geometrical Model Matching
محورهای موضوعی : Signal ProcessingZahra Hossein-Nejad 1 , Hamed Agahi 2 , Azar Mahmoodzadeh 3
1 - Department of Electrical Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran
2 - Department of Electrical Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran
3 - Department of Electrical Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran
کلید واژه: SIFT, Matching Method, Remote Sensing Image Registration, Transformation Model,
چکیده مقاله :
Remote sensing image registration is the method of aligning two images from the same scene taken under different imaging circumstances containing different times, angles, or sensors. Scale-invariant feature transform (SIFT) is one of the most common matching methods previously used in the remote sensing image registration. The defects of SIFT are the large number of mismatches and high execution time due to the high dimensions of classical SIFT descriptor. These drawbacks reduce the efficiency of the SIFT algorithm. To enhance the performance of the remote sensing image registration, this paper proposes an approach consisting of three different steps. At first, the keypoints of both reference and second images are extracted using SIFT algorithm. Then, to increase the speed of the algorithm and accuracy of the matching, the SIFT descriptor with the vector length of 64 is used for keypoints description. Finally, a new method has been proposed for the image matching. The proposed matching method is based on calculating the distances of keypoints and their transformed points. Simulation results of applying the proposed method to some standard databases demonstrated the superiority of this approach compared with some other existing methods, according to the root mean square error (RMSE), precision and running time criteria.
Remote sensing image registration is the method of aligning two images from the same scene taken under different imaging circumstances containing different times, angles, or sensors. Scale-invariant feature transform (SIFT) is one of the most common matching methods previously used in the remote sensing image registration. The defects of SIFT are the large number of mismatches and high execution time due to the high dimensions of classical SIFT descriptor. These drawbacks reduce the efficiency of the SIFT algorithm. To enhance the performance of the remote sensing image registration, this paper proposes an approach consisting of three different steps. At first, the keypoints of both reference and second images are extracted using SIFT algorithm. Then, to increase the speed of the algorithm and accuracy of the matching, the SIFT descriptor with the vector length of 64 is used for keypoints description. Finally, a new method has been proposed for the image matching. The proposed matching method is based on calculating the distances of keypoints and their transformed points. Simulation results of applying the proposed method to some standard databases demonstrated the superiority of this approach compared with some other existing methods, according to the root mean square error (RMSE), precision and running time criteria.
[1] A. Wong and D. Clausi, "ARRSI: automatic registration of remote-sensing images," Geoscience and Remote Sensing, IEEE Transactions on, vol. 45, pp. 1483-1493, 2007.
[2] W.-J. Lee and S.-J. Oh, "Remote sensing image registration using equivariance features," in 2021 International Conference on Information Networking (ICOIN), 2021, pp. 776-781.
[3] Z. Hossein-Nejad and M. Nasri, "A New Method in Image Matching Based on Spatial Relationships in Multi-Sensor Remote Sensing Images " Iranian Remote Sensing & GIS, pp. 73-94, 2018.
[4] Y. Liu, H. Cao, Y. Zhao, Q. He, Y. Yang, L. Wang, et al., "A Remote sensing image registration algorithm based on multiple constraints and a variational Bayesian framework," Remote Sensing Letters, vol. 12, pp. 296-305, 2021.
[5] Z. Hossein-Nejad and M. Nasri, "A Review on Image Registration Methods, Concepts and applications," Journal of Machine Vision and Image Processing, pp. 39-67, 2017.
[6] W. Lee, D. Sim, and S.-J. Oh, "A CNN-Based High-Accuracy Registration for Remote Sensing Images," Remote Sensing, vol. 13, p. 1482, 2021.
[7] H.-M. Chen, M. K. Arora, and P. K. Varshney, "Mutual information-based image registration for remote sensing data," International Journal of Remote Sensing, vol. 24, pp. 3701-3706, 2003.
[8] X. Xie, Y. Zhang, X. Ling, and X. Wang, "A novel extended phase correlation algorithm based on Log-Gabor filtering for multimodal remote sensing image registration," International Journal of Remote Sensing, vol. 40, pp. 5429-5453, 2019.
[9] M. I. Patel, V. K. Thakar, and S. K. Shah, "Image registration of satellite images with varying illumination level using HOG descriptor based SURF," Procedia computer science, vol. 93, pp. 382-388, 2016.
[10] P. Schwind, S. Suri, P. Reinartz, and A. Siebert, "Applicability of the SIFT operator to geometric SAR image registration," International Journal of Remote Sensing, vol. 31, pp. 1959-1980, 2010.
[11] G. Sreeja and O. Saraniya, "A Comparative Study on Image Registration Techniques for SAR Images," in 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), 2019, pp. 947-953.
[12] Y. Ye, J. Shan, L. Bruzzone, and L. Shen, "Robust registration of multimodal remote sensing images based on structural similarity," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 2941-2958, 2017.
[13] A. Sedaghat, M. Mokhtarzade, and H. Ebadi, "Uniform robust scale-invariant feature matching for optical remote sensing images," IEEE Transactions on Geoscience and Remote Sensing, vol. 49, pp. 4516-4527, 2011.
[14] Z. Hossein-Nejad and M. Nasri, "A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration," Biomedical Signal Processing and Control, vol. 45, pp. 325-338, 2018.
[15] C. Harris and M. Stephens, "A combined corner and edge detector," in Alvey vision conference, 1988, p. 50.
[16] D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, pp. 91-110, 2004.
[17] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-up robust features (SURF)," Computer vision and image understanding, vol. 110, pp. 346-359, 2008.
[18] Z. Hossein-Nejad and M. Nasri, "RKEM: Redundant Keypoint Elimination Method in Image Registration," IET Image Processing, vol. 11, pp. 273-284, 2017.
[19] A. Sedaghat and N. Mohammadi, "High-resolution image registration based on improved SURF detector and localized GTM," International Journal of Remote Sensing, vol. 40, pp. 2576-2601, 2019.
[20] A. Sedaghat and H. Ebadi, "Remote sensing image matching based on adaptive binning SIFT descriptor," IEEE transactions on geoscience remote Sensing Technology and Application, vol. 53, pp. 5283-5293, 2015.
[21] M. Hasan, M. R. Pickering, and X. Jia, "Modified SIFT for multi-modal remote sensing image registration," in 2012 IEEE International Geoscience and Remote Sensing Symposium, 2012, pp. 2348-2351.
[22] B. Kupfer, N. S. Netanyahu, and I. Shimshoni, "An Efficient SIFT-Based Mode-Seeking Algorithm for Sub-Pixel Registration of Remotely Sensed Images," Geoscience and Remote Sensing Letters, IEEE, vol. 12, pp. 379-383, 2015.
[23] Z. Yi, C. Zhiguo, and X. Yang, "Multi-spectral remote image registration based on SIFT," Electronics Letters, vol. 44, pp. 107-108, 2008.
[24] K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, pp. 1615-1630, 2005.
[25] N. Y. Khan, B. McCane, and G. Wyvill, "SIFT and SURF performance evaluation against various image deformations on benchmark dataset," in 2011 International Conference on Digital Image Computing: Techniques and Applications, 2011, pp. 501-506.
[26] R. Bouchiha and K. Besbes, "Comparison of local descriptors for automatic remote sensing image registration," Signal, Image and Video Processing, vol. 9, pp. 463-469, 2015.
[27] M. Deshmukh and U. Bhosle, "A survey of image registration," International Journal of Image Processing (IJIP), vol. 5, p. 245, 2011.
[28] M. Hasan, X. Jia, A. Robles-Kelly, J. Zhou, and M. R. Pickering, "Multi-spectral remote sensing image registration via spatial relationship analysis on sift keypoints," in Geoscience and Remote Sensing Symposium (IGARSS), 2010 IEEE International, 2010, pp. 1011-1014.
[29] S. Saxena and R. K. Singh, "A survey of recent and classical image registration methods," International journal of signal processing, image processing and pattern recognition, vol. 7, pp. 167-176, 2014.
[30] S. Chen, S. Zheng, Z. Xu, C. Guo, and X. Ma, "AN IMPROVED IMAGE MATCHING METHOD BASED ON SURF ALGORITHM," International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, vol. 42, 2018.
[31] P. Etezadifar and H. Farsi, "A New Sample Consensus Based on Sparse Coding for Improved Matching of SIFT Features on Remote Sensing Images," IEEE Transactions on Geoscience and Remote Sensing, 2020.
[32] W. He and X. Deng, "A modified SUSAN corner detection algorithm based on adaptive gradient threshold for remote sensing image," in 2010 International Conference on Optoelectronics and Image Processing, 2010, pp. 40-43.
[33] Z. Hossein-Nejad and M. Nasri, "An adaptive image registration method based on SIFT features and RANSAC transform," Computers & Electrical Engineering, vol. 62, pp. 524-537, 2017.
Remote Sensing Image Registration based on a Geometrical Model Matching
Zahra Hossein-Nejad Department of Electrical Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran. hoseinnejad.zahra@yahoo.com Hamed Agahi* Department of Electrical Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran. Azar Mahmoodzadeh Department of Electrical Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran.
Received: 31/Aug/2021 Revised: 24/Nov/2021 Accepted: 07/Dec/2021 |
Abstract
Remote sensing image registration is the method of aligning two images from the same scene taken under different imaging circumstances containing different times, angles, or sensors. Scale-invariant feature transform (SIFT) is one of the most common matching methods previously used in the remote sensing image registration. The defects of SIFT are the large number of mismatches and high execution time due to the high dimensions of classical SIFT descriptor. These drawbacks reduce the efficiency of the SIFT algorithm. To enhance the performance of the remote sensing image registration, this paper proposes an approach consisting of three different steps. At first, the keypoints of both reference and second images are extracted using SIFT algorithm. Then, to increase the speed of the algorithm and accuracy of the matching, the SIFT descriptor with the vector length of 64 is used for keypoints description. Finally, a new method has been proposed for the image matching. The proposed matching method is based on calculating the distances of keypoints and their transformed points. Simulation results of applying the proposed method to some standard databases demonstrated the superiority of this approach compared with some other existing methods, according to the root mean square error (RMSE), precision and running time criteria.
Keywords: SIFT; Matching Method; Remote Sensing Image Registration; Transformation Model.
1- Introduction
Image registration is a keystone in programmed remote sensing image analysis such as change detection, image synthesis and image mosaic [1, 2]. Remote sensing image registration (RSIR) is the procedure for finding geometric transformation models between multiple remote sensing images, taken at different times or viewpoints or by different cameras or sensors [3, 4]. In general, registration methods of remote sensing images are categorized in intensity-based and feature-based approaches [3, 5, 6]. The former techniques use the intensity distribution in the masks of the same sizes. To this end, using similarity arrays, parameters of the geometric transformation model are tuned. Common similarity indices for RS images include mutual information (MI) [7] and phase correlation [8]. Accordingly, intensity-based techniques encounter various challenges [9, 10]. One main challenge is that for images with large intensity differences, overlapping regions in the images become limited [11]. Additionally, the run-time of these approaches is high, since they make use of the whole content in the image [12]. In [3, 13] several registration methods are compared, confirming that the feature-based techniques attain higher performance indices for remote sensing images registration. Feature-based approaches find salient features of the images; and afterwards they compute the matching and all parameters of the transformation [14]. Harris [15], scale-invariant feature transform (SIFT) [16], and speed-up robust feature (SURF) [17] are typically used in the feature-based RSIR. The SIFT algorithm is robust against scale and rotation changes and also intensity variations, affine distortion and noise [18]. Although these advantages made this algorithm significant in the registration process, the complex nature of remote sensing images has resulted in many mismatches [13, 19, 20].
Several studies have been conducted on the enhancement of the matching accuracy of the RS images in the feature extraction and descriptor computation steps. In [21], SIFT was adapted for multi-modal remote sensing image registration. This method considered a threshold value of 0.08 to remove keypoints with low contrast, and an 8×8 window to create a descriptor for keypoints. In [20] the AB-SIFT descriptor is proposed to improve the matching of RS images. In this method, the features are identified using the Hessian. Then for each feature, one direction is considered according to the SIFT method. Finally, the proposed AB-SIFT descriptor and the nearest neighbor-based matching method are used to match the images. In [13] the uniform robust SIFT (URSIFT) algorithm was introduced, in which extracted features were made uniform in terms of spatial and scale distribution. Via this method, the features were distributed uniformly in the image and then, the nearest neighbor-based matching is used to match the remote sensing images. In [22] an effective SIFT-based mode-seeking algorithm was proposed in order to register the remote sensing images. To eliminate the outliers, the authors used mode seeking of scale, rotational difference, and vertical and horizontal shifts between all SIFT keypoints. In [23], the scale restriction SIFT (SRSIFT) was introduced that eliminated a number of mismatches by gradient orientation modification and scale restriction criteria. The main challenge in the SRSIFT algorithm is high execution time due to the high dimensions of its descriptor. These problems led to interference in the process of remote sensing image registration. Although these improvements have increased the accuracy of the correspondence, on the one hand, there are still a number of incorrect correspondences in the SIFT algorithm. Indeed, in some applications, including object identification, even an incorrect correspondence interferes with this process. On the other hand, all the reviewed methods have increased the accuracy of the comparisons by reducing the total number of correspondences, while the total number of correspondences is very effective in subsequent processes. Nowadays, creating matching methods which can match the maximum keypoints and create the most correct matches has been one of the important challenges.
In this paper, first the keypoints are identified using the SIFT. The first innovation of this paper is the use of a 64-dimensional SIFT descriptor to describe keypoints in remote sensing images, which improves the speed and increases the matching accuracy compared to the classic 128-dimensional SIFT descriptor. The NNDR matching method is one of the common methods of local matching, which has problems, such as increasing the number of local incorrect matches and matching multiple errors. Next innovation, a global matching method is introduced to solve these problems. The proposed matching method is based on calculating the distances of keypoints and their transformed ones. The proposed matching method does not create multiple matches and local incorrect matches. The proposed matching method was able to match the maximum keypoints and maximized the correct matches.
The remainder of article is organized as follows. Section 2 presents the NNDR matching method and its problems. Section 3 presented the proposed method. In section 4 results are described and the paper is concluded in section 5.
2- NNDR Matching Method and Its Problems
In this section, the NNDR matching method is described briefly, and then the disadvantages of this algorithm in the remote sensing image registration are reported.
2-1- NNDR Matching Method
The matching operation is done using descriptors of each feature. First the Euclidean distance between descriptors in both images is computed and then an appropriate nearest neighbor distance ratio (NNDR) criterion is used. For each feature descriptor (
) in the first image, its Euclidean distance to the first nearest (
) and the second nearest (
) neighbors in the second image are calculated, and their ratio is computed according to (1) [24]. If this ratio is smaller than a pre-determined threshold value 
, the matching is performed. The value of is set at 0.8 [16].
| (1) |
2-2- NNDR Matching Method Problems
The NNDR matching method is a common method, which one of its drawbacks is creating multiple matches, as shown in the schematic Fig. 1. In the figure, the NNDR method has matched the keypoints of p3, p4, and p5 to a same point at q4 (i.e., multiple matches). In this illustration, matches done for keypoints p5 and p3 are incorrect. On the other hand, the NNDR is a local matching method; thus, it cannot generate an accurate model for the whole image. For example, the point p5 is located in the vicinity of point p4 which is another weakness of this matching method; that causes p5 to be mistaken for q4. Multiple matches and local matchings increase incorrect correspondences and decrease matching accuracy; the importance of which leads to the proposal of a novel matching method based on a global transformation model.
Fig. 1 The NNDR based matching method
Fig. 2 Remote sensing image registration system
3- The proposed Method
This paper focuses on the feature-based registration method for RS images. Fig. 2 shows the proposed system containing four main phases including (1) feature extraction using the SIFT, (2) descriptors creation using 64-D SIFT descriptor, (3) matching process using the proposed method, and (4) making the transformation model. In this matching approach, other feature extraction algorithms can also be replaced for different circumstances, for example the improved versions of SIFT, Fast, KAZE and etc.
3-1- Feature Extraction
Features of images are identified by the SIFT method. Details of the SIFT are referred to [16].
3-2- 64-D SIFT Descriptor Creation
Once the keypoints are extracted using the SIFT algorithm, the next phase is to create a descriptor as a tool for finding matches between images. In this paper, the 64-D SIFT descriptor is used instead of the 128-D one, using a smaller window [25]; as shown in Fig. 3. In this figure, the keypoint is represented by a blue circle, whose eight neighborhoods are considered according to each keypoint around it. Then, eight directions are found for each neighborhood and a 64-D descriptor is created. This descriptor window increases the matching accuracy and reduces the run-time.
Fig. 3 a 64-D SIFT descriptor
3-3- Matching Process using the Proposed Global Matching
In this sub-section, to solve the problems of the NNDR matching method due to its locality, a new global approach is proposed. In this method, each keypoint in the first image is checked with all the keypoints in the second image, and then the point that has a better fit than the whole set is selected to be match to the desired candidate. This global matching method is based on the Euclidean distances and the transformation model, which is shown in Fig. 4 and described next.
· First, the Euclidean distance between each keypoint in the first image and all keypoints in the second image is computed, and this operation is performed for all keypoints in the first image, to create an m×n matrix. Here, 
and 
are the numbers of keypoints in the first and second images respectively.
· In each row of this matrix, the smallest Euclidean distance is considered as the candidate matching of that keypoints in the first and second images. For example, in Fig. 4, the pairs 
, 
, 
, 
, 
, 
and 
are considered as candidate matches.
· To compute the transformation parameters and the model, three pairs of candidates with the least Euclidean distance, while simultaneously far away each other at least by three pixels are selected. In Fig. 4, the pairs , and are selected to compute the parameters and the model. It is noted that the pair is not chosen since the distance of 
to 
is less than three pixels.
· The transformation model is computed from the mentioned three pairs of candidate points, as in
· (2). Here, 
represents the model parameters.
| (2) |
· The distance between each candidate matching point and its transformed point is computed in the second image. If this distance is less than a specified threshold, the candidate matching point will be approved, otherwise it will not be matched according to (3).
| (3) |
In (3), 
is the distance between the candidate point 
and the transformed point 
. The threshold value 
is simply set at 1.
3-4- Transformation Model Estimation
Several transformation models including affine, similarity, and the thin-plate spline (TPS) are utilized for the geometric distortion between images [26]. The transformation model type is selected according to the number of matches. Similarity [27] and affine distortion [28] are applied in the system of this work. The similarity transformation is chosen where the matches are two and the affine one is selected when this number is in the range of 3-6. Finally, the transformation parameters are computed by the least-square method [29].
4- Results
In this part, the functioning of the proposed method was evaluated using the database in [13]. This database contains five sets of remote sensing images with different texture conditions, such as urban areas and natural landscapes (see Table 1). To perform a fair evaluation, a comparison is made with the classical SIFT-NNDR matching [16], SURF-Delaunay triangulation matching(SURF-DTM) [30] , SURF-NNDR matching [17], SIFT-Sparse Coding(SIFT-SC) [31] and improved SUSAN [32]. The tests were executed on a PC with Intel® CoreTM i7 processor and 16 GB RAM memory, implemented in MATLAB®2018A software. Evaluation criteria and the results of the tests are reported next.
4-1- Evaluation Criteria
To assess the functioning of the proposed method, four assessment indices including RMSE, SITMMR [33], SITMMC [33], and precision are used, as formulated respectively in (4-7). In (4), (xi,yi) and (xi',yi') are the coordinates of the ith matched points pair, and NBTM is the total number of matches. Also, in (5-7), NBCM and NBFM denote the number of true and false matches, respectively. A system with high SITMMC and precision, and also low SITMMR and RMSE is appropriate for the registration task.
| (4) |
| (5) |
| (6) |
| (7) |
Fig. 4 Flowchart of the proposed matching method
Table 1: Properties of the datasets of the remote sensing images
No. | Satellite | Spectral Mode | Image Size | Pixel Size | Bits Per Pixel | Acquisition Date | Location |
1 | SPOT4 | Multispectral Band: 1 | 611×1235 | 20 | 8 | 2001 | China-Hangzhou |
Landsat TM | Multispectral Band: 3 | 648×1230 | 30 | 8 | 2004 | China-Hangzhou | |
2 | SPOT5 | Panchromatic | 1311×1215 | 10 | 8 | 2006 | Canada-Halifax |
Landsat ETM+ | Multispectral Band: 3 | 440×410 | 30 | 8 | 1999 | Canada-Halifax | |
3 | IRS-1C | Panchromatic | 1346×1135 | 5 | 8 | 1998 | Iran-Tehran |
SPOT4 | Panchromatic | 700×590 | 10 | 8 | 1996 | Iran-Tehran | |
4 | IKONOS | Panchromatic | 1288×1085 | 1 | 11 | 2001 | Iran-Shiraz |
IRS-P6 | Panchromatic | 651×548 | 2.5 | 10 | 2006 | Iran-Shiraz | |
5 | IKONOS | Panchromatic | 2335×1987 | 1 | 11 | 2004 | Iran-Tehran |
IRS-1C | Panchromatic | 567×482 | 5 | 8 | 1998 | Iran-Tehran |
4-2- Experimental Setup
Four experiments were performed to assess the functioning of the proposed method in the remote sensing image registration task. In the first experiment, the functioning of the matching phase is examined. In the second experiment, the registration process is evaluated. Then, images artificially contaminated by different speckle noises were used. Finally, the proposed matching method was evaluated on other detectors such as SURF, FAST and KAZE.
4-2-1- Experiments for the Matching Process
In this experiment, the database [13] was used to assess the functioning of the proposed method for the matching process; the results of which are shown in Fig. 5 and Table 2. The number of matchs in SURF-NNDR and SURF-DTM is low. In SIFT-NNDR and SIFT-SC, the number of matches is good, but there are a lot of incorrect matches. In the proposed method, compared to other methods, the number of matches is good and there is no wrong match in Fig.5. As it is clear, the proposed method had higher performance than the classic SIFT-NNDR, SURF-DTM, SURF-NNDR and SIFT-SC. In the SIFT and SURF matching algorithms, using the method based on the NNDR, the total number of matches is reduced and the number of incorrect matches is increased. However, the proposed method increased the number of correct matches and reduced the number of mismatches. Also, the results reported in Table 2 confirmed that our approach revealed higher performance rates with respect to the others. This reflects that this approach outperformed the classic algorithms (SIFT-NNDR matching, SURF-NNDR matching, SURF-DTM, Improved SUSAN) in terms of the mentioned performance criteria. As shown in Table 2, the run-time of our approach was higher than that of the SURF; but the run-time of the proposed method was shorter than the SIFT-SC and SURF-DT algorithms; since the proposed method uses a smaller-length descriptor.
4-2-2- Experiments for the Registration Process
In this test, four pairs of remote sensing images [13] are used and the functioning of the registration methods is investigated; the results are shown in Fig. 6 and Table 3. The black areas in Fig. 6 represent the incorrect registrations. These areas in the SIFT-NNDR matching and SURF-DTM algorithms were due to several incorrect matches. While, the proposed method, the registration process performed well thanks to the improvement of the matching algorithm. Based on the RMSE (as one of the important indices in the registration process), our approach was shown to do well.
4-2-3- Experiments for the Images with Simulated Noise
In this experiment, simulated remote sensing images with different speckle noises were used to evaluate the performance of the proposed method. We added a multiplicative noise to any remote sensing image 
using the relation 
; in which 
denotes a speckle noise with mean 
and different variances 
. The results are shown in Fig. 7.
From Fig. 7, it is deduced that the number of correct matches decreased with the increase of the noise variance in both SIFT-NNDR matching and the proposed method. However, the values for our approach were higher than those for the basic SIFT-NNDR matching. This validates the higher robustness of the proposed method against the speckle noise. Based on the results of this experiment and due to the persistent existence of speckle noises in natural remote sensing images, our approach is expected to work well for matching these images. Also, this technique could be generalized to other types of images such as SAR images, which are usually corrupted by speckle noises.
(a) |
(b) |
(c) |
(d) |
(e) |
Fig. 5 Results of the remote sensing image matching; (a) the classic SIFT- NNDR matching [16]; (b) SURF-NNDR matching [17]; (c) SIFT-SC [31] (d) the SURF-DTM [23]; (e) the proposed method. The squares in this figure show the mismatches.
|
|
|
|
|
|
|
|
|
(a) |
(b) |
(c) |
Fig. 6 Remote sensing image registration. Labels are set for the subfigure columns. (a) the SIFT-NNDR matching [16]; (b) the SURF- DTM[23]; (c) the proposed method
Fig. 7 Comparison of the performance of the SIFT-NNDR matching and proposed method for noisy images. The vertical and horizontal axis represent the number of correct matches and noise variances, respectively.
Table 2: Performance results of the proposed method and other methods
Method | SITMMC | SITMMR | Precision | Time (s) |
SIFT [16] | 0.69 | 0.31 | 0.69 | 0.52 |
SURF [17] | 0.66 | 0.34 | 0.54 | 0.36 |
SURF- DTM [30] | 0.55 | 0.44 | 0.66 | 0.68 |
Imp- SUSAN [32] | 0.63 | 0.36 | 0.72 | 0.62 |
SIFT-SC [31] | 0.74 | 0.26 | 0.79 | 0.72 |
The proposed method | 0.90 | 0.09 | 0.92 | 0.65 |
Table 3: RMSE results of the classic methods and the proposed method
Method | RMSE |
SIFT-NNDR matching [16] | 11.43 |
SURF-NNDR matching [17] | 12.02 |
SURF-DTM [30] | 10.98 |
RKEM-SIFT [18] | 9.82 |
Improved SUSAN [32] | 8.46 |
SIFT-SC [31] | 7.38 |
The proposed method | 6.26 |
Table 4: Results of the proposed matching method on other detectors
Method | Precision |
SURF-proposed global matching | 0.73 |
SURF-NNDR matching | 0.52 |
FAST- proposed global matching | 0.59 |
FAST-NNDR matching | 0.48 |
KAZE- proposed global matching | 0.65 |
KAZE-NNDR matching | 0.57 |
4.2.1 Applying the proposed Matching Method to Other Detectors
In this experiment, the functioning of the proposed matching method on other feature extraction algorithms such as SURF, FAST, KAZE was evaluated and the results are shown in Table 4. As shown in Table 4, performance of the proposed matching method is more effective on SURF, FAST and KAZE identifiers. Precision in SURF-proposed global matching and KAZE-proposed global matching is better than FAST-proposed global matching.
5- Conclusion
In this paper, a novel approach to the remote sensing image registration process was introduced using a combination of the SIFT method, the 64-D SIFT descriptor and the proposed global matching method. Initially, using the SIFT algorithm, keypoints are extracted from images and a 64-D SIFT descriptor is used to describe keypoints. Finally, a new method based on the Euclidean distances of keypoints and the transformation model is proposed to improve matching. The main goals of the proposed method were to improve the matching performance criteria and the run-time. For the future work, improvement of the next processes including remote sensing image mosaicking could be considered.
6- References
[1] A. Wong and D. Clausi, "ARRSI: automatic registration of remote-sensing images," Geoscience and Remote Sensing, IEEE Transactions on, vol. 45, pp. 1483-1493, 2007.
[2] W.-J. Lee and S.-J. Oh, "Remote sensing image registration using equivariance features," in 2021 International Conference on Information Networking (ICOIN), 2021, pp. 776-781.
[3] Z. Hossein-Nejad and M. Nasri, "A New Method in Image Matching Based on Spatial Relationships in Multi-Sensor Remote Sensing Images " Iranian Remote Sensing & GIS, pp. 73-94, 2018.
[4] Y. Liu, H. Cao, Y. Zhao, Q. He, Y. Yang, L. Wang, et al., "A Remote sensing image registration algorithm based on multiple constraints and a variational Bayesian framework," Remote Sensing Letters, vol. 12, pp. 296-305, 2021.
[5] Z. Hossein-Nejad and M. Nasri, "A Review on Image Registration Methods, Concepts and applications," Journal of Machine Vision and Image Processing, pp. 39-67, 2017.
[6] W. Lee, D. Sim, and S.-J. Oh, "A CNN-Based High-Accuracy Registration for Remote Sensing Images," Remote Sensing, vol. 13, p. 1482, 2021.
[7] H.-M. Chen, M. K. Arora, and P. K. Varshney, "Mutual information-based image registration for remote sensing data," International Journal of Remote Sensing, vol. 24, pp. 3701-3706, 2003.
[8] X. Xie, Y. Zhang, X. Ling, and X. Wang, "A novel extended phase correlation algorithm based on Log-Gabor filtering for multimodal remote sensing image registration," International Journal of Remote Sensing, vol. 40, pp. 5429-5453, 2019.
[9] M. I. Patel, V. K. Thakar, and S. K. Shah, "Image registration of satellite images with varying illumination level using HOG descriptor based SURF," Procedia computer science, vol. 93, pp. 382-388, 2016.
[10] P. Schwind, S. Suri, P. Reinartz, and A. Siebert, "Applicability of the SIFT operator to geometric SAR image registration," International Journal of Remote Sensing, vol. 31, pp. 1959-1980, 2010.
[11] G. Sreeja and O. Saraniya, "A Comparative Study on Image Registration Techniques for SAR Images," in 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), 2019, pp. 947-953.
[12] Y. Ye, J. Shan, L. Bruzzone, and L. Shen, "Robust registration of multimodal remote sensing images based on structural similarity," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 2941-2958, 2017.
[13] A. Sedaghat, M. Mokhtarzade, and H. Ebadi, "Uniform robust scale-invariant feature matching for optical remote sensing images," IEEE Transactions on Geoscience and Remote Sensing, vol. 49, pp. 4516-4527, 2011.
[14] Z. Hossein-Nejad and M. Nasri, "A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration," Biomedical Signal Processing and Control, vol. 45, pp. 325-338, 2018.
[15] C. Harris and M. Stephens, "A combined corner and edge detector," in Alvey vision conference, 1988, p. 50.
[16] D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, pp. 91-110, 2004.
[17] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-up robust features (SURF)," Computer vision and image understanding, vol. 110, pp. 346-359, 2008.
[18] Z. Hossein-Nejad and M. Nasri, "RKEM: Redundant Keypoint Elimination Method in Image Registration," IET Image Processing, vol. 11, pp. 273-284, 2017.
[19] A. Sedaghat and N. Mohammadi, "High-resolution image registration based on improved SURF detector and localized GTM," International Journal of Remote Sensing, vol. 40, pp. 2576-2601, 2019.
[20] A. Sedaghat and H. Ebadi, "Remote sensing image matching based on adaptive binning SIFT descriptor," IEEE transactions on geoscience remote Sensing Technology and Application, vol. 53, pp. 5283-5293, 2015.
[21] M. Hasan, M. R. Pickering, and X. Jia, "Modified SIFT for multi-modal remote sensing image registration," in 2012 IEEE International Geoscience and Remote Sensing Symposium, 2012, pp. 2348-2351.
[22] B. Kupfer, N. S. Netanyahu, and I. Shimshoni, "An Efficient SIFT-Based Mode-Seeking Algorithm for Sub-Pixel Registration of Remotely Sensed Images," Geoscience and Remote Sensing Letters, IEEE, vol. 12, pp. 379-383, 2015.
[23] Z. Yi, C. Zhiguo, and X. Yang, "Multi-spectral remote image registration based on SIFT," Electronics Letters, vol. 44, pp. 107-108, 2008.
[24] K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, pp. 1615-1630, 2005.
[25] N. Y. Khan, B. McCane, and G. Wyvill, "SIFT and SURF performance evaluation against various image deformations on benchmark dataset," in 2011 International Conference on Digital Image Computing: Techniques and Applications, 2011, pp. 501-506.
[26] R. Bouchiha and K. Besbes, "Comparison of local descriptors for automatic remote sensing image registration," Signal, Image and Video Processing, vol. 9, pp. 463-469, 2015.
[27] M. Deshmukh and U. Bhosle, "A survey of image registration," International Journal of Image Processing (IJIP), vol. 5, p. 245, 2011.
[28] M. Hasan, X. Jia, A. Robles-Kelly, J. Zhou, and M. R. Pickering, "Multi-spectral remote sensing image registration via spatial relationship analysis on sift keypoints," in Geoscience and Remote Sensing Symposium (IGARSS), 2010 IEEE International, 2010, pp. 1011-1014.
[29] S. Saxena and R. K. Singh, "A survey of recent and classical image registration methods," International journal of signal processing, image processing and pattern recognition, vol. 7, pp. 167-176, 2014.
[30] S. Chen, S. Zheng, Z. Xu, C. Guo, and X. Ma, "AN IMPROVED IMAGE MATCHING METHOD BASED ON SURF ALGORITHM," International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, vol. 42, 2018.
[31] P. Etezadifar and H. Farsi, "A New Sample Consensus Based on Sparse Coding for Improved Matching of SIFT Features on Remote Sensing Images," IEEE Transactions on Geoscience and Remote Sensing, 2020.
[32] W. He and X. Deng, "A modified SUSAN corner detection algorithm based on adaptive gradient threshold for remote sensing image," in 2010 International Conference on Optoelectronics and Image Processing, 2010, pp. 40-43.
[33] Z. Hossein-Nejad and M. Nasri, "An adaptive image registration method based on SIFT features and RANSAC transform," Computers & Electrical Engineering, vol. 62, pp. 524-537, 2017.
Zahra Hossein-Nejad received the B.Sc and M.Sc degrees in Electrical Engineering from the Islamic Azad University, Jahrom, and Sirjan branches, Iran, in 2013 and 2016, respectively. She is a Ph.D. student in Electrical Engineering at Shiraz University, Shiraz, Iran. Her research interests include image processing and computer vision.
Hamed Agahi received B.Sc, M.Sc and Ph.D degrees in Electrical Engineering from University of Shiraz, Amirkabir University of Technology and University of Tehran, Iran, in 2005, 2008 and 2013, respectively. From 2009, he was with the Islamic Azad University, Shiraz Branch, Shiraz, Iran. His research interests include pattern recognition, image and signal processing, and fault detection and diagnosis applications.
Azar Mahmoodzadeh received B.Sc, M.Sc and Ph.D degrees in Electrical Engineering from University of Shiraz, University of Shahed and University of Yazd, Iran, in 2005, 2008 and 2013, respectively. From 2009, she was with the Islamic Azad University, Shiraz Branch, Shiraz, Iran. Her research interests include pattern recognition and image and signal processing.
