GOA-ISR: A Grasshopper Optimization Algorithm for Improved Image Super-Resolution
محورهای موضوعی : Image Processing
Bahar Ghaderi
1
,
Hamid Azad
2
,
Hamed Agahi
3
1 - Department of Electrical-Telecommunication Engineering,Faculty of Engineering Shiraz Branch ,Islamic Azad University,Shiraz,Iran
2 - Department of Electrical-Telecommunication Engineering,Faculty of Engineering Shiraz Branch ,Islamic Azad University,Shiraz,Iran
3 - Department of Electrical-Telecommunication Engineering,Faculty of Engineering Shiraz Branch ,Islamic Azad University,Shiraz,Iran
کلید واژه: Super Resolution (SR), High-Resolution (HR), Low-Resolution (LR), Learning-based Methods, Grasshopper Optimization Algorithm (GOA).,
چکیده مقاله :
The image super-resolution (ISR) process provides a high-resolution (HR) image from an input low-resolution (LR) image. This process is an important and challenging issue in computer vision and image processing. Various methods are used for ISR, that learning-based methods are one of the most widely used methods in this field. In this approach, a set of training images is used in various learning based ISR methods to reconstruct the input LR image. To this end, appropriate reconstruction weights for the image must be computed. In general, the least-squares estimation (LSE) approach is used for obtaining optimal reconstruction weights. The accuracy of SR depends on the effectiveness of minimizing the LSE problem. Therefore, it is still a challenge to obtain more accurate reconstruction weights for better SR processing. In this study, a Grasshopper Optimization Algorithm (GOA)-based ISR method (GOA-ISR) is proposed in order to minimize the LSE problem more effectively. A new formulation for the upper bound and the lower bound is introduced to make the search process of the GOA algorithm suitable for ISR. The simulation results on DIVerse 2K (DIV2K) dataset, URBAN100, BSD100, Set 14 and Set 5 datasets affirm the advantage of the proposed GOA-ISR approach in comparison with some other basic Neighbor Embedding (NE), Sparse Coding (SC), Adaptive Sparse, Iterative Kernel Correction (IKC), Second-order Attention Network (SAN), Sparse Neighbor Embedding and Grey Wolf Optimizer (GWO) methods in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The results of the experiments show the superiority of the proposed method comparing to the best compared method (DWSR) increases 8.613 % PSNR.
The image super-resolution (ISR) process provides a high-resolution (HR) image from an input low-resolution (LR) image. This process is an important and challenging issue in computer vision and image processing. Various methods are used for ISR, that learning-based methods are one of the most widely used methods in this field. In this approach, a set of training images is used in various learning based ISR methods to reconstruct the input LR image. To this end, appropriate reconstruction weights for the image must be computed. In general, the least-squares estimation (LSE) approach is used for obtaining optimal reconstruction weights. The accuracy of SR depends on the effectiveness of minimizing the LSE problem. Therefore, it is still a challenge to obtain more accurate reconstruction weights for better SR processing. In this study, a Grasshopper Optimization Algorithm (GOA)-based ISR method (GOA-ISR) is proposed in order to minimize the LSE problem more effectively. A new formulation for the upper bound and the lower bound is introduced to make the search process of the GOA algorithm suitable for ISR. The simulation results on DIVerse 2K (DIV2K) dataset, URBAN100, BSD100, Set 14 and Set 5 datasets affirm the advantage of the proposed GOA-ISR approach in comparison with some other basic Neighbor Embedding (NE), Sparse Coding (SC), Adaptive Sparse, Iterative Kernel Correction (IKC), Second-order Attention Network (SAN), Sparse Neighbor Embedding and Grey Wolf Optimizer (GWO) methods in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The results of the experiments show the superiority of the proposed method comparing to the best compared method (DWSR) increases 8.613 % PSNR.
[1] B. Ghaderi and H. Azad, "Deep Learning Algorithms in Super-Resolution Images," Journal of Circuits, Data and Systems Analysis, vol. 1, no. 1, p. 47, 2023.
[2] P. Behjati, P. Rodriguez, C. Fernández, I. Hupont, A. Mehri, and J. Gonzàlez, "Single image super-resolution based on directional variance attention network," Pattern Recognition, vol. 133, p. 108997, 2023.
[3] T. Goto, T. Fukuoka, F. Nagashima, S. Hirano, and M. Sakurai, "Super-resolution System for 4K-HDTV," in 2014 22nd International Conference on Pattern Recognition, 2014: IEEE, pp. 4453-4458.
[4] A. Rapuano, G. Iovane, and M. Chinnici, "A scalable Blockchain based system for super resolution images manipulation," in 2020 IEEE 6th International Conference on Dependability in Sensor, Cloud and Big Data Systems and Application (DependSys), 2020: IEEE, pp. 8-15.
[5] H. Dastmalchi and H. Aghaeinia, "Super-resolution of very low-resolution face images with a wavelet integrated, identity preserving, adversarial network," Signal Processing: Image Communication, p. 116755, 2022.
[6] I. Taghavi et al., "Ultrasound super-resolution imaging with a hierarchical Kalman tracker," Ultrasonics, vol. 122, p. 106695, 2022.
[7] P. Wang, B. Bayram, and E. Sertel, "A comprehensive review on deep learning based remote sensing image super-resolution methods," Earth-Science Reviews, p. 104110, 2022.
[8] K. Zhu, H. Guo, S. Li, and X. Lin, "Online tool wear monitoring by super-resolution based machine vision," Computers in Industry, vol. 144, p. 103782, 2023.
[9] H. Hou and H. Andrews, "Cubic splines for image interpolation and digital filtering," IEEE Transactions on acoustics, speech, and signal processing, vol. 26, no. 6, pp. 508-517, 1978.
[10] M. Li and T. Q. Nguyen, "Markov random field model-based edge-directed image interpolation," IEEE Transactions on Image Processing, vol. 17, no. 7, pp. 1121-1128, 2008.
[11] J. Sun, J. Zhu, and M. F. Tappen, "Context-constrained hallucination for image super-resolution," in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010: IEEE, pp. 231-238.
[12] L. Wang, S. Xiang, G. Meng, H. Wu, and C. Pan, "Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 8, pp. 1289-1299, 2013.
[13] W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based super-resolution," IEEE Computer graphics and Applications, vol. 22, no. 2, pp. 56-65, 2002.
[14] C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in European conference on computer vision, 2014: Springer, pp. 184-199.
[15] K. Zhang, J. Li, H. Wang, X. Liu, and X. Gao, "Learning local dictionaries and similarity structures for single image super-resolution," Signal Processing, vol. 142, pp. 231-243, 2018.
[16] N. Kumar and A. Sethi, "Fast learning-based single image super-resolution," IEEE Transactions on Multimedia, vol. 18, no. 8, pp. 1504-1515, 2016.
[17] J. Jiang, C. Wang, X. Liu, and J. Ma, "Deep learning-based face super-resolution: A survey," ACM Computing Surveys (CSUR), vol. 55, no. 1, pp. 1-36, 2021.
[18] P. P. Gajjar and M. V. Joshi, "New learning based super-resolution: use of DWT and IGMRF prior," IEEE Transactions on Image Processing, vol. 19, no. 5, pp. 1201-1213, 2010.
[19] S. S. Rajput, V. K. Bohat, and K. Arya, "Grey wolf optimization algorithm for facial image super-resolution," Applied Intelligence, vol. 49, no. 4, pp. 1324-1338, 2019.
[20] K. Nguyen, C. Fookes, S. Sridharan, M. Tistarelli, and M. Nixon, "Super-resolution for biometrics: A comprehensive survey," Pattern Recognition, vol. 78, pp. 23-42, 2018.
[21] N. Wang, D. Tao, X. Gao, X. Li, and J. Li, "A comprehensive survey to face hallucination," International journal of computer vision, vol. 106, no. 1, pp. 9-30, 2014.
[22] Y. Tang, P. Yan, Y. Yuan, and X. Li, "Single-image super-resolution via local learning," International Journal of Machine Learning and Cybernetics, vol. 2, no. 1, pp. 15-23, 2011.
[23] K. Zhang, X. Gao, D. Tao, and X. Li, "Single image super-resolution with non-local means and steering kernel regression," IEEE Transactions on Image Processing, vol. 21, no. 11, pp. 4544-4556, 2012.
[24] K. Zhang, X. Gao, D. Tao, and X. Li, "Single image super-resolution with multiscale similarity learning," IEEE transactions on neural networks and learning systems, vol. 24, no. 10, pp. 1648-1659, 2013.
[25] Z. Wang, Y. Yang, Z. Wang, S. Chang, J. Yang, and T. S. Huang, "Learning super-resolution jointly from external and internal examples," IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4359-4371, 2015.
[26] X. Lu, H. Yuan, Y. Yuan, P. Yan, L. Li, and X. Li, "Local learning-based image super-resolution," in 2011 IEEE 13th International Workshop on Multimedia Signal Processing, 2011: IEEE, pp. 1-5.
[27] L. An and B. Bhanu, "Image super-resolution by extreme learning machine," in 2012 19th IEEE international conference on image processing, 2012: IEEE, pp. 2209-2212.
[28] M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding," 2012.# [29] S. Saremi, S. Mirjalili, and A. Lewis, "Grasshopper optimisation algorithm: theory and application," Advances in engineering software, vol. 105, pp. 30-47, 2017.
[30] I. J. Cox, M. L. Miller, J. A. Bloom, and C. Honsinger, Digital watermarking. Springer, 2002.
[31] P. Tumuluru and B. Ravi, "GOA-based DBN: Grasshopper optimization algorithm-based deep belief neural networks for cancer classification," International Journal of Applied Engineering Research, vol. 12, no. 24, pp. 14218-14231, 2017.
[32] P.-H. Dinh, "A novel approach based on grasshopper optimization algorithm for medical image fusion," Expert Systems with Applications, vol. 171, p. 114576, 2021.
[33] J. H. Holland, "Genetic algorithms," Scientific american, vol. 267, no. 1, pp. 66-73, 1992.
[34] R. Eberhart and J. Kennedy, "A new optimizer using particle swarm theory," in MHS'95. Proceedings of the sixth international symposium on micro machine and human science, 1995: Ieee, pp. 39-43.
[35] A. Kamalinia and A. Ghaffari, "Hybrid task scheduling method for cloud computing by genetic and PSO algorithms," J. Inf. Syst. Telecommun, vol. 4, pp. 271-281, 2016.
[36] X.-S. Yang, "Firefly algorithms for multimodal optimization," in International symposium on stochastic algorithms, 2009: Springer, pp. 169-178.
[37] A. Mahmoodzadeh, H. Agahi, and M. Salehi, "Handwritten Digits Recognition Using an Ensemble Technique Based on the Firefly Algorithm," Journal of Information Systems and Telecommunication (JIST), vol. 3, no. 23, p. 136, 2019.
[38] X.-S. Yang, "A new metaheuristic bat-inspired algorithm," in Nature inspired cooperative strategies for optimization (NICSO 2010): Springer, 2010, pp. 65-74.
[39] E. Rashedi, E. Rashedi, and H. Nezamabadi-Pour, "A comprehensive survey on gravitational search algorithm," Swarm and evolutionary computation, vol. 41, pp. 141-158, 2018.
[40] M. Tourani, "Improvement of Firefly Algorithm using Particle Swarm Optimization and Gravitational Search Algorithm," Journal of Information Systems and Telecommunication (JIST), vol. 2, no. 34, p. 123, 2021.
[41] C. M. Topaz, A. J. Bernoff, S. Logan, and W. Toolson, "A model for rolling swarms of locusts," The European Physical Journal Special Topics, vol. 157, no. 1, pp. 93-109, 2008.
[42] S. M. Rogers, T. Matheson, E. Despland, T. Dodgson, M. Burrows, and S. J. Simpson, "Mechanosensory-induced behavioural gregarization in the desert locust Schistocerca gregaria," Journal of Experimental Biology, vol. 206, no. 22, pp. 3991-4002, 2003.
[43]Y. ChangH, "XiongY. Super-resolutionthroughneighborembedding," Proceedingsofthe2004IEEEComputer Society C o nference on ComputerVision and Pattern Rec ogni—tion, pp. 275-282, 2004.
[44]J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE transactions on image processing, vol. 19, no. 11, pp. 2861-2873, 2010.
[45]J. Gu, H. Lu, W. Zuo, and C. Dong, "Blind super-resolution with iterative kernel correction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1604-1613.
[46]X. Gao, K. Zhang, D. Tao, and X. Li, "Image super-resolution with sparse neighbor embedding," IEEE Transactions on Image Processing, vol. 21, no. 7, pp. 3194-3205, 2012.
[47]W. Dong, L. Zhang, G. Shi, and X. Wu, "Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization," IEEE Transactions on image processing, vol. 20, no. 7, pp. 1838-1857, 2011.
[48]T. Dai, J. Cai, Y. Zhang, S.-T. Xia, and L. Zhang, "Second-order attention network for single image super-resolution," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 11065-11074.
[49]E. Agustsson and R. Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 126-135.
[50]F. Akhlaghian Tab, K. Ghaderi, and P. Moradi, "A New Robust Digital Image Watermarking Algorithm Based on LWT-SVD and Fractal Images," Journal of Information Systems and Telecommunication (JIST), vol. 1, no. 9, p. 1, 2015.
[51]K. Li, S. Yang, R. Dong, X. Wang, and J. Huang, "Survey of single image super‐resolution reconstruction," IET Image Processing, vol. 14, no. 11, pp. 2273-2290, 2020.
[52]V. K. Bohat and K. Arya, "An effective gbest-guided gravitational search algorithm for real-parameter optimization and its application in training of feedforward neural networks," Knowledge-Based Systems, vol. 143, pp. 192-207, 2018.
[53] S.-C. Chu, Z.-C. Dou, J.-S. Pan, L. Kong, V. Snášel, and J. Watada, "DWSR: an architecture optimization framework for adaptive super-resolution neural networks based on meta-heuristics," Artificial Intelligence Review, vol. 57, no. 2, p. 23, 2024.
[54] Y. Meraihi, A. B. Gabis, S. Mirjalili, and A. Ramdane-Cherif, "Grasshopper optimization algorithm: theory, variants, and applications," IEEE Access, vol. 9, pp. 50001-50024, 2021.
[55] J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5197-5206.
[56] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proceedings eighth IEEE international conference on computer vision. ICCV 2001, 2001, vol. 2: IEEE, pp. 416-423.
[57] R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7, 2012: Springer, pp. 711-730.
http://jist.acecr.org ISSN 2322-1437 / EISSN:2345-2773 |
Journal of Information Systems and Telecommunication
|
GOA-ISR: A Grasshopper Optimization Algorithm for Improved Image Super-Resolution |
Hamid Azad1*, Bahar Ghaderi1 , Hamed Agahi1
|
1.Department of Electrical-Telecommunication Engineering ,Faculty of Engineering Shiraz Branch ,Islamic Azad University, Shiraz ,Iran
|
Received: 03 Dec 2023/ Revised: 04 Sep 2024/ Accepted: 14 Oct 2024 |
|
Abstract
The image super-resolution (ISR) process provides a high-resolution (HR) image from an input low-resolution (LR) image. This process is an important and challenging issue in computer vision and image processing. Various methods are used for ISR, that learning-based methods are one of the most widely used methods in this field. In this approach, a set of training images is used in various learning based ISR methods to reconstruct the input LR image. To this end, appropriate reconstruction weights for the image must be computed. In general, the least-squares estimation (LSE) approach is used for obtaining optimal reconstruction weights. The accuracy of SR depends on the effectiveness of minimizing the LSE problem. Therefore, it is still a challenge to obtain more accurate reconstruction weights for better SR processing. In this study, a Grasshopper Optimization Algorithm (GOA)-based ISR method (GOA-ISR) is proposed in order to minimize the LSE problem more effectively. A new formulation for the upper bound and the lower bound is introduced to make the search process of the GOA algorithm suitable for ISR. The simulation results on DIVerse 2K (DIV2K) dataset, URBAN100, BSD100, Set 14 and Set 5 datasets affirm the advantage of the proposed GOA-ISR approach in comparison with some other basic Neighbor Embedding (NE), Sparse Coding (SC), Adaptive Sparse, Iterative Kernel Correction (IKC), Second-order Attention Network (SAN), Sparse Neighbor Embedding and Grey Wolf Optimizer (GWO) methods in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The results of the experiments show the superiority of the proposed method comparing to the best compared method (DWSR) increases 8.613 % PSNR.
Keywords: Super Resolution (SR); High-Resolution (HR); Low-Resolution (LR); Learning-based Methods; Grasshopper Optimization Algorithm (GOA).
1- Introduction
The super-resolution (SR) process creates a high-resolution (HR) image using low-resolution (LR) images[1]. In other words, it converts LR images to HR images [2]. This process has been used in HDTV [3], image manipulation [4], face recognition [5], medical imaging [6], remote sensing [7], and monitoring [8]. In general, image SR methods can be divided into three categories: interpolation-based methods [9, 10], reconstruction-based methods [11, 12], and learning-based methods [13, 14]. In interpolation-based methods and reconstruction-based methods, only the features of the input LR images are used to produce a HR image. In learning-based methods, however, the information of external images along with input LR image is used to produce HR image [15]. Learning-based methods have received more attention in recent years due to their superiority for the SR process. [15-18]
Learning-based SR methods can be classified into two categories: global image-based approaches and local patch-based approaches [19]. In global image-based approaches, the entire content of the input LR image is used, whereas in the local patch-based approaches, the content of the input LR image is divided into several parts, and each part is used separately to recover the HR image. Local patch-based methods are more suitable than global based-methods for image reconstruction. [20, 21]
In [22], a local learning-based method was presented for the SR image processing task. In this method, a local training set was developed according to the similarity between the training samples and the test sample, and the local regression function was used on the local training set.
In [23], local and non-local learning-based methods of LR images were proposed for the SR process. In this method, the non-local mean filter was used for the non-local learning, and the regression of the steering kernel was used for local learning.
In [24], the multi-scale similarity learning method was presented for the SR process. In this approach, the input LR image patch was first iterated several times at the same scale and also across different scales. Then, HR-LR patch pairs were created to preserve details, using the original LR input and its down-sampled version to extract similarities at different scales from the images. The neighbor embedding algorithm was finally used to estimate the relationships between LR and HR image pairs.
In [25], a joint SR model was proposed, which had the advantages of the external and the internal SR methods. Two loss functions, (sparse coding-based external samples and epitomic matching-based internal samples) were used in this method.
In [26], a new local learning method, which was based on the kernel ridge regression (KRR), was presented for the SR process. Gabor filter was used to extract texture information from LR patches as features. Then, each input LR feature patch was used by the K-nearest neighbor algorithm to create a local structure. Finally, the KRR was used to map the input LR feature patches to HR feature patches in the local structure.
In [27], an SR approach was proposed based on extreme learning machine (ELM). In the training phase of algorithm, the high-frequency components of the original HR images were given as target values and the image features of the LR images were imported to the ELM to learn a model. In this method, the details and fine structures in LR images were reconstructed well.
In [28], a new method based on non-negative neighbor embedding was presented for the SR process. In this method, a dictionary containing patches of LR images and patches of HR images was used for training. Each LR feature vector in the input image was expressed as the weighted combination of its K nearest neighbors in the dictionary; the corresponding HR feature vector was reconstructed under the assumption that the local LR embedding was preserved.
In all these learning-based methods, the optimal reconstruction weights are obtained by calculating the least-squares error between the input LR image and training LR patches, and then the generated weights are applied to the same HR training patches to reconstruct the output HR patch. All reconstructed HR patches are finally combined together to create a complete HR image. The accuracy of SR depends on the effectiveness of minimizing the least-squares error problem. Therefore, it is still a challenge to obtain more accurate reconstruction weights for better SR processing. The various meta-heuristic algorithms introduced so far can be used for obtaining the weight value for the optimum reconstruction. The grasshopper optimization algorithm (GOA) [29] is one of these methods. This algorithm was introduced by Saremi [29] based on the cooperative behavior of grasshoppers in 2017. The GOA has been widely used in various applications, such as the digital watermarking [30], cancer classification [31], and medical image fusion [32]. Compared to other meta-heuristic algorithms, including Genetic Algorithm (GA) [33], Particle Swarm Optimization (PSO) [34, 35], Firefly Algorithm (FA) [36, 37], Bat Algorithm (BA) [38], and Gravitational Search Algorithm (GSA) [39, 40], this algorithm can avoid local optima, showing a good balance between exploration and exploitation, due to the high amount of exploitation and convergence features. These advantages encouraged the proposal of the GOA for the optimal reconstruction weight value in the SR process.
The rest of this study are organized as follows. Section II is dedicated to a review of GOA. In Section III and Section IV, the proposed method and the simulation results are presented respectively. Section V represents the conclusions of the study.
2- Grasshopper Optimization Algorithm (GOA)
As chewing herbivorous insects, grasshoppers are one of the largest groups among all those creatures. The unique aspect of a cloud of grasshopper is that the group life behavior can be observed in both adult and infant grasshoppers. Millions of newborn grasshoppers jump and move like spinning cylinders. As they become older, they form a group in the air. This is how grasshoppers migrate over long distances. The main characteristic of these groups in the larval stage is the slow movement and small steps of the grasshoppers. In contrast, prolonged and sudden movement is a key feature of these groups among older grasshoppers. Searching for food resources is an important feature of group life among grasshoppers [41]. The life of these insects and their group search for food were the inspirations for generating the GOA. Nature-inspired algorithms generally split the search process into two phases: the exploration and the exploitation. In the exploration phase, search agents are stimulated to move abruptly while tending to passage locally during the exploitation step. These two operations as well as searching for the target are done instinctively by the grasshoppers. The mathematical model for simulating the group behavior of the grasshopper’s movements is described according to Equation (1). [29]
| (1) |
Where 
defines the location of the 
grasshopper, 
is the so-called interaction computed according to Equation (2), 
is the gravity force on the grasshopper, and 
represents the wind advection [41]. To provide a random behavior, Equation (1) is rewritten as 
where
, 
and 
are random numbers belong to
. The interaction is calculated using the following equation. [29]
| (2) |
Where 
is the distance between and 
grasshoppers computed as
, and 
is a unit vector from the former grasshopper to the latter one. Moreover, s is a function that defines the strength of social forces according to Equation (3). [29]
| (3) |
Where f represents the intensity of attraction, and l is the attractive length scale. The detailed description of the GOA is available in the main references [41, 42]. In an optimization problem involving p parameters, a vector of length p is constructed, representing the position of an individual grasshopper within a swarm consisting of multiple insects. According to Equation (1), the position of each grasshopper in the swarm is updated, with respect to the mentioned factors and the optimization objective function in each iteration. After a specified number of iterations, the grasshopper with the optimal objective function is selected as the best answer for the optimization problem.
3- Proposed Grasshopper Optimization Algorithm-Based Image Super-Resolution Method
In this article, a new approach based on the GOA is proposed to obtain optimal reconstruction weights in the SR process. First, each of the input LR image and LR training images is divided into several patches. Then, the distance between each input LR patch and the same patch position in all the LR training images is calculated according to Equation (4).
(4)
Where
and 
are input LR image and LR training images, respectively, n is the number of patches, and m is the number of LR training images.
The upper bound and lower bound are then calculated according to equations 5 and 6 to limit the spatial range of searching for optimal weights. The GOA is used as the optimizer for the objective function in Equation (7). This algorithm returns the optimal weight vector 
for the input patch 
calculated from the same position training LR patches, i.e., 
. The generated weight vectors for each patch are finally used with similar patches in the training HR images to reconstruct the super-resolved patch according to Equation (8). All the 
patches are combined to create the final HR image. In overlapping regions, the final value is the average of pixel values. The flowchart and pseudo-code of the proposed method are shown in Fig. 1 and Fig. 2, respectively.
(5)
(6)
(7)
(8)
The 
and 
are adjustable parameters with fixed values.
Algorithm 1: Pseudo-code of the proposed GOA-ISR method in the SR process |
Input: LR input image, LR and HR training sets: Output: Output HR image · Each LR input image and LR training images are divided into 4×4 size patches. · Each HR training image is divided into 16×16 size patches. · For n=1 Select patch Compute similarity between input patch and training patches according to Equation (4), and they are sorted in ascending order in a vector then. Calculate the upper bound and lower bound according to equations (5 and 6) Call the GOA with cost function: Obtain the optimal Create HR patches using training HR patches and optimal weights · End for · Combine all the |
Fig. 1. Pseudo-code of the proposed GOA-ISR method in the SR process
PP |
P |
PP |
P |
P |
P |
P |
P |
Fig.2. Description of the proposed GOA-ISR method First, optimal reconstruction weights are achieved by expressing an input patch in-terms of training LR patches using proposed GOA-ISR. Then, generated weights are applied to counterpart HR training patches for reconstructing the output patch. Finally, all reconstructed patches are joined to build a complete final HR image
4- Implementation and Examination of the Results
In this section, a comprehensive evaluation of the proposed GOA-ISR method is conducted, comparing it against existing methods, including Bicubic [9], Neighbor Embedding (NE) [43], Sparse Coding (SC) [44], Iterative Kernel Correction (IKC) [45], Sparse Neighbor Embedding [46], Adaptive Sparse [47], Second-order Attention Network (SAN) [48], and Grey Wolf Optimizer (GWO) [19]. All of the experiments are performed using MATLAB® 2019A software on a personal computer with an Intel Core i7 processor and 16G RAM. Two databases are used in order to examine the capability of the suggested GOA-ISR method. The database of natural images includes thousands of high-quality and low-quality images [44]. The DIV2K database is published by Timofte et al. for ISR [49]. DIV2K consists of 800 training images, 100 validation images, and 100 test images.
4-1- Evaluation Criteria
To evaluate the effectiveness of the proposed GOA-ISR method in IRS, peak signal-to-noise ratio (PSNR)[50] and structural similarity index measure (SSIM)[51] can be used besides the subjective visual appearance.
(9)
(10)
(11)
Where, 
and 
are the original and the reconstructed images, respectively; m and n are the height and width of the image; 
and 
are mean intensities of images (original and reconstructed images); 
, and 
represent the standard deviation of original and reconstructed image, respectively; 
is the covariance of images; 
and 
are constants that is 0.01 and is 0.03.
4-2- Experimental Results
In these experiments, the algorithm parameters are selected as constant in order to prevent the selection of the parameters’ values from affecting the results of SR process (Table 1). The test and training sets are completely non-overlapped. For HR images, the patch size and overlap between patches are set to 16 × 16 and 12 pixels, respectively. Similarly, it is set to 4×4 and 3 pixels, respectively, for LR images. Four sets of tests are done as follows: The qualitative performance of the proposed method is checked in the first set; the quantitative performance of the proposed method is examined in the second set; and the performance of other meta-heuristic algorithms for the SR process is examined in the third set. The performance of the proposed method is checked on different databases in the fourth set.
Table 1. Parameter values in different methods
Algorithm | Parameters | Value |
GWO[19] | a Population size Maximum iteration | 2 to 0 100 100 |
NE[46] | K | 12 |
Sparse Neighbor Embedding[46] |
| 0.0001
4 0.9 |
FA |
| 1
1
0 to 1 |
PSO[19] | w c1 c2 Population size Maximum iterations | 0.9 to 0.2 2 to 0 0 to 2 100 100 |
GSA[52] |
c1 c2 Population size Maximum iterations | 5 100 2 to 0.1 0 to 1 100 100 |
GOA | Population size Maximum iteration | 100 100 |
GOA-ISR | Population size Maximum iteration
| 100 100 0.8 0.1 |
4-2-1- Qualitative Investigation of the Proposed GOA-ISR Method Performance
In this experiment, natural images [44] are used to qualitatively check the performance of the proposed GOA-ISR method in the SR process, and an example of SR results using the proposed method is shown in Fig. 3.
(a) |
(b) |
(c) |
|
|
|
|
|
|
(d) |
(e) |
(f) |
Fig. 3. Results of different methods of SR process, (a) Bicubic[9] (RMSE=4.18 ),(b) NE [43] (RMSE=4.23), (c) SC[44] (RMSE=4.03), (d) IKC [45] (RMSE=3.52), (e) Proposed GOA-ISR method (RMSE=3.21) (f) Ground truth
As shown in Fig. 3, the SR process in the Bicubic and NE methods has created artifact edges, while the SC method has caused the removal of sharp edges and blurring. IKC method has fewer artifact edges than Bicubic and NE methods, and thus, this method performs better than the Bicubic and NE methods in the SR process. Sharp edges in the proposed GOA-ISR method are recovered better than in other methods and have much clearer details with less artifacts. The image of the proposed GOA-ISR method is very close to the ground truth image, which indicates the proper performance of the proposed method.
4-2-2- Quantitative Investigation of the Proposed GOA-ISR Method Performance
In this experiment, 20 images from the DVI2K database were used to check the quantitative performance of the proposed GOA-ISR method, and the results are shown in Table 2.
Table 2. Quantitative results of SR methods
IMAGE | Sparse Neighbor Embedding[46] | Adaptive Sparse[47] | GWO[19] | SAN[48] | DWSR[53] | Proposed GOA-ISR method |
PSNR(dB) SSIM | PSNR(dB) SSIM | PSNR(dB) SSIM | PSNR(dB) SSIM | PSNR(dB) SSIM | PSNR(dB) SSIM | |
Image 1 | 19.44 0.691 | 26.37 0.821 | 27.31 0.801 | 31.10 0.890 | 32.20 0.897 | 34.72 0.945 |
Image 2 | 17.84 0.704 | 20.24 0.685 | 23.40 0.806 | 26.91 0.831 | 28.37 0.871 | 32.56 0.923 |
Image 3 | 21.52 0.715 | 26.11 0.847 | 25.16 0.812 | 28.53 0.834 | 26.08 0.786 | 31.87 0.909 |
Image 4 | 17.81 0.702 | 24.71 0.834 | 24.02 0.791 | 29.09 0.856 | 33.11 0.905 | 34.39 0.934 |
Image 5 | 16.61 0.566 | 21.42 0.739 | 21.75 0.725 | 27.30 0.871 | 32.42 0.930 | 34.33 0.935 |
Image 6 | 19.89 0.722 | 26.89 0.813 | 25.63 0.835 | 28.38 0.835 | 32.24 0.896 | 35.32 0.940 |
Image 7 | 18.78 0.706 | 22.87 0.835 | 24.74 0.829 | 27.75 0.845 | 32.84 0.891 | 34.96 0.925 |
Image 8 | 20.81 0.714 | 23.30 0.833 | 25.72 0.823 | 28.24 0.847 | 29.75 0.90 | 33.23 0.926 |
Image 9 | 21.63 0.736 | 27.73 0.848 | 28.45 0.837 | 31.13 0.897 | 32.48 0.92 | 33.0 5 0.933 |
Image 10 | 20.90 0.7303 | 23.45 0.821 | 27.23 0.849 | 28.49 0.869 | 31.54 0.88 | 33.89 0.939 |
Image 11 | 17.28 0.651 | 22.83 0.811 | 22.78 0.786 | 28.45 0.837 | 32.23 0.896 | 34.40 0.929 |
Image 12 | 16.58 0.645 | 22.61 0.810 | 21.94 0.772 | 27.86 0.842 | 32.02 0.893 | 33.89 0.931 |
Image 13 | 19.37 0.716 | 25.17 0.806 | 25.12 0.830 | 28.05 0.835 | 28.74 0.857 | 31.74 0.913 |
Image 14 | 20.48 0.675 | 26.09 0.830 | 25.84 0.798 | 21.74 0.690 | 29.17 0.843 | 31.45 0.917 |
Image 15 | 26.00 0.702 | 25.96 0.667 | 23.14 0.657 | 24.52 0.722 | 31.35 0.883 | 32.49 0.931 |
Image 16 | 20.87 0.628 | 24.23 0.821 | 22.31 0.812 | 23.61 0.718 | 30.69 0.910 | 32.83 0.925 |
Image 17 | 21.20 0.723 | 27.24 0.850 | 26.84 0.855 | 28.06 0.846 | 29.09 0.889 | 33.60 0.938 |
Image 18 | 18.02 0.691 | 20.31 0.791 | 22.34 0.762 | 25.03 0.763 | 30.38 0.786 | 32.19 0.873 |
Image 19 | 19.31 0.701 | 24.03 0.892 | 23.62 0.780 | 26.82 0.820 | 28.01 0.871 | 29.34 0.908 |
Image 20 | 17.98 0.618 | 23.79 0.810 | 24.05 0.742 | 28.49 0.858 | 29.21 0.861 | 34.38 0.943 |
As seen in Table 2, the sparse neighbor embedding method has a weaker performance than the other methods, but the GWO and SAN methods have an acceptable performance in the SR process. The quantitative evaluation results show that the performance of the proposed GOA-ISR method is better than other methods, such as the SAN method.
4-2-3- Investigating Other Meta-Heuristic Algorithms in the SR Process
In this test, other meta-heuristic algorithms, such as Particle Swarm Optimization (PSO) and Gravitational Search Algorithm (GSA), and Firefly Algorithm (FA) are used to recover the weight value in the SR process. The DVI2K database is used in this study.
Table 3. The results of the SR process on meta-heuristic algorithms
Algorithms | PSNR(dB) |
PSO | 16.23 |
GSA | 18.71 |
FA | 19.08 |
GOA | 22.43 |
Proposed GOA-ISR Method | 24.19 |
As seen in Table 3, the value of PSNR in GOA and the proposed GOA-ISR methods are better than that in other meta-heuristic algorithms, such as PSO, GSA, and FA. The effective and better performance of this algorithm has been proven against other meta-heuristic algorithms [54]. The reason that the performance of the proposed GOA-ISR method is better than the classic GOA is that the search space in the proposed GOA-ISR method is limited, and the local optimum is prevented from getting stuck by providing upper bound and lower bound formulas.
4-2-4- Checking the Proposed GOA-ISR Method on Other Databases
In this experiment, URBAN100 [55], BSD100[56], Set 14 [57] and Set 5 [28]databases were used to evaluate the performance of the proposed GOA-ISR method, and the results are shown in Fig 4 and Table 4.
Table 4. Performance evaluation of the proposed GOA-ISR method on different databases
Database | Average PSNR |
26.43 | |
27.65 | |
31.24 | |
Set 5 | 30.09 |
(a) |
(b) |
(c) |
(d) |
Fig. 4. The results of the SR process, (a)(c) HR image ,(b)(d) Proposed GOA-ISR method
The images of the proposed GOA-ISR method are very close to the HR images, which indicates the proper performance of the proposed GOA-ISR method (Fig. 4).
5- Conclusion
In this study, a new approach based on the GOA is proposed to obtain optimal reconstruction weights in the ISR process. The suggested GOA-ISR obtains optimal reconstruction weights for training LR images, which leads to promising reconstruction of HR images. In this approach, the distances between the training patches and input are calculated, so the performance of the SR process can be better compared with the classical GOA algorithm. In the future studies, it will be tried to achieve the maximum improvement of the SR process through using a suitable method to optimize the fixed parameters of the algorithm, that is, instead of the trial and error method, the parameters of the algorithm should be obtained systematically.
References
[1] B. Ghaderi and H. Azad, "Deep Learning Algorithms in Super-Resolution Images," Journal of Circuits, Data and Systems Analysis, vol. 1, no. 1, p. 47, 2023.
[2] P. Behjati, P. Rodriguez, C. Fernández, I. Hupont, A. Mehri, and J. Gonzàlez, "Single image super-resolution based on directional variance attention network," Pattern Recognition, vol. 133, p. 108997, 2023.
[3] T. Goto, T. Fukuoka, F. Nagashima, S. Hirano, and M. Sakurai, "Super-resolution System for 4K-HDTV," in 2014 22nd International Conference on Pattern Recognition, 2014: IEEE, pp. 4453-4458.
[4] A. Rapuano, G. Iovane, and M. Chinnici, "A scalable Blockchain based system for super resolution images manipulation," in 2020 IEEE 6th International Conference on Dependability in Sensor, Cloud and Big Data Systems and Application (DependSys), 2020: IEEE, pp. 8-15.
[5] H. Dastmalchi and H. Aghaeinia, "Super-resolution of very low-resolution face images with a wavelet integrated, identity preserving, adversarial network," Signal Processing: Image Communication, p. 116755, 2022.
[6] I. Taghavi et al., "Ultrasound super-resolution imaging with a hierarchical Kalman tracker," Ultrasonics, vol. 122, p. 106695, 2022.
[7] P. Wang, B. Bayram, and E. Sertel, "A comprehensive review on deep learning based remote sensing image super-resolution methods," Earth-Science Reviews, p. 104110, 2022.
[8] K. Zhu, H. Guo, S. Li, and X. Lin, "Online tool wear monitoring by super-resolution based machine vision," Computers in Industry, vol. 144, p. 103782, 2023.
[9] H. Hou and H. Andrews, "Cubic splines for image interpolation and digital filtering," IEEE Transactions on acoustics, speech, and signal processing, vol. 26, no. 6, pp. 508-517, 1978.
[10] M. Li and T. Q. Nguyen, "Markov random field model-based edge-directed image interpolation," IEEE Transactions on Image Processing, vol. 17, no. 7, pp. 1121-1128, 2008.
[11] J. Sun, J. Zhu, and M. F. Tappen, "Context-constrained hallucination for image super-resolution," in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010: IEEE, pp. 231-238.
[12] L. Wang, S. Xiang, G. Meng, H. Wu, and C. Pan, "Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 8, pp. 1289-1299, 2013.
[13] W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based super-resolution," IEEE Computer graphics and Applications, vol. 22, no. 2, pp. 56-65, 2002.
[14] C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in European conference on computer vision, 2014: Springer, pp. 184-199.
[15] K. Zhang, J. Li, H. Wang, X. Liu, and X. Gao, "Learning local dictionaries and similarity structures for single image super-resolution," Signal Processing, vol. 142, pp. 231-243, 2018.
[16] N. Kumar and A. Sethi, "Fast learning-based single image super-resolution," IEEE Transactions on Multimedia, vol. 18, no. 8, pp. 1504-1515, 2016.
[17] J. Jiang, C. Wang, X. Liu, and J. Ma, "Deep learning-based face super-resolution: A survey," ACM Computing Surveys (CSUR), vol. 55, no. 1, pp. 1-36, 2021.
[18] P. P. Gajjar and M. V. Joshi, "New learning based super-resolution: use of DWT and IGMRF prior," IEEE Transactions on Image Processing, vol. 19, no. 5, pp. 1201-1213, 2010.
[19] S. S. Rajput, V. K. Bohat, and K. Arya, "Grey wolf optimization algorithm for facial image super-resolution," Applied Intelligence, vol. 49, no. 4, pp. 1324-1338, 2019.
[20] K. Nguyen, C. Fookes, S. Sridharan, M. Tistarelli, and M. Nixon, "Super-resolution for biometrics: A comprehensive survey," Pattern Recognition, vol. 78, pp. 23-42, 2018.
[21] N. Wang, D. Tao, X. Gao, X. Li, and J. Li, "A comprehensive survey to face hallucination," International journal of computer vision, vol. 106, no. 1, pp. 9-30, 2014.
[22] Y. Tang, P. Yan, Y. Yuan, and X. Li, "Single-image super-resolution via local learning," International Journal of Machine Learning and Cybernetics, vol. 2, no. 1, pp. 15-23, 2011.
[23] K. Zhang, X. Gao, D. Tao, and X. Li, "Single image super-resolution with non-local means and steering kernel regression," IEEE Transactions on Image Processing, vol. 21, no. 11, pp. 4544-4556, 2012.
[24] K. Zhang, X. Gao, D. Tao, and X. Li, "Single image super-resolution with multiscale similarity learning," IEEE transactions on neural networks and learning systems, vol. 24, no. 10, pp. 1648-1659, 2013.
[25] Z. Wang, Y. Yang, Z. Wang, S. Chang, J. Yang, and T. S. Huang, "Learning super-resolution jointly from external and internal examples," IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4359-4371, 2015.
[26] X. Lu, H. Yuan, Y. Yuan, P. Yan, L. Li, and X. Li, "Local learning-based image super-resolution," in 2011 IEEE 13th International Workshop on Multimedia Signal Processing, 2011: IEEE, pp. 1-5.
[27] L. An and B. Bhanu, "Image super-resolution by extreme learning machine," in 2012 19th IEEE international conference on image processing, 2012: IEEE, pp. 2209-2212.
[28] M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding," 2012.
[29] S. Saremi, S. Mirjalili, and A. Lewis, "Grasshopper optimisation algorithm: theory and application," Advances in engineering software, vol. 105, pp. 30-47, 2017.
[30] I. J. Cox, M. L. Miller, J. A. Bloom, and C. Honsinger, Digital watermarking. Springer, 2002.
[31] P. Tumuluru and B. Ravi, "GOA-based DBN: Grasshopper optimization algorithm-based deep belief neural networks for cancer classification," International Journal of Applied Engineering Research, vol. 12, no. 24, pp. 14218-14231, 2017.
[32] P.-H. Dinh, "A novel approach based on grasshopper optimization algorithm for medical image fusion," Expert Systems with Applications, vol. 171, p. 114576, 2021.
[33] J. H. Holland, "Genetic algorithms," Scientific american, vol. 267, no. 1, pp. 66-73, 1992.
[34] R. Eberhart and J. Kennedy, "A new optimizer using particle swarm theory," in MHS'95. Proceedings of the sixth international symposium on micro machine and human science, 1995: Ieee, pp. 39-43.
[35] A. Kamalinia and A. Ghaffari, "Hybrid task scheduling method for cloud computing by genetic and PSO algorithms," J. Inf. Syst. Telecommun, vol. 4, pp. 271-281, 2016.
[36] X.-S. Yang, "Firefly algorithms for multimodal optimization," in International symposium on stochastic algorithms, 2009: Springer, pp. 169-178.
[37] A. Mahmoodzadeh, H. Agahi, and M. Salehi, "Handwritten Digits Recognition Using an Ensemble Technique Based on the Firefly Algorithm," Journal of Information Systems and Telecommunication (JIST), vol. 3, no. 23, p. 136, 2019.
[38] X.-S. Yang, "A new metaheuristic bat-inspired algorithm," in Nature inspired cooperative strategies for optimization (NICSO 2010): Springer, 2010, pp. 65-74.
[39] E. Rashedi, E. Rashedi, and H. Nezamabadi-Pour, "A comprehensive survey on gravitational search algorithm," Swarm and evolutionary computation, vol. 41, pp. 141-158, 2018.
[40] M. Tourani, "Improvement of Firefly Algorithm using Particle Swarm Optimization and Gravitational Search Algorithm," Journal of Information Systems and Telecommunication (JIST), vol. 2, no. 34, p. 123, 2021.
[41] C. M. Topaz, A. J. Bernoff, S. Logan, and W. Toolson, "A model for rolling swarms of locusts," The European Physical Journal Special Topics, vol. 157, no. 1, pp. 93-109, 2008.
[42] S. M. Rogers, T. Matheson, E. Despland, T. Dodgson, M. Burrows, and S. J. Simpson, "Mechanosensory-induced behavioural gregarization in the desert locust Schistocerca gregaria," Journal of Experimental Biology, vol. 206, no. 22, pp. 3991-4002, 2003.
[43]Y. ChangH, "XiongY. Super-resolutionthroughneighborembedding," Proceedingsofthe2004IEEEComputer Society C o nference on ComputerVision and Pattern Rec ogni—tion, pp. 275-282, 2004.
[44]J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE transactions on image processing, vol. 19, no. 11, pp. 2861-2873, 2010.
[45]J. Gu, H. Lu, W. Zuo, and C. Dong, "Blind super-resolution with iterative kernel correction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1604-1613.
[46]X. Gao, K. Zhang, D. Tao, and X. Li, "Image super-resolution with sparse neighbor embedding," IEEE Transactions on Image Processing, vol. 21, no. 7, pp. 3194-3205, 2012.
[47]W. Dong, L. Zhang, G. Shi, and X. Wu, "Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization," IEEE Transactions on image processing, vol. 20, no. 7, pp. 1838-1857, 2011.
[48]T. Dai, J. Cai, Y. Zhang, S.-T. Xia, and L. Zhang, "Second-order attention network for single image super-resolution," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 11065-11074.
[49]E. Agustsson and R. Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 126-135.
[50]F. Akhlaghian Tab, K. Ghaderi, and P. Moradi, "A New Robust Digital Image Watermarking Algorithm Based on LWT-SVD and Fractal Images," Journal of Information Systems and Telecommunication (JIST), vol. 1, no. 9, p. 1, 2015.
[51]K. Li, S. Yang, R. Dong, X. Wang, and J. Huang, "Survey of single image super‐resolution reconstruction," IET Image Processing, vol. 14, no. 11, pp. 2273-2290, 2020.
[52]V. K. Bohat and K. Arya, "An effective gbest-guided gravitational search algorithm for real-parameter optimization and its application in training of feedforward neural networks," Knowledge-Based Systems, vol. 143, pp. 192-207, 2018.
[53] S.-C. Chu, Z.-C. Dou, J.-S. Pan, L. Kong, V. Snášel, and J. Watada, "DWSR: an architecture optimization framework for adaptive super-resolution neural networks based on meta-heuristics," Artificial Intelligence Review, vol. 57, no. 2, p. 23, 2024.
[54] Y. Meraihi, A. B. Gabis, S. Mirjalili, and A. Ramdane-Cherif, "Grasshopper optimization algorithm: theory, variants, and applications," IEEE Access, vol. 9, pp. 50001-50024, 2021.
[55] J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5197-5206.
[56] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proceedings eighth IEEE international conference on computer vision. ICCV 2001, 2001, vol. 2: IEEE, pp. 416-423.
[57] R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7, 2012: Springer, pp. 711-730.
