Spatial resolution enhancement is usually required in both astronomy and
earth observing remote sensing fields, especially in satellite images
taken with the aim of recognizing objects whose size approaches the
limiting spatial resolution scale. One approach to improve the spatial
resolution is to use longer focal ratios, which requires larger,
better-stabilized and more expensive orbital platforms. Another approach
is to use sensor chips with smaller pixel size and increased pixel
density. The later is technically difficult, it decreases the amount of
collected light by each pixel and increases shot and readout noise. At
present, the best cost/benefit ratio seems to be achieved when using
digital image processing techniques known as spatial resolution
enhancement, super-resolution image reconstruction or simply
super-resolution (SR) (Merino and Núñez 2007).
SR refers to the reconstruction methods that can be applied to obtain an image with higher spatial resolution through the use of lower-resolution (LR) image(s). SR techniques are closely related to the problems of image restoration and image interpolation. The purpose of image restoration is to recover a degraded image without changing the pixel density of the image. Based on similar theories as image restoration, SR can be considered as a second generation of image restoration techniques which also change pixel density. Image interpolation techniques can be used to increase the pixel density of an image. Therefore, SR image reconstruction techniques combine image restoration and interpolation to reconstruct one or a set of high-resolution (HR) images from LR image(s).
SR image reconstruction has widely been researched in the last two decades. Most of the researches have been carried out for combining multiple LR images of the same scene to reconstruct a single or more HR image(s) (e.g., Shen et al. 2009; Tsai and Huang 1984; Kim et al. 1990; Kim and Su 1993; Rhee and Kang 1999; Chan et al. 2003). The basic principle underlying most of the aforementioned techniques is to take multiple images from the same object under similar lighting conditions but from slightly different sensor locations or orientations. When two images provide different views of the same object or landscape, the motion or motion vector field (a set of displacements of the pixel grid points between the images) is used to keep the grid points tied to their corresponding fixed locations on the viewed surface. The true motion between the images is not known, and must therefore be approximated that is a complicated and difficult task (Horn 1986; Packalén et al. 2006). Furthermore, most of the aforementioned SR techniques are computationally expensive. Another major limitation of these techniques is perhaps requirement of multiple images that are often costly to procure and difficult to acquire at the same temporal instant, especially for remote sensing. Commonly, the improvement of spatial resolution of multi-frame SR algorithm always has to sacrifice the temporal resolution (Tsai and Huang 1984). Other limitations include non-suitability of LR images for SR reconstruction; the application of SR algorithms is possible only if the images are sub-pixel shifted.
Alternatively, many researchers tackled the image fusion problem of reconstructing an LR image using an HR image. A typical example is the use of panchromatic image for sharpening multi/hyper-spectral images (Wang et al. 2005; Ranchin et al. 2003; Gonzalez-Audicana et al. 2006; Joshi et al. 2006; Park and Kang 2004; Bhatta 2008). However, SR reconstruction and image fusion are different. Image fusion combines one or several LR images with one or more HR images in order to obtain a useful final image with better spatial resolution than LR image. Therefore, fusion methods require the use of at least one HR image and the spatial resolution of their results is limited by that HR pixel-size. In contrast, SR algorithms do not use any HR image; they only depend on LR image(s).
However, most of the researchers of the existing literatures believe that the quality of a single LR image is limited; and interpolation based on an under-sampled image does not allow recovering the lost high-frequency information. Hence single LR image can not be used for SR reconstruction and multiple observations of the same scene are needed. Typical single-frame SR construction techniques have been criticized widely as ‘image enhancement’ by means of image scaling, interpolation, zooming and enlargement (Chan et al. 2008). Despite the criticisms, these approaches are preferred where multi-frame techniques are not applicable or affordable.
Although several techniques for single-frame SR reconstruction have been demonstrated by several researchers, a few of them have addressed remote sensing imageries. Tao et al. (2006) have shown an effective point spread function for single-frame SR reconstruction. Jiji et al. (2004) have proposed a single-frame SR algorithm using a wavelet-based technique where the HR edge primitives are learned from the HR data set locally. Chang et al. (2004) have proposed a single-frame image SR method where the generation of the HR image depends simultaneously on multiple nearest neighbors in the training set in a way similar to the concept of locally linear embedding for manifold learning. This method requires fewer training examples than other learning-based SR methods. The SR method proposed by Begin and Ferrie (2004) is the extension of a Markov-based learning algorithm, capable of processing an LR image with unknown degradation parameters. A different method for enhancing the resolution of LR facial images using an error back projection method based on top-down learning is proposed by Park and Lee (2004). An image hallucination approach based on primal sketch priors is presented by Sun et al. 2003; where reconstruction constraint is also applied to further improve the quality of the hallucinated image. Jiji and Chaudhuri (2006) have demonstrated a single-frame image SR through contourlet learning. This process ensures capturing the HR edges from the training set given a LR observation, as well as captures the smoothness along contours. Other approaches include low level vision learning (Freeman et al. 2000), incorporating the distribution of pixel intensity derivatives (Tappen et al. 2003), pixel classification (Wu et al. 2004; Atkins et al. 1999), locally-adaptive zooming algorithm (Battiato et al. 2002), smart interpolation by anisotropic diffusion (Battiato et al. 2003); triangulation on pixel level (Su and Willis 2004; Yu et al. 2001), Subpixel edge localization (Jensen and Anastassiou 1995), and neural network for interpolation (Staelin 2003). Ouwerkerk (2006) has surveyed several single-frame SR techniques by looking at theoretical backgrounds and practical results.
References
Atkins, C.B., Bouman, C.A. and Allebach, J.P., 1999. Tree-based resolution synthesis. Proceedings of IEEE ICIP (1999), pp. 405–410.
Battiato, S., Gallo, G. and Stanco, F., 2002. A locally-adaptive zooming algorithm for digital images. Image Vision and Computing Journal, 20(11), 805–812.
Battiato, S., Gallo, G. and Stanco, F., 2003. Smart interpolation by anisotropic diffusion. Proceedings of 12th International Conference on Image Analysis and Processing, pp. 572–577.
Begin, I. and Ferrie, F.R., 2004. Blind super-resolution using a learning-based approach. Proceedings of 17th IEEE International Conference on Pattern Recognition (ICPR ’04), vol. 2, pp. 85–89, Cambridge, UK.
Bhatta, B., 2008. Remote Sensing and GIS. Oxford University Press, pp. 872.
Chan, J.C.W., Ma, J. and Canters, F., 2008. A comparison of superresolution reconstruction methods for multi-angle CHRIS/Proba images. Proceedings of the SPIE, Vol. 7109, available online at http://dx.doi.org/10.1117/12.800256
Chan, R., Chan, T., Shen, L. and Shen, Z., 2003. Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput., 24, 1408–1432.
Freeman, W.T., Pasztor, E.C. and Carmichael, O.T., 2000. Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.
Gonzalez-Audicana, M., Otazu, X., Fors, O. and Alvarez-Mozos, J., 2006. A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors. IEEE Trans. Geosci. Remote Sens., 44, 1683–1691.
Horn, B., 1986. Robot Vision, MIT Press, Cambridge, MA.
Jensen, K. and Anastassiou, D., 1995. Subpixel edge localization and the interpolation of still images. IEEE Transactions on Image Processing, 4(3), 285–295.
Jiji C.V. and Chaudhuri S., 2006. Single-Frame Image Super-resolution through Contourlet Learning. EURASIP Journal on Applied Signal Processing, DOI 10.1155/ASP/2006/73767
Jiji, C.V., Joshi, M.V. and Chaudhuri, S., 2004. Single-frame image super-resolution using learned wavelet coefficients. International Journal of Imaging Systems and Technology, 14(3), 105–112.
Joshi, M.V., Bruzzone, L. and Chaudhuri, S., 2006. A modelbased approach to multiresolution fusion in remotely sensed images. IEEE Trans. Geosci. Remote Sens., 44, 2549–2562.
Kim, S.P. and Su, W.Y., 1993. Recursive high-resolution reconstruction of blurred multiframe images. IEEE Trans. Image Process., 2, 534–539.
Kim, S.P., Bose, N.K. and Valenzuela, H.M., 1990. Recursive reconstruction of high-resolution image from noisy undersampled multiframes. IEEE Trans. Acoust. Speech Signal Process., 38, 1013–1027.
Merino, M.T. and Núñez, J., 2007. Super-Resolution of remotely sensed images using SRVPLR and SRASW, Proc. of IEEE International Conference on Geoscience and Remote Sensing Symposium, 23-28 July, pp. 4866-4869. Available online at http://ieeexplore.ieee.org/stamp/stamp. ... r=04423951.
Ouwerkerk, J.D., 2006. Image super-resolution survey. Image and Vision Computing, 24, 1039–1052.
Packalén, P., Tokola, T., Saastamoinen, J. and Maltamo, M., 2006. Use of a super-resolution method in interpretation of forests from multiple NOAA/AVHRR images. International Journal of Remote Sensing, 27(24), 5341–5357.
Park, J.H. and Kang, M.G., 2004. Spatially adaptive multi-resolution multispectral image fusion. International Journal of Remote Sensing, 25, 5491–5508.
Park, J.S. and Lee, S.W., 2004. Enhancing low-resolution facial images using error back-projection for human identification at a distance. Proceedings of 17th IEEE International Conference on Pattern Recognition (ICPR ’04), vol. 1, pp. 346–349, Cambridge, UK.
Ranchin, T., Aiazzi, B., Alparone, L., Baronti, S. and Wald, L., 2003. Image fusion—the ARSIS concept and some successful implementation schemes. ISPRS J. Photogramm. Remote Sens., 58, 4–18.
Rhee, S. and Kang, M.G., 1999. Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt. Eng., 38, 1348–1356.
Shen, H., Ng, M.K., Li, P., and Zhang, L., 2009. Super-Resolution Reconstruction Algorithm To MODIS Remote Sensing Images. The Computer Journal, 52(1), 90–100.
Staelin, C., Greig, D., Fischer, M. and Maurer, R., 2003. Neural network image scaling using spatial errors. HP Laboratories, Israel.
Su, D. and Willis, P., 2004. Image interpolation by pixel level data-dependent triangulation. Computer Graphics Forum, 23(2).
Sun, J., Zheng, N.N., Tao, H. and Shum, H.Y., 2003. Image hallucination with primal sketch priors. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’03), vol. 2, pp. II-729–II-736, Madison, Wis, USA.
Tao, S., Zhigao, Y., Liyu, L., and Qianqing, Q., 2006. Single-Frame Remote Sensing Image Superresolution Reconstruction with Sub-blocks Noniterative Scheme. Proc. of IEEE International Conference on Geoscience and Remote Sensing Symposium, 31 July – 4 August, pp. 1382–1385. Available online at http://ieeexplore.ieee.org/xpls/abs_all ... er=4241504.
Tappen, M.F., Russell, B.C. and Freeman, W.T., 2003. Exploiting the Sparse Derivative Prior for Super-Resolution and Image Demosaicing, 2003.
Tsai, R.Y. and Huang, T.S., 1984. Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process., 1, 317–339.
Wang, Z.J., Ziou, D., Armenakis, C., Li, D. and Li, Q.Q., 2005. A comparative analysis of image fusion methods. IEEE Trans. Geosci. Remote Sens., 43, 1391–1402.
Wu, J., Trivedi, M. and Rao, B., 2004. Resolution enhancement by AdaBoost. Proceedings of 17th IEEE International Conference on Pattern Recognition (ICPR ’04), vol. 4, pp. 893–896, Cambridge, UK.
Yu, X., Morse, B.S. and Sederberg, T.W., 2001. Image reconstruction using data dependent triangulation. IEEE Computer Graphics and Application, 21(3), 62–68.
SR refers to the reconstruction methods that can be applied to obtain an image with higher spatial resolution through the use of lower-resolution (LR) image(s). SR techniques are closely related to the problems of image restoration and image interpolation. The purpose of image restoration is to recover a degraded image without changing the pixel density of the image. Based on similar theories as image restoration, SR can be considered as a second generation of image restoration techniques which also change pixel density. Image interpolation techniques can be used to increase the pixel density of an image. Therefore, SR image reconstruction techniques combine image restoration and interpolation to reconstruct one or a set of high-resolution (HR) images from LR image(s).
SR image reconstruction has widely been researched in the last two decades. Most of the researches have been carried out for combining multiple LR images of the same scene to reconstruct a single or more HR image(s) (e.g., Shen et al. 2009; Tsai and Huang 1984; Kim et al. 1990; Kim and Su 1993; Rhee and Kang 1999; Chan et al. 2003). The basic principle underlying most of the aforementioned techniques is to take multiple images from the same object under similar lighting conditions but from slightly different sensor locations or orientations. When two images provide different views of the same object or landscape, the motion or motion vector field (a set of displacements of the pixel grid points between the images) is used to keep the grid points tied to their corresponding fixed locations on the viewed surface. The true motion between the images is not known, and must therefore be approximated that is a complicated and difficult task (Horn 1986; Packalén et al. 2006). Furthermore, most of the aforementioned SR techniques are computationally expensive. Another major limitation of these techniques is perhaps requirement of multiple images that are often costly to procure and difficult to acquire at the same temporal instant, especially for remote sensing. Commonly, the improvement of spatial resolution of multi-frame SR algorithm always has to sacrifice the temporal resolution (Tsai and Huang 1984). Other limitations include non-suitability of LR images for SR reconstruction; the application of SR algorithms is possible only if the images are sub-pixel shifted.
Alternatively, many researchers tackled the image fusion problem of reconstructing an LR image using an HR image. A typical example is the use of panchromatic image for sharpening multi/hyper-spectral images (Wang et al. 2005; Ranchin et al. 2003; Gonzalez-Audicana et al. 2006; Joshi et al. 2006; Park and Kang 2004; Bhatta 2008). However, SR reconstruction and image fusion are different. Image fusion combines one or several LR images with one or more HR images in order to obtain a useful final image with better spatial resolution than LR image. Therefore, fusion methods require the use of at least one HR image and the spatial resolution of their results is limited by that HR pixel-size. In contrast, SR algorithms do not use any HR image; they only depend on LR image(s).
However, most of the researchers of the existing literatures believe that the quality of a single LR image is limited; and interpolation based on an under-sampled image does not allow recovering the lost high-frequency information. Hence single LR image can not be used for SR reconstruction and multiple observations of the same scene are needed. Typical single-frame SR construction techniques have been criticized widely as ‘image enhancement’ by means of image scaling, interpolation, zooming and enlargement (Chan et al. 2008). Despite the criticisms, these approaches are preferred where multi-frame techniques are not applicable or affordable.
Although several techniques for single-frame SR reconstruction have been demonstrated by several researchers, a few of them have addressed remote sensing imageries. Tao et al. (2006) have shown an effective point spread function for single-frame SR reconstruction. Jiji et al. (2004) have proposed a single-frame SR algorithm using a wavelet-based technique where the HR edge primitives are learned from the HR data set locally. Chang et al. (2004) have proposed a single-frame image SR method where the generation of the HR image depends simultaneously on multiple nearest neighbors in the training set in a way similar to the concept of locally linear embedding for manifold learning. This method requires fewer training examples than other learning-based SR methods. The SR method proposed by Begin and Ferrie (2004) is the extension of a Markov-based learning algorithm, capable of processing an LR image with unknown degradation parameters. A different method for enhancing the resolution of LR facial images using an error back projection method based on top-down learning is proposed by Park and Lee (2004). An image hallucination approach based on primal sketch priors is presented by Sun et al. 2003; where reconstruction constraint is also applied to further improve the quality of the hallucinated image. Jiji and Chaudhuri (2006) have demonstrated a single-frame image SR through contourlet learning. This process ensures capturing the HR edges from the training set given a LR observation, as well as captures the smoothness along contours. Other approaches include low level vision learning (Freeman et al. 2000), incorporating the distribution of pixel intensity derivatives (Tappen et al. 2003), pixel classification (Wu et al. 2004; Atkins et al. 1999), locally-adaptive zooming algorithm (Battiato et al. 2002), smart interpolation by anisotropic diffusion (Battiato et al. 2003); triangulation on pixel level (Su and Willis 2004; Yu et al. 2001), Subpixel edge localization (Jensen and Anastassiou 1995), and neural network for interpolation (Staelin 2003). Ouwerkerk (2006) has surveyed several single-frame SR techniques by looking at theoretical backgrounds and practical results.
References
Atkins, C.B., Bouman, C.A. and Allebach, J.P., 1999. Tree-based resolution synthesis. Proceedings of IEEE ICIP (1999), pp. 405–410.
Battiato, S., Gallo, G. and Stanco, F., 2002. A locally-adaptive zooming algorithm for digital images. Image Vision and Computing Journal, 20(11), 805–812.
Battiato, S., Gallo, G. and Stanco, F., 2003. Smart interpolation by anisotropic diffusion. Proceedings of 12th International Conference on Image Analysis and Processing, pp. 572–577.
Begin, I. and Ferrie, F.R., 2004. Blind super-resolution using a learning-based approach. Proceedings of 17th IEEE International Conference on Pattern Recognition (ICPR ’04), vol. 2, pp. 85–89, Cambridge, UK.
Bhatta, B., 2008. Remote Sensing and GIS. Oxford University Press, pp. 872.
Chan, J.C.W., Ma, J. and Canters, F., 2008. A comparison of superresolution reconstruction methods for multi-angle CHRIS/Proba images. Proceedings of the SPIE, Vol. 7109, available online at http://dx.doi.org/10.1117/12.800256
Chan, R., Chan, T., Shen, L. and Shen, Z., 2003. Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput., 24, 1408–1432.
Freeman, W.T., Pasztor, E.C. and Carmichael, O.T., 2000. Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.
Gonzalez-Audicana, M., Otazu, X., Fors, O. and Alvarez-Mozos, J., 2006. A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors. IEEE Trans. Geosci. Remote Sens., 44, 1683–1691.
Horn, B., 1986. Robot Vision, MIT Press, Cambridge, MA.
Jensen, K. and Anastassiou, D., 1995. Subpixel edge localization and the interpolation of still images. IEEE Transactions on Image Processing, 4(3), 285–295.
Jiji C.V. and Chaudhuri S., 2006. Single-Frame Image Super-resolution through Contourlet Learning. EURASIP Journal on Applied Signal Processing, DOI 10.1155/ASP/2006/73767
Jiji, C.V., Joshi, M.V. and Chaudhuri, S., 2004. Single-frame image super-resolution using learned wavelet coefficients. International Journal of Imaging Systems and Technology, 14(3), 105–112.
Joshi, M.V., Bruzzone, L. and Chaudhuri, S., 2006. A modelbased approach to multiresolution fusion in remotely sensed images. IEEE Trans. Geosci. Remote Sens., 44, 2549–2562.
Kim, S.P. and Su, W.Y., 1993. Recursive high-resolution reconstruction of blurred multiframe images. IEEE Trans. Image Process., 2, 534–539.
Kim, S.P., Bose, N.K. and Valenzuela, H.M., 1990. Recursive reconstruction of high-resolution image from noisy undersampled multiframes. IEEE Trans. Acoust. Speech Signal Process., 38, 1013–1027.
Merino, M.T. and Núñez, J., 2007. Super-Resolution of remotely sensed images using SRVPLR and SRASW, Proc. of IEEE International Conference on Geoscience and Remote Sensing Symposium, 23-28 July, pp. 4866-4869. Available online at http://ieeexplore.ieee.org/stamp/stamp. ... r=04423951.
Ouwerkerk, J.D., 2006. Image super-resolution survey. Image and Vision Computing, 24, 1039–1052.
Packalén, P., Tokola, T., Saastamoinen, J. and Maltamo, M., 2006. Use of a super-resolution method in interpretation of forests from multiple NOAA/AVHRR images. International Journal of Remote Sensing, 27(24), 5341–5357.
Park, J.H. and Kang, M.G., 2004. Spatially adaptive multi-resolution multispectral image fusion. International Journal of Remote Sensing, 25, 5491–5508.
Park, J.S. and Lee, S.W., 2004. Enhancing low-resolution facial images using error back-projection for human identification at a distance. Proceedings of 17th IEEE International Conference on Pattern Recognition (ICPR ’04), vol. 1, pp. 346–349, Cambridge, UK.
Ranchin, T., Aiazzi, B., Alparone, L., Baronti, S. and Wald, L., 2003. Image fusion—the ARSIS concept and some successful implementation schemes. ISPRS J. Photogramm. Remote Sens., 58, 4–18.
Rhee, S. and Kang, M.G., 1999. Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt. Eng., 38, 1348–1356.
Shen, H., Ng, M.K., Li, P., and Zhang, L., 2009. Super-Resolution Reconstruction Algorithm To MODIS Remote Sensing Images. The Computer Journal, 52(1), 90–100.
Staelin, C., Greig, D., Fischer, M. and Maurer, R., 2003. Neural network image scaling using spatial errors. HP Laboratories, Israel.
Su, D. and Willis, P., 2004. Image interpolation by pixel level data-dependent triangulation. Computer Graphics Forum, 23(2).
Sun, J., Zheng, N.N., Tao, H. and Shum, H.Y., 2003. Image hallucination with primal sketch priors. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’03), vol. 2, pp. II-729–II-736, Madison, Wis, USA.
Tao, S., Zhigao, Y., Liyu, L., and Qianqing, Q., 2006. Single-Frame Remote Sensing Image Superresolution Reconstruction with Sub-blocks Noniterative Scheme. Proc. of IEEE International Conference on Geoscience and Remote Sensing Symposium, 31 July – 4 August, pp. 1382–1385. Available online at http://ieeexplore.ieee.org/xpls/abs_all ... er=4241504.
Tappen, M.F., Russell, B.C. and Freeman, W.T., 2003. Exploiting the Sparse Derivative Prior for Super-Resolution and Image Demosaicing, 2003.
Tsai, R.Y. and Huang, T.S., 1984. Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process., 1, 317–339.
Wang, Z.J., Ziou, D., Armenakis, C., Li, D. and Li, Q.Q., 2005. A comparative analysis of image fusion methods. IEEE Trans. Geosci. Remote Sens., 43, 1391–1402.
Wu, J., Trivedi, M. and Rao, B., 2004. Resolution enhancement by AdaBoost. Proceedings of 17th IEEE International Conference on Pattern Recognition (ICPR ’04), vol. 4, pp. 893–896, Cambridge, UK.
Yu, X., Morse, B.S. and Sederberg, T.W., 2001. Image reconstruction using data dependent triangulation. IEEE Computer Graphics and Application, 21(3), 62–68.
No comments:
Post a Comment