HSI, UHR, and LiDAR Data Fusion for the Urban Environment Characterization
Article Sidebar
Main Article Content
Abstract
The study of the urban environment is undoubtedly the key to moving towards sustainable transformations. However, remotely sensed observations within such domain are complex and challenging, as these areas present many similar spectral characteristics, making image analysis of urban areas a difficult task. Although sensors systems have been recently improved, they are alone still unable to attain a sufficient level of detail to qualitatively and quantitatively analyze targets of interest in an urban image. In this sense, multisource data fusion emerges as a feasible solution for detailed detection and interpretation of elements that compose an urban scene. This work aims to perform data fusion using a hyperspectral image (HSI), an optical RGB ultra-high-resolution image, and Light Detection and Ranging (LiDAR) data for a detailed characterization of an urban environment under the perspective of land cover. Seven datasets will be employed, including the separate RGB, HSI, and LiDAR data as well as their fusion. The latter one is used to demonstrate the potential of integrating information from manifold sensors when compared with the accuracy results of a unique sensor. The algorithm chosen to perform such classifications is Random Forest since it can handle large amounts of data and achieve satisfactory accuracy. The overall accuracy reached by the data fusion set shows to be significantly superior to the ones obtained by the other datasets, demonstrating that the combined use of multisource data refines the classification results, allowing for an accurate and detailed level of classification legend.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish in this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors can enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) before and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see "The Effect of Open Access").
How to Cite
References
Ali, A., Martelli, R., Lupia, F., & Barbanti, L. (2019). Assessing Multiple Years’ Spatial Variability of Crop Yields Using Satellite Vegetation Indices. Remote Sensing, 11 (20), 2384–2406. https://doi.org/10.3390/rs11202384.
Anjos, C. S., Almeida, C. M., & Galvão, L. S (2017a). Classificação por árvores de decisão: Avaliação de acurácia com e sem a pré-seleção de atributos. In: Simpósio Brasileiro de Sensoriamento Remoto, Santos. http://marte2.sid.inpe.br/col/sid.inpe.br/marte2/2017/10.27.15.47/doc/thisInformationItemHomePage.html.
Anjos, C. S., Almeida, C. M., Galvão, L. S., Souza-Filho, C. R., Lacerda, M.G., & Prati, R.C. (2017b). Analysis of the level of detail in classifications of urban areas with optical VHR and hyperspectral images using C4.5 decison tree and random forest methods. Bulletin of Geodetic Sciences, 2 (23), 371−388. https://dx.doi.org/10.1590/S1982-21702017000200024.
Anjos, C. S., Lacerda, M. G., Andrade, L., Salles R. N., & Almeida, C. M. (2019). Fusão de Dados: Nova Tendência em Processamento Digital de Imagens. In: Simpósio Brasileiro De Sensoriamento Remoto, Santos. https://proceedings.science/sbsr-2019/trabalhos/fusao-de-dados-nova-tendencia-em-processamento-digital-de-imagens?lang=pt-br.
Anjos, C. S. (2016) Classificação de áreas urbanas com imagens multiespectrais e hiperespectrais utilizando métodos não paramétricos. [Tese de doutorado, Instituto Nacional de Pesquisas Espaciais]. Repositório digital de teses e dissertações INPE. http://mtc-m21b.sid.inpe.br/col/sid.inpe.br/mtc-m21b/2016/04.04.19.03/doc/publicacao.pdf.
Baatz, M., & Schäpe, A. (2000). Multiresolution Segmentation–an optimization approach for high quality multi-scale image segmentation. Angewandte Geographische Informations-Verarbeitung XII. 12, 12–23. https://www.semanticscholar.org/paper/Multiresolution-Segmentation-%3A-an-optimization-for-Baatz-Sch%C3%A4pe/364cc1ff514a2e11d21a101dc072575e5487d17e.
Blaschke, T. (2010). Object based image analysis for remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 65 (1), 2–16. https://doi.org/10.1016/j.isprsjprs.2009.06.004.
Congalton, R. G. (1988). A comparison of sampling schemes used in generating error matrices for assessing the accuracy of maps generated from remotely sensed data. Photogrammetric Engineering and Remote Sensing, 54 (5), 593–600. https://www.asprs.org/wp-content/uploads/pers/1988journal/may/1988_may_593-600.pdf.
Congalton, R. G. & Green, K. (2009). Assessing the accuracy of remotely sensed data: principles and practices. Boca Raton. https://doi.org/10.1201/9781420055139.
Costa, J.D.S., Liesenberg, V., Schimalski, M.B., Sousa, R.V.D., Biffi, L.J., Gomes, A.R., Neto, S.L.R., Mitishita, E., & Bispo, P.D.C. (2021). Benefits of Combining ALOS/PALSAR-2 and Sentinel-2A Data in the Classification of Land Cover Classes in the Santa Catarina Southern Plateau. Remote Sensing, 13 (2), 229–261. https://doi.org/10.3390/rs13020229.
Definiens. Definiens Documentation: Developer XD 2.0.4. (2012) Definiens AG. https://www.imperial.ac.uk/media/imperial-college/medicine/facilities/film/Definiens-Developer-Reference-Book-XD-2.0.4.pdf.
Durán, G., Filho, W. P., & Kuplich, T. M. (2018). Identificação espectral de materiais urbanos com a técnica mapeador de ângulo espectral (SAM) e o sensor de alta resolução espacial Geoeye-1. Geographic Bulletin of Rio Grande do Sul, (31), 9–34. https://revistas.planejamento.rs.gov.br/index.php/boletim-geografico-rs/article/view/4011.
Fan, Y., Ni, D., & Ma, H. (2017). HyperDB: A Hyperspectral Land Class Database Designed for an Image Processing System. IEEE Tsinghua Science and Technology, 22 (1), 112−118. https://doi.org/10.1109/TST.2017.7830901.
Fernandez-Diaz, J.C., Carter, W.E., Glennie, C., Shrestha, R.L., Pan, Z., Ekhtari, N., Singhania, A., Hauser, D., & Sartori, M. (2016). Capability Assessment and Performance Metrics for the Titan Multispectral Mapping Lidar. Remote Sensing, 8 (11), 936–968. https://doi.org/10.3390/rs8110936.
Freire, F. C. J. (2016). Características fisiológicas de mudas de craibeira sob condições de deficiência hídrica. [Dissertação de Mestrado, Universidade Federal de Alagoas]. Repositório digital de teses e dissertações UFAL. https://www.repositorio.ufal.br/bitstream/riufal/1340/1/Caracter%C3%ADsticas%20fisiol%C3%B3gicas%20de%20mudas%20de%20craibeira%20sob%20condi%C3%A7%C3%B5es%20de%20defici%C3%AAncia%20h%C3%ADdrica.pdf.
Gao, D., Hu, Z., & Ye, R. (2018). Self-Dictionary Regression for Hyperspectral Image Super-Resolution. Remote Sensing, 10 (10), 1574–1596. https://doi.org/10.3390/rs10101574.
Ghamisi, P., Höfle, B., & Zhu, X.X. (2017). Hyperspectral and LiDAR Data Fusion Using Extinction Profiles and Deep Convolutional Neural Network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10 (6), 3011−3024. https://doi.org/10.1109/JSTARS.2016.2634863.
Gitelson, A.A., Kaufman, Y.J., & Merzlyak, M.N. (1996). Use of a green channel in remote sensing of global vegetation from EOS-MODIS. Remote Sensing Environment, 58 (3), 289–298. https://doi.org/10.1016/S0034-4257(96)00072-7.
Green, A. A., Berman, M., Switzer, P., & Craig, M. D. (1988). A transformation for ordering multispectral data in terms of image quality with implications for noise removal, IEEE Transactions on Geoscience and Remote Sensing, 26 (1), 65–74, 1988. https://ieeexplore.ieee.org/document/3001.
Happ, N. P., Ferreira, R. S., Bentes, C., Costa, G. A. O. P., & Feitosa, R. Q. (2009). Segmentação multiresolução: uma abordagem paralela para segmentação de imagens de alta resolução em arquiteturas de múltiplos núcleos. In: Proceedings of the XIV Brazilian Symposium of Remote Sensing, Natal. http://marte.sid.inpe.br/col/dpi.inpe.br/sbsr@80/2008/11.18.01.10.13/doc/6935-6942.pdf.
Hu, L., Robinson, C. & Dilkina, B. (2021). Model Generalization in Deep Learning Applications for Land Cover Mapping. In: Fragile Earth’21: KDD 2021 Workshop, New York. https://doi.org/10.48550/arXiv.2008.10351.
Huang, S., Tang, L., Hupy, J.P., Wang, Y., & Shao, G (2021). A commentary review on the use of normalized difference vegetation index (NDVI) in the era of popular remote sensing. Journal of Forestry Research, 32 (1), 1–6. https://doi.org/10.1007/s11676-020-01155-1.
Jensen, J.R., & Cowen, D.C. (1999). Remote Sensing of Urban/Suburban Infrastructure and Socio-Economic Attributes. In: The Map Reader: Theories of Mapping Practice and Cartographic Representation, Oxford, United Kingdom. https://www.asprs.org/wp-content/uploads/pers/99journal/may/1999_may_611-622.pdf.
Instituto Brasileiro de Geografia e Estatística (IBGE). (2017). Classificação e caracterização dos espaços rurais e urbanos do Brasil: uma primeira aproximação. IBGE. https://biblioteca.ibge.gov.br/visualizacao/livros/liv100643.pdf.
Kuchler, P. (2021). Utilização do sensoriamento remoto para o mapeamento dos sistemas integrados de produção agrícola: contribuição ao monitoramento da agricultura de baixa emissão de carbono no estado do Mato Grosso, Brasil. [Tese de doutorado, Universidade do Estado do Rio de Janeiro]. Repositório digital de teses e dissertações UERJ. http://www.alice.cnptia.embrapa.br/alice/handle/doc/1135585.
Kuras, A., Brell, M., Rizzi, J., & Burud, I. (2021). Hyperspectral and Lidar Data Applied to the Urban Land Cover Machine Learning and Neural-Network-Based Classification: A Review. Remote Sensing, 13 (17), 3393–3432. https://doi.org/10.3390/rs13173393.
Lacerda, M.G. (2020). Abordagem GEOBIA para Imagens VHR Obtidas por Aeronaves Remotamente Pilotadas e Sensores Satelitais com o Uso de Classicadores Individuais e Ensemble. [ Tese de doutorado, Instituto Tecnológico de Aeronáutica São José dos Campos]. Repositório digital de teses e dissertações ITA. https://www.researchgate.net/profile/Marielcio-Lacerda-2/publication/373514773_Abordagem_GEOBIA_para_imagens_VHR_obtidas_por_aeronaves_remotamente_pilotadas_e_sensores_satelitais_com_o_uso_de_classificadores_individuais_e_ensemble/links/64ef99c90f7ab20a8666f6d0/Abordagem-GEOBIA-para-imagens-VHR-obtidas-por-aeronaves-remotamente-pilotadas-e-sensores-satelitais-com-o-uso-de-classificadores-individuais-e-ensemble.pdf
Leonardi, F. (2010). Abordagens Cognitivas e Mineração de Dados Aplicadas a Dados Ópticos Orbitais e de Laser para a Classificação de Cobertura do Solo Urbano. [Dissertação de mestrado, Instituto Nacional de Pesquisas Espaciais]. Repositório digital de teses e dissertações INPE. http://mtc-m16d.sid.inpe.br/col/sid.inpe.br/mtc-m19@80/2010/03.17.11.42/doc/publicacao.pdf.
Li, J., Hong, D., Gao, L., Yao, J., Zhengda, K., Zhang, B., & Chanussot, J. (2022). Deep learning in multimodal remote sensing data fusion: A comprehensive review. ELSEVIER International Journal of Applied Earth Observation and Geoinformation, 112, 1–16,. https://doi.org/10.1016/j.jag.2022.102926.
Li, Z., Chen, B., Wu, S., Su, M., Chen, J. M., & Xu, B. (2024). Deep learning for urban land use category classification: A review and experimental assessment. Remote Sensing of Environment, 311 (114290). https://doi.org/10.1016/j.rse.2024.114290.
Lillesand, T. M., Kiefer, R. W., & Chipman, J. W. (2015). Remote sensing and image interpretation. Wiley and Sons. https://www.geokniga.org/bookfiles/geokniga-remote-sensing-and-image-interpretation.pdf.
Liu, S., Marinelli, D., Bruzzone, L., & Bovolo, F. (2019). A review of change detection in multitemporal hyperspectral images: Current techniques, applications, and challenges. IEEE Geoscience and Remote Sensing Magazine, 7 (2), 140–158. https://doi.org/10.1109/MGRS.2019.2898520.
Lopes, T. D., Goedtel, A., Palacios, R.H. C., & Godoy, W. F. (2017). Aplicação do Algoritmo Random Forest como Classificador de Padrões de Falhas em Rolamentos de Motores de Indução. In: XIII Brazilian Symposium on Intelligent Automation, Porto Alegre. https://www.sba.org.br/Proceedings/SBAI/SBAI2017/SBAI17/papers/paper_98.pdf.
Martins, R. C. (2017). Definição de zonas de manejo por índice de vegetação obtidos por sensoriamento remoto e mapas de produtividade. [Dissertação de mestrado, Universidade Federal de Santa Maria]. Repositório digital de teses e dissertações UFSM. https://repositorio.ufsm.br/bitstream/handle/1/11346/Martins%2c%20Raffael%20Chielle.pdf?sequence=1&isAllowed=y.
Matsuoka, J. V. (2007). Investigação do processo de segmentação multiresolução utilizando o critério de ponderação de formas e cores aplicadas às imagens de áreas urbanas de alta resolução espacial do satélite Ikonos. In: XIII Brazilian Symposium of Remote Sensing, Florianópolis. http://marte.sid.inpe.br/col/dpi.inpe.br/sbsr@80/2006/11.15.11.19/doc/589-596.pdf.
Motohka, T.; Nasahara, K.N.; Oguma, H.; & Tsuchida, S. (2010). Applicability of Green-Red Vegetation Index for Remote Sensing of Vegetation Phenology. Remote Sensing, 2 (10), 2369–2387. https://doi.org/10.3390/rs2102369.
Nistor, C., Vîrghileanu, M., Cârlan, I., Mihai, B.-A., Toma, L., & Olariu, B. (2021). Remote Sensing-Based Analysis of Urban Landscape Change in the City of Bucharest, Romania. Remote Sensing, 13 (12), 2323–2342. https://doi.org/10.3390/rs13122323.
Ouma, Y. O., Keitsilea, A., Nkwaea, B., Odirilea, P. Moalafhib, D., Qi, J. (2023). Urban land-use classification using machine learning classifiers: comparative evaluation and post-classification multi-feature fusion approach. European Journal Of Remote Sensing, 56 (1), 2–21. https://doi.org/10.1080/22797254.2023.2173659.
Pande, S.; Uzun, B.; Guiotte, F.; Corpetti, T.; Delerue, F.; Lefèvre, S. (2024). Plant detection from ultra high resolution remote sensing images: a semantic segmentation approach based on fuzzy loss. Computer Vision and Pattern Recognition. 1, 1–5. https://www.researchgate.net/publication/383700535_Plant_detection_from_ultra_high_resolution_remote_sensing_images_A_Semantic_Segmentation_approach_based_on_fuzzy_loss.
Panosso, A. R. (2019). Estatística e Bioestatística. (2019). Faculdade de Ciências Agrárias e Veterinárias (FCAV) – UNESP. https://www.fcav.unesp.br/Home/departamentos/cienciasexatas/alanrodrigopanosso/apostila_bioestatistica_2019.pdf.
Qiu, C., Mou, L., Schmitt, M., & Zhu, X. X. (2019). Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network. ISPRS Journal of Photogrammetry and Remote Sensing, 154, 151–162. https://doi.org/10.1016/j.isprsjprs.2019.05.004.
Rasti, B., Ghamisi, P., & Gloaguen, R. (2020). Fusion of Multispectral Lidar and Hyperspectral Imagery. In: 2020 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Waikoloa, United States of America. https://doi.org/10.1109/IGARSS39084.2020.9323179.
Richardson, A.J., & Wiegand, C.L. (1977). Distinguishing vegetation from soil background information. Photogrammetric Engineering & Remote Sensing, 43 (12), 1541–1552. https://www.asprs.org/wp-content/uploads/pers/1977journal/dec/1977_dec_1541-1552.pdf.
Rocha, B. O., Körting, T. S., & Namikawa, L. M. (2024). Processamento de Imagens dos Satélites Brasileiros CBERS-4 e CBERS-4A para Respostas Rápidas a Desastres. Rev. Bras. Cartogr, 76 (1), 1–17. http://dx.doi.org/10.14393/rbcv76n0a-70194.
Rondeaux, G., STEVEN, M., BARET, F. (1996). Optimization of soil-adjusted vegetation indices. Remote Sensing of Environment, 55 (2), 95–107. https://doi.org/10.1016/0034-4257(95)00186-7.
Rouse, J.W., Haas, R.H. Schell, J.A., & Deering, W.D. (1974). Monitoring vegetation systems in the Great Plains with ERTS. NASA Spec, 351, (1), 309–317. https://ntrs.nasa.gov/api/citations/19740022614/downloads/19740022614.pdf.
Siddiqui, A., Chauhan, P., Kumar, V., Jain, G., Deshmukh, A. & Kumar, P.(2020). Characterization of urban materials in AVIRISNG data using a mixture tuned matched filtering (MTMF) approach. Geocarto International, 37 (1), 332–347. https://doi.org/10.1080/10106049.2020.1720312.
Son, J.D., Niu, G., Yang, B.S., Hwang, D.H., & Kang, D.S.(2009). Development of smart sensors system for machine fault diagnosis. Expert Systems with Applications, 36 (9), 11981–11991. https://doi.org/10.1016/j.eswa.2009.03.069.
Stewart, I. D. & Oke, T. R. (2012). Local climate zones for urban temperature studies. Bulletin of the American Meteorological Society, 93 (12), 154–196,. https://doi.org/10.1175/BAMS-D-11-00019.1.
Tucker, C.J. (1979). Red and photographic infrared linear combinations for monitoring vegetation. Remote Sensing Environment, 8 (2), 127–150. https://doi.org/10.1016/0034-4257(79)90013-0.
United States Census Bureau (USCB). (2018). Cartographic Boundary Files – Shapefile. https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html.
Xu, Y., Du, B., Zhang, L., Cerra, D., Pato, M., Carmona, E., Prasad, S., Yokoya, N., Hänsch, R., & Le Saux, B. (2019). Advanced multi-sensor optical remote sensing for urban land use and land cover classification: Outcome of the 2018 IEEE GRSS Data Fusion Contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12 (6), 1709–1724. https://doi.org/10.1109/JSTARS.2019.2911113.
Yang, R., Luo, F., Ren, F., Huang,W., Li, Q., Du, K., & Yuan, D. (2022). Identifying UrbanWetlands through Remote Sensing Scene Classification Using Deep Learning: A Case Study of Shenzhen, China. ISPRS International Journal of Geo-Information, 11 (2), 131–148. https://doi.org/10.3390/ijgi11020131.
Yang, X. (2011). What is urban remote sensing?: Urban Remote Sensing: Monitoring, Synthesis, and Modeling in the Urban Environment. John Wiley and Sons.
Zhang, J. (2010). Multi-source remote sensing data fusion: status and trends. International Journal of Image and Data Fusion, 1 (1), 5−24. https://doi.org/10.1080/19479830903561035.
Zhou, F., & Zhong, D. (2020). Kalman filter method for generating time-series synthetic Landsat images and their uncertainty from Landsat and MODIS observations. Remote Sensing of Environment, 239 (1), 111628–111653. https://doi.org/10.1016/j.rse.2019.111628.