Diffusion-Based Multimodal Remote Sensing Segmentation for Land-Cover Mapping and Environmental Monitoring

Authors

  • Rosaura Espejo Department of Computer Science, George Mason University, Fairfax, VA, USA. Author
  • Birgitt Klotz Department of Electrical Engineering and Computer Science, University of Kansas, Lawrence, KS, USA. Author
  • Matthijs van Engelen Department of Computer Science, University of Alabama at Birmingham, Birmingham, AL, USA. Author
  • Marjo Muhonen Department of Computer Science, University of North Texas, Denton, TX, USA. Author

Keywords:

diffusion models, multimodal remote sensing, semantic segmentation, land-cover mapping, environmental monitoring, geospatial governance, robustness, sustainability

Abstract

Land-cover mapping and environmental monitoring increasingly depend on the integration of heterogeneous remote sensing modalities, including optical imagery, synthetic aperture radar, hyperspectral cubes, thermal bands, and light detection and ranging products. Although deep learning has substantially advanced remote sensing segmentation, conventional discriminative models often struggle with missing modalities, domain shift, class imbalance, and the need for calibrated uncertainty in operational decision-making. Diffusion-based generative models offer a promising alternative because they learn complex data distributions through iterative denoising and can be conditioned on multiple sensor streams, semantic prompts, and temporal context. This paper examines diffusion-based multimodal remote sensing segmentation as a socio-technical system rather than a narrow model architecture. It analyzes fusion strategies, deployment infrastructure, robustness, fairness, governance, and sustainability implications for land-cover mapping and environmental monitoring. The discussion emphasizes structural trade-offs among accuracy, computational cost, latency, interpretability, and institutional trust. It also considers how cloud-edge orchestration, data provenance, model lifecycle management, and policy frameworks shape whether diffusion segmentation can move from benchmark performance to reliable public and private environmental intelligence. The paper concludes that diffusion-based multimodal segmentation is best understood as an infrastructural capability whose value depends on careful alignment among sensing systems, computational resources, regulatory expectations, and the communities affected by land-cover decisions.

References

1. Zhu, X. X., Tuia, D., Mou, L., Xia, G. S., Zhang, L., Xu, F., & Fraundorfer, F. (2017). Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8-36. https://doi.org/10.1109/MGRS.2017.2762307

2. Tuia, D., Persello, C., & Bruzzone, L. (2016). Domain adaptation for the classification of remote sensing data: An overview of recent advances. IEEE Geoscience and Remote Sensing Magazine, 4(2), 41-57. https://doi.org/10.1109/MGRS.2016.2548504

3. Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., & Chanussot, J. (2021). Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(7), 5966-5978. https://doi.org/10.1109/TGRS.2020.3015157

4. Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022). Deep learning in multimodal remote sensing data fusion: A comprehensive review. International Journal of Applied Earth Observation and Geoinformation, 112, 102926. https://doi.org/10.1016/j.jag.2022.102926

5. Schmitt, M., & Zhu, X. X. (2016). Data fusion and remote sensing: An ever-growing relationship. IEEE Geoscience and Remote Sensing Magazine, 4(4), 6-23. https://doi.org/10.1109/MGRS.2016.2561021

6. Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33, 6840-6851.

7. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations.

8. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10684-10695.

9. Dhariwal, P., & Nichol, A. (2021). Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 34, 8780-8794.

10. Chen, T., Li, L., Saxena, S., Hinton, G., & Fleet, D. J. (2023). A generalist framework for panoptic segmentation of images and videos. Proceedings of the IEEE/CVF International Conference on Computer Vision, 909-919.

11. Amit, T., Nachmani, E., Shaharbany, T., & Wolf, L. (2021). SegDiff: Image segmentation with diffusion probabilistic models. arXiv preprint arXiv:2112.00390.

12. Wolleb, J., Sandkühler, R., Bieder, F., Valmaggia, P., & Cattin, P. C. (2022). Diffusion models for medical anomaly detection. Medical Image Computing and Computer Assisted Intervention, 35-45.

13. Zhang, L., Zhang, L., & Du, B. (2016). Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2), 22-40. https://doi.org/10.1109/MGRS.2016.2540798

14. Wang Z, He D, Zhao L, et al. Difusionseg: Diffusion-driven semantic segmentation with multi-modal image fusion for enhanced perception[J]. Knowledge-Based Systems, 2025: 114481.

15. Peebles, W., & Xie, S. (2023). Scalable diffusion models with transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, 4195-4205.

16. Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., & Kreis, K. (2023). Align your latents: High-resolution video synthesis with latent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22563-22575.

17. Li, Y., Zhang, H., Xue, X., Jiang, Y., & Shen, Q. (2018). Deep learning for remote sensing image classification: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(6), e1264. https://doi.org/10.1002/widm.1264

18. Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019). Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152, 166-177. https://doi.org/10.1016/j.isprsjprs.2019.04.015

19. Persello, C., Wegner, J. D., Hänsch, R., Tuia, D., Ghamisi, P., Kooistra, L., & Herold, M. (2022). Deep learning and earth observation to support the sustainable development goals. IEEE Geoscience and Remote Sensing Magazine, 10(2), 176-198. https://doi.org/10.1109/MGRS.2021.3136100

20. Ghamisi, P., Maggiori, E., Li, S., Souza, R., Tarablaka, Y., Moser, G., De Giorgi, A., Fang, L., Chen, Y., Chi, M., Serpico, S. B., & Benediktsson, J. A. (2018). New frontiers in spectral-spatial hyperspectral image classification: The latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning. IEEE Geoscience and Remote Sensing Magazine, 6(3), 10-43. https://doi.org/10.1109/MGRS.2018.2854840

Downloads

Published

2026-09-06