Skip to content

satellite-image-deep-learning/techniques

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

Deep learning has revolutionized the analysis and interpretation of satellite and aerial imagery, addressing unique challenges such as vast image sizes and a wide array of object classes. This repository provides an exhaustive overview of deep learning techniques specifically tailored for satellite and aerial image processing. It covers a range of architectures, models, and algorithms suited for key tasks like classification, segmentation, and object detection.

How to use this repository: use Command + F (Mac) or CTRL + F (Windows) to search this page for e.g. 'SAM'

Techniques

Classification


The UC merced dataset is a well known classification dataset.

Classification is a fundamental task in remote sensing data analysis, where the goal is to assign a semantic label to each image, such as 'urban', 'forest', 'agricultural land', etc. The process of assigning labels to an image is known as image-level classification. However, in some cases, a single image might contain multiple different land cover types, such as a forest with a river running through it, or a city with both residential and commercial areas. In these cases, image-level classification becomes more complex and involves assigning multiple labels to a single image. This can be accomplished using a combination of feature extraction and machine learning algorithms to accurately identify the different land cover types. It is important to note that image-level classification should not be confused with pixel-level classification, also known as semantic segmentation. While image-level classification assigns a single label to an entire image, semantic segmentation assigns a label to each individual pixel in an image, resulting in a highly detailed and accurate representation of the land cover types in an image. Read A brief introduction to satellite image classification with neural networks

Segmentation


(left) a satellite image and (right) the semantic classes in the image.

Image segmentation is a crucial step in image analysis and computer vision, with the goal of dividing an image into semantically meaningful segments or regions. The process of image segmentation assigns a class label to each pixel in an image, effectively transforming an image from a 2D grid of pixels into a 2D grid of pixels with assigned class labels. One common application of image segmentation is road or building segmentation, where the goal is to identify and separate roads and buildings from other features within an image. To accomplish this task, single class models are often trained to differentiate between roads and background, or buildings and background. These models are designed to recognize specific features, such as color, texture, and shape, that are characteristic of roads or buildings, and use this information to assign class labels to the pixels in an image. Another common application of image segmentation is land use or crop type classification, where the goal is to identify and map different land cover types within an image. In this case, multi-class models are typically used to recognize and differentiate between multiple classes within an image, such as forests, urban areas, and agricultural land. These models are capable of recognizing complex relationships between different land cover types, allowing for a more comprehensive understanding of the image content. Read A brief introduction to satellite image segmentation with neural networks. Note that many articles which refer to 'hyperspectral land classification' are often actually describing semantic segmentation.

Segmentation - Land use & land cover

  • Automatic Detection of Landfill Using Deep Learning

  • laika -> The goal of this repo is to research potential sources of satellite image data and to implement various algorithms for satellite image segmentation

  • CDL-Segmentation -> Deep Learning Based Land Cover and Crop Type Classification: A Comparative Study. Compares UNet, SegNet & DeepLabv3+

  • LoveDA -> A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

  • DeepGlobe Land Cover Classification Challenge solution

  • CNN_Enhanced_GCN -> CNN-Enhanced Graph Convolutional Network With Pixel- and Superpixel-Level Feature Fusion for Hyperspectral Image Classification

  • LULCMapping-WV3images-CORINE-DLMethods -> Land Use and Land Cover Mapping Using Deep Learning Based Segmentation Approaches and VHR Worldview-3 Images

  • MCANet -> A joint semantic segmentation framework of optical and SAR images for land use classification. Uses WHU-OPT-SAR-dataset

  • land-cover -> Model Generalization in Deep Learning Applications for Land Cover Mapping

  • generalizablersc -> Cross-dataset Learning for Generalizable Land Use Scene Classification

  • Large-scale-Automatic-Identification-of-Urban-Vacant-Land -> Large-scale automatic identification of urban vacant land using semantic segmentation of high-resolution remote sensing images

  • SSLTransformerRS -> Self-supervised Vision Transformers for Land-cover Segmentation and Classification

  • LULCMapping-WV3images-CORINE-DLMethods -> Land Use and Land Cover Mapping Using Deep Learning Based Segmentation Approaches and VHR Worldview-3 Images

  • DCSA-Net -> Dynamic Convolution Self-Attention Network for Land-Cover Classification in VHR Remote-Sensing Images

  • CHeGCN-CNN_enhanced_Heterogeneous_Graph -> CNN-Enhanced Heterogeneous Graph Convolutional Network: Inferring Land Use from Land Cover with a Case Study of Park Segmentation

  • TCSVT_2022_DGSSC -> DGSSC: A Deep Generative Spectral-Spatial Classifier for Imbalanced Hyperspectral Imagery

  • DeepForest-Wetland-Paper -> Deep Forest classifier for wetland mapping using the combination of Sentinel-1 and Sentinel-2 data, GIScience & Remote Sensing

  • Wetland_UNet -> UNet models that can delineate wetlands using remote sensing data input including bands from Sentinel-2 LiDAR and geomorphons. By the Conservation Innovation Center of Chesapeake Conservancy and Defenders of Wildlife

  • DPA -> DPA is an unsupervised domain adaptation (UDA) method applied to different satellite images for larg-scale land cover mapping.

  • dynamicworld -> Dynamic World, Near real-time global 10 m land use land cover mapping

  • spada -> Land Cover Segmentation with Sparse Annotations from Sentinel-2 Imagery

  • M3SPADA -> Multi-Sensor Temporal Unsupervised Domain Adaptation for Land Cover Mapping with spatial pseudo labelling and adversarial learning

  • GLNet -> Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images

  • LoveNAS -> LoveNAS: Towards Multi-Scene Land-Cover Mapping via Hierarchical Searching Adaptive Network

  • FLAIR-2 challenge -> Semantic segmentation and domain adaptation challenge proposed by the French National Institute of Geographical and Forest Information (IGN)

  • flair-2 8th place solution

  • igarss-spada -> Dataset and code for the paper Land Cover Segmentation with Sparse Annotations from Sentinel-2 Imagery IGARSS 2023

  • cnn-land-cover-eco -> Multi-stage semantic segmentation of land cover in the Peak District using high-resolution RGB aerial imagery

Segmentation - Vegetation, deforestation, crops & crop boundaries

Note that deforestation detection may be treated as a segmentation task or a change detection task

Segmentation - Water, coastlines, rivers & floods

  • Houston_flooding -> labeling each pixel as either flooded or not using data from Hurricane Harvey. Dataset consisted of pre and post flood images, and a ground truth floodwater mask was created using unsupervised clustering (with DBScan) of image pixels with human cluster verification/adjustment

  • ml4floods -> An ecosystem of data, models and code pipelines to tackle flooding with ML

  • 1st place solution for STAC Overflow: Map Floodwater from Radar Imagery hosted by Microsoft AI for Earth -> combines Unet with Catboostclassifier, taking their maxima, not the average

  • hydra-floods -> an open source Python application for downloading, processing, and delivering surface water maps derived from remote sensing data

  • CoastSat -> tool for mapping coastlines which has an extension CoastSeg using segmentation models

  • deepwatermap -> a deep model that segments water on multispectral images

  • rivamap -> an automated river analysis and mapping engine

  • deep-water -> track changes in water level

  • WatNet -> A deep ConvNet for surface water mapping based on Sentinel-2 image, uses the Earth Surface Water Dataset

  • A-U-Net-for-Flood-Extent-Mapping

  • floatingobjects -> TOWARDS DETECTING FLOATING OBJECTS ON A GLOBAL SCALE WITHLEARNED SPATIAL FEATURES USING SENTINEL 2. Uses U-Net & pytorch

  • SpaceNet8 -> baseline Unet solution to detect flooded roads and buildings

  • dlsim -> Breaking the Limits of Remote Sensing by Simulation and Deep Learning for Flood and Debris Flow Mapping

  • Water-HRNet -> HRNet trained on Sentinel 2

  • semantic segmentation model to identify newly developed or flooded land using NAIP imagery provided by the Chesapeake Conservancy, training on MS Azure

  • BandNet -> Analysis and application of multispectral data for water segmentation using machine learning. Uses Sentinel-2 data

  • mmflood -> MMFlood: A Multimodal Dataset for Flood Delineation From Satellite Imagery (Sentinel 1 SAR)

  • Urban_flooding -> Towards transferable data-driven models to predict urban pluvial flood water depth in Berlin, Germany

  • MECNet -> Rich CNN features for water-body segmentation from very high resolution aerial and satellite imagery

  • SWRNET -> A Deep Learning Approach for Small Surface Water Area Recognition Onboard Satellite

  • elwha-segmentation -> fine-tuning Meta's Segment Anything (SAM) for bird's eye view river pixel segmentation

  • RiverSnap -> code for paper: A Comparative Performance Analysis of Popular Deep Learning Models and Segment Anything Model (SAM) for River Water Segmentation in Close-Range Remote Sensing Imagery

  • SAR-water-segmentation -> Deep Learning based Water Segmentation Using KOMPSAT-5 SAR Images

Segmentation - Fire, smoke & burn areas

Segmentation - Landslides

Segmentation - Glaciers

  • HED-UNet -> a model for simultaneous semantic segmentation and edge detection, examples provided are glacier fronts and building footprints using the Inria Aerial Image Labeling dataset

  • glacier_mapping -> Mapping glaciers in the Hindu Kush Himalaya, Landsat 7 images, Shapefile labels of the glaciers, Unet with dropout

  • GlacierSemanticSegmentation

  • Antarctic-fracture-detection -> uses UNet with the MODIS Mosaic of Antarctica to detect surface fractures

  • sentinel_lakeice -> Lake Ice Detection from Sentinel-1 SAR with Deep Learning

Segmentation - methane

Segmentation - Other environmental

  • Detection of Open Landfills -> uses Sentinel-2 to detect large changes in the Normalized Burn Ratio (NBR)

  • sea_ice_remote_sensing -> Sea Ice Concentration classification

  • EddyNet -> A Deep Neural Network For Pixel-Wise Classification of Oceanic Eddies

  • schisto-vegetation -> Deep Learning Segmentation of Satellite Imagery Identifies Aquatic Vegetation Associated with Snail Intermediate Hosts of Schistosomiasis in Senegal, Africa

  • Earthformer -> Exploring space-time transformers for earth system forecasting

  • weather4cast-2022 -> Unet-3D baseline model for Weather4cast Rain Movie Prediction competition

  • WeatherFusionNet -> Predicting Precipitation from Satellite Data. weather4cast-2022 1st place solution

  • marinedebrisdetector -> Large-scale Detection of Marine Debris in Coastal Areas with Sentinel-2

  • kaggle-identify-contrails-4th -> 4th place Solution, Google Research - Identify Contrails to Reduce Global Warming

  • MineSegSAT -> An automated system to evaluate mining disturbed area extents from Sentinel-2 imagery

  • asos -> Recognizing protected and anthropogenic patterns in landscapes using interpretable machine learning and satellite imagery

  • SinkSAM -> Knowledge-Driven Self-Supervised Sinkhole Segmentation Using Topographic Priors and Segment Anything Model

Segmentation - Roads & sidewalks

Extracting roads is challenging due to the occlusions caused by other objects and the complex traffic environment

Segmentation - Buildings & rooftops

Segmentation - Solar panels

Segmentation - Ships & vessels

Segmentation - Other manmade

  • Aarsh2001/ML_Challenge_NRSC -> Electrical Substation detection

  • electrical_substation_detection

  • MCAN-OilSpillDetection -> Oil Spill Detection with A Multiscale Conditional Adversarial Network under Small Data Training

  • mining-detector -> detection of artisanal gold mines in Sentinel-2 satellite imagery for Amazon Mining Watch. Also covers clandestine airstrips

  • EG-UNet Deep Feature Enhancement Method for Land Cover With Irregular and Sparse Spatial Distribution Features: A Case Study on Open-Pit Mining

  • plastics -> Detecting and Monitoring Plastic Waste Aggregations in Sentinel-2 Imagery

  • MADOS -> Detecting Marine Pollutants and Sea Surface Features with Deep Learning in Sentinel-2 Imagery on the MADOS dataset

  • SADMA -> Residual Attention UNet on MARIDA: Marine Debris Archive is a marine debris-oriented dataset on Sentinel-2 satellite images

  • MAP-Mapper -> Marine Plastic Mapper is a tool for assessing marine macro-plastic density to identify plastic hotspots, underpinned by the MARIDA dataset.

  • substation-seg -> segmenting substations in Sentinel 2 satellite imagery

  • SAMSelect -> An Automated Spectral Index Search for Marine Debris using Segment-Anything (SAM)

Panoptic segmentation

Segmentation - Miscellaneous