This document lists resources for performing deep learning (DL) on satellite imagery. To a lesser extent classical Machine learning (ML, e.g. random forests) are also discussed, as are classical image processing techniques. Note there is a huge volume of academic literature published on these topics, and this repo does not seek to index them all but rather list approachable resources with published code that will benefit both the research and developer communities.
The classic cats vs dogs image classification task, which in the remote sensing domain is used to assign a label to an image, e.g. this is an image of a forest. The more complex case is applying multiple labels to an image. This approach of image level classification is not to be confused with pixel-level classification which is called semantic segmentation. In general, aerial images cover large geographical areas that include multiple classes of land, so treating this is as a classification problem is less common than using semantic segmentation. I recomend to get started with the EuroSAT dataset.
RSI-CB -> A Large Scale Remote Sensing Image Classification Benchmark via Crowdsource Data
NAIP_PoolDetection -> modelled as an object recognition problem, a CNN is used to identify images as being swimming pools or something else - specifically a street, rooftop, or lawn
SSTN -> PyTorch Implementation of SSTNs for hyperspectral image classifications from the IEEE T-GRS paper "Spectral-Spatial Transformer Network for Hyperspectral Image Classification: A FAS Framework." Demonstrates a novel spectral-spatial transformer network (SSTN), which consists of spatial attention and spectral association modules, to overcome the constraints of convolution kernels
SatellitePollutionCNN -> A novel algorithm to predict air pollution levels with state-of-art accuracy using deep learning and GoogleMaps satellite images
PropertyClassification -> Classifying the type of property given Real Estate, satellite and Street view Images
remote-sense-quickstart -> classification on a number of datasets, including with attention visualization
IGARSS2020_BWMS -> Band-Wise Multi-Scale CNN Architecture for Remote Sensing Image Scene Classification with a novel CNN architecture for the feature embedding of high-dimensional RS images
hurricane_damage -> Post-hurricane structure damage assessment based on aerial imagery with CNN
openai-drivendata-challenge -> Using deep learning to classify the building material of rooftops (aerial imagery from South America)
is-it-abandoned -> Can we tell if a house is abandoned based on aerial LIDAR imagery?
BoulderAreaDetector -> CNN to classify whether a satellite image shows an area would be a good rock climbing spot or not
ISPRS_S2FL -> code for paper: Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively
tf-crash-severity -> predict the crash severity for given road features contained within satellite images
Segmentation
Segmentation will assign a class label to each pixel in an image. Segmentation is typically grouped into semantic, instance or panoptic segmentation. In semantic segmentation objects of the same class are assigned the same label, whilst in instance segmentation each object is assigned a unique label. Panoptic segmentation combines instance and semantic predictions. Read this beginner’s guide to segmentation. Single class models are often trained for road or building segmentation, with multi class for land use/crop type classification. Image annotation can take longer than for object detection since every pixel must be annotated. Note that many articles which refer to 'hyperspectral land classification' are actually describing semantic segmentation. Note that cloud detection can be addressed with semantic segmentation and has its own section Cloud detection & removal
SpectralNET -> a 2D wavelet CNN for Hyperspectral Image Classification, uses Salinas Scene dataset & Keras
FactSeg -> Foreground Activation Driven Small Object Semantic Segmentation in Large-Scale Remote Sensing Imagery (TGRS), also see FarSeg and FreeNet, implementations of research paper
SCAttNet -> Semantic Segmentation Network with Spatial and Channel Attention Mechanism
laika -> The goal of this repo is to research potential sources of satellite image data and to implement various algorithms for satellite image segmentation
PEARL -> a human-in-the-loop AI tool to drastically reduce the time required to produce an accurate Land Use/Land Cover (LULC) map, blog post, uses Microsoft Planetary Computer and ML models run locally in the browser. Code for backelnd and frontend
unetseg -> A set of classes and CLI tools for training a semantic segmentation model based on the U-Net architecture, using Tensorflow and Keras. This implementation is tuned specifically for satellite imagery and other geospatial raster data.
clusternet_segmentation -> Unsupervised Segmentation by applying K-Means clustering to the features generated by Neural Network
DCA -> Code for TGRS 2022 paper "Deep Covariance Alignment for Domain Adaptive Remote Sensing Image Segmentation"
weakly_supervised -> code for the paper Weakly Supervised Deep Learning for Segmentation of Remote Sensing Imagery. Demonstrates that segmentation can be performed using small datasets comprised of pixel or image labels
Collection of different Unet Variant -> demonstrates VggUnet, ResUnet, DenseUnet, Unet. AttUnet, MobileNetUnet, NestedUNet, R2AttUNet, R2UNet, SEUnet, scSEUnet, Unet_Xception_ResNetBlock, in keras
Efficient-Transformer -> an efficient transformer to achieve light-weight segmentation with improved edge segmentation by applying explicit and implicit edge enhancement methods, achieves state-of-the-art performance on Potsdam and Vaihingen datasets. Paper
IEEE_TGRS_SpectralFormer -> code for 2021 paper: Spectralformer: Rethinking hyperspectral image classification with transformers
Semantic segmentation - multiclass classification
Land Cover Classification with U-Net -> Satellite Image Multi-Class Semantic Segmentation Task with PyTorch Implementation of U-Net, uses DeepGlobe Land Cover Segmentation dataset, with code
CDL-Segmentation -> code for the paper: "Deep Learning Based Land Cover and Crop Type Classification: A Comparative Study", comparing UNet, SegNet & DeepLabv3+
floatingobjects -> code for the paper: TOWARDS DETECTING FLOATING OBJECTS ON A GLOBAL SCALE WITHLEARNED SPATIAL FEATURES USING SENTINEL 2. Uses U-Net & pytorch
LoveDA -> code for the paper "A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation"
Remote-sensing-image-semantic-segmentation -> The project uses Unet-based improved networks to study Remote sensing image semantic segmentation, which is based on keras
solar-panel-locator -> the number of solar panel pixels was only ~0.2% of the total pixels in the dataset, so solar panel data was upsampled to account for the class imbalance
HybridSN -> A keras based implementation of Hybrid-Spectral-Net as in IEEE GRSL paper "HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification"
TNNLS_2022_X-GPN -> Code for paper: Semisupervised Cross-scale Graph Prototypical Network for Hyperspectral Image Classification
Semantic Segmentation on Aerial Images using fastai uses U-Net on the Inria Aerial Image Labeling Dataset of urban settlements in Europe and the United States, and is labelled as a building and not building classes (no repo)
automated-building-detection -> Input: very-high-resolution (<= 0.5 m/pixel) RGB satellite images. Output: buildings in vector format (geojson), to be used in digital map products. Built on top of robosat and robosat.pink.
project_sunroof_india -> Analyzed Google Satellite images to generate a report on individual house rooftop's solar power potential, uses a range of classical computer vision techniques (e.g Canny Edge Detection) to segment the roofs
TF-SegNet -> AirNet is a segmentation network based on SegNet, but with some modifications
rgb-footprint-extract -> a Semantic Segmentation Network for Urban-Scale Building Footprint Extraction Using RGB Satellite Imagery, DeepLavV3+ module with a Dilated ResNet C42 backbone
SpaceNetExploration -> A sample project demonstrating how to extract building footprints from satellite images using a semantic segmentation model. Data from the SpaceNet Challenge
solar-farms-mapping -> An Artificial Intelligence Dataset for Solar Energy Locations in India
poultry-cafos -> This repo contains code for detecting poultry barns from high-resolution aerial imagery and an accompanying dataset of predicted barns over the United States
ssai-cnn -> This is an implementation of Volodymyr Mnih's dissertation methods on his Massachusetts road & building dataset
segmentation-enhanced-resunet -> Urban building extraction in Daejeon region using Modified Residual U-Net (Modified ResUnet) and applying post-processing
GRSL_BFE_MA -> Deep Learning-based Building Footprint Extraction with Missing Annotations using a novel loss function
FER-CNN -> Detection, Classification and Boundary Regularization of Buildings in Satellite Imagery Using Faster Edge Region Convolutional Neural Networks, with paper
building-footprint-segmentation -> pip installable library to train building footprint segmentation on satellite and aerial imagery, applied to Massachusetts Buildings Dataset and Inria Aerial Image Labeling Dataset
SemSegBuildings -> Project using fast.ai framework for semantic segmentation on Inria building segmentation dataset
FCNN-example -> overfit to a given single image to detect houses
SAT2LOD2 -> an open-source, python-based GUI-enabled software that takes the satellite images as inputs and returns LoD2 building models as outputs, with paper
SatFootprint -> building segmentation on the Spacenet 7 dataset
Building-Detection -> code for running a Raster Vision experiment to train a model to detect buildings from satellite imagery in three cities in Latin America
Multi-building-tracker -> code for paper: Multi-target building tracker for satellite images using deep learning
LGPNet-BCD -> code for 2021 paper: Building Change Detection for VHR Remote Sensing Images via Local-Global Pyramid Network and Cross-Task Transfer Learning Strategy
Semantic segmentation - roads
Extracting roads is challenging due to the occlusions caused by other objects and the complex traffic environment
awesome-deep-map -> A curated list of resources dedicated to deep learning / computer vision algorithms for mapping. The mapping problems include road network inference, building footprint extraction, etc.
road_detection_mtl -> Road Detection using a multi-task Learning technique to improve the performance of the road detection task by incorporating prior knowledge constraints, uses the SpaceNet Roads Dataset
road_connectivity -> Improved Road Connectivity by Joint Learning of Orientation and Segmentation (CVPR2019)
CoANet -> Connectivity Attention Network for Road Extraction From Satellite Imagery. The CoA module incorporates graphical information to ensure the connectivity of roads are better preserved. With paper
ML_EPFL_Project_2 -> U-Net in Pytorch to perform semantic segmentation of roads on satellite images
kenya-crop-mask -> Annual and in-season crop mapping in Kenya - LSTM classifier to classify pixels as containing crop or not, and a multi-spectral forecaster that provides a 12 month time series given a partial input. Dataset downloaded from GEE and pytorch lightning used for training
crop-type-classification -> using Sentinel 1 & 2 data with a U-Net + LSTM, more features (i.e. bands) and higher resolution produced better results (article, no code)
Houston_flooding -> labeling each pixel as either flooded or not using data from Hurricane Harvey. Dataset consisted of pre and post flood images, and a ground truth floodwater mask was created using unsupervised clustering (with DBScan) of image pixels with human cluster verification/adjustment
ml4floods -> An ecosystem of data, models and code pipelines to tackle flooding with ML
AI Geospatial Wildfire Risk Prediction -> A predictive model using geospatial raster data to asses wildfire hazard potential over the contiguous United States using Unet
rescue -> code of the paper: Attention to fires: multi-channel deep-learning models forwildfire severity prediction
Semantic segmentation - glaciers
HED-UNet -> a model for simultaneous semantic segmentation and edge detection, examples provided are glacier fronts and building footprints using the Inria Aerial Image Labeling dataset
glacier_mapping -> Mapping glaciers in the Hindu Kush Himalaya, Landsat 7 images, Shapefile labels of the glaciers, Unet with dropout
glacier-detect-ML -> a simple logistic regression model to identify a glacier in Landsat satellite imagery
hyperion_solar_net -> trained classificaton & segmentation models on RGB imagery from Google Maps. Provides app for viewing predictions, and has arxiv paper and a nice webpage
3D-PV-Locator -> Large-scale detection of rooftop-mounted photovoltaic systems in 3D
PV_Pipeline -> PyTorch models and pipeline developed for "DeepSolar for Germany"
predict_pv_yield -> Using optical flow & machine learning to predict PV yield
Large-scale-solar-plant-monitoring -> code for the paper "Remote Sensing for Monitoring of Photovoltaic Power Plants in Brazil Using Deep Semantic Segmentation"
Panel-Segmentation -> Determine the presence of a solar array in the satellite image (boolean True/False), using a VGG16 classification model
Roofpedia -> an open registry of green roofs and solar roofs across the globe identified by Roofpedia through deep learning
solar-pv-global-inventory -> code from the Nature paper of Kruitwagen et al, used to produce a global inventory of utility-scale solar photvoltaic generating stations
remote-sensing-solar-pv -> A repository for sharing progress on the automated detection of solar PV arrays in sentinel-2 remote sensing imagery
请发表评论