🛰️ RSINet: Inpainting Remotely Sensed Images Using a Triple GAN Framework

Restoring missing/corrupted regions in high-resolution aerial & satellite imagery with edge-, colour-, and texture-aware GANs.

Advait Kumar* · Dipesh Tamboli* · Shivam Pande · Biplab Banerjee   _{(* equal contribution)}

IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2022

Inpainting results on the Earth-on-Canvas dataset — (a–b) rectangular masks, (c–d) salt-and-pepper noise, (e–f) irregular masks. For each triplet: original · corrupted · RSINet reconstruction.

---

💡 TL;DR

Remote-sensing images are high-resolution and highly variable, so conventional inpainting models struggle to capture their spectral, spatial, and textural nuances. RSINet tackles this by handling each aspect of an image — edges, colour, and texture — with a task-specific GAN. Each GAN uses a gated attention mechanism to explicitly extract spectral and spatial features, and a residual-learning paradigm to keep gradients flowing across high- and low-level features. On the Open Cities AI and Earth on Canvas benchmarks, RSINet achieves competitive-to-state-of-the-art performance across rectangular, salt-and-pepper, and irregular corruption patterns.

🧠 Method

Each generator follows an encoder → residual-blocks → decoder structure with a skip connection, and encoder feature maps act as gates for gated attention layers in the decoder. A convolutional discriminator distinguishes real from reconstructed images.

Three such GANs specialize on complementary cues:

• Edge GAN — reconstructs structural edges over the masked region (Canny edge priors).
• Inpainting GAN — fills in colour and texture conditioned on the completed edges.
• Gated attention + residual learning — explicitly extract spectral/spatial features and stabilize gradient flow across scales.

📦 Repository structure

File	Description
main.py	Training driver — alternates the inpainting and edge GANs over epochs
inpaint.py	Inpainting (colour/texture) GAN — training loop
edge.py	Edge GAN — training loop
global_gan.py	Full/global RSINet GAN
models.py	Generator & discriminator architectures
attention.py	Gated (spectral–spatial) attention modules
dataset.py	Dataset loader + mask generation (rectangular / salt-and-pepper / irregular)
metrics.py	Evaluation metrics (PSNR, edge accuracy)
testing.py	Evaluation / inference
utilities.py, init_weights.py	Helpers and weight initialization

🚀 Usage

Train (alternates the edge and inpainting GANs each epoch):

python3 main.py

You can also train a component or the combined model directly:

python3 inpaint.py      # inpainting (colour/texture) GAN
python3 edge.py         # edge GAN
python3 global_gan.py   # full/global RSINet GAN

Evaluate:

python3 testing.py

Dataset roots and hyper-parameters (image size 256, batch_size 4, Adam lr 1e-4) are configured inside the scripts (dataset.py / the training files). Irregular masks are read from an irregular_masks/ folder.

📊 Datasets

RSINet is evaluated on two remote-sensing datasets:

• Open Cities AI — high-resolution aerial imagery of African cities.
• Earth on Canvas — a remote-sensing dataset with diverse land-cover scenes.

⚙️ Requirements

This code was developed in a 2021-era Colab environment and uses scipy.misc.imread (removed in SciPy ≥ 1.2). To run it, use a matching legacy stack — e.g. scipy<1.2, torch, torchvision, opencv-python, scikit-image, matplotlib, numpy. Porting to modern SciPy (imageio.imread) is straightforward.

📚 Citation

If you find RSINet useful, please cite:

@inproceedings{kumar2022rsinet,
  title     = {RSINet: Inpainting Remotely Sensed Images Using Triple GAN Framework},
  author    = {Kumar, Advait and Tamboli, Dipesh and Pande, Shivam and Banerjee, Biplab},
  booktitle = {IGARSS 2022 - IEEE International Geoscience and Remote Sensing Symposium},
  pages     = {1--4},
  year      = {2022},
  doi       = {10.1109/IGARSS46834.2022.9884330}
}