Deep Learning Hackathon @ AmritaEuropean Space Agency Sentinel-2 L2ADeterministic Evaluation (N=40 Test)

Ilādṛṣṭi: Multispectral Wildfire
Semantic Segmentation

Sanskrit: इला (Earth) + दृष्टि (Vision) — Autonomous Earth Observation Intelligence

Engineered for high-precision autonomous wildfire perimeter delineation and burned area scar mapping. Combines 4-Band Transfer Learning with Focal-Tversky Loss Optimization to penetrate dense smoke hazes, resolve fine perimeter boundaries, and accelerate tactical disaster response and aerial retardant drops.

Test Dice Score (F1)
0.7834
Held-out test split (N=40)
Test Mean IoU
0.7044
Strict spatial Jaccard index
NIR Spectral Gain
+7.69%
+15.3% relative boost
Edge GPU Latency
36.4 FPS
< 50s per full Sentinel tile
Model Precision
ResNet-50
4-Band Transfer Adapted U-Net
Methodology & Deep Learning Architecture

ResNet-50 U-Net & Focal-Tversky Loss

Resolving severe spatial class imbalance and smoke occlusion through multispectral weight transfer and gradient-focusing loss formulations.

Symmetrical 5-Stage U-Net Architecture

Standard optical segmentation models fail when wildfires occupy less than 5% of a satellite tile. Ilādṛṣṭi couples a deep ResNet-50 encoder with a customized 5-stage expansive decoder to retain fine perimeter details.

1

4-Band Initial Convolution Adaptation

Adapts ImageNet-pretrained weights to 4-channel input (Red, Green, Blue, NIR). The 4th NIR channel kernel is initialized via channel-wise mean averaging, preserving feature transfer without cold-start penalties.

2

Multi-Scale Residual Skip Connections

High-resolution feature maps from ResNet stages (C1, C2, C3, C4) are routed directly into the decoder stages, restoring thin fire breaks and boundary perimeters lost during downsampling.

3

Expansive Decoder with Double Convolutions

Each decoder stage applies bilinear upsampling followed by double 3x3 convolutions, Batch Normalization, and ReLU activations, terminating in a 1x1 logit projection head.

Multispectral Weight Transfer Formula:
Wconv1[:,3:4,:,:]←13∑c=02WImageNet[:,c:c+1,:,:]\mathbf{W}_{\text{conv1}}[:, 3:4, :, :] \leftarrow \frac{1}{3} \sum_{c=0}^{2} \mathbf{W}_{\text{ImageNet}}[:, c:c+1, :, :]

Focal-Tversky Loss

Phase 2 Objective

Standard Binary Cross-Entropy (BCE) treats every pixel equally, allowing overwhelming background pixels (95%) to suppress fire gradient updates. The Focal-Tversky Loss (FTL) penalizes False Negatives and concentrates learning on ambiguous boundary pixels.

FTL=(1−TI)γ=(1−TI)1.333\mathrm{FTL} = (1 - \mathrm{TI})^\gamma = (1 - \mathrm{TI})^{1.333}
Non-Linear Boundary Focusing Objective
TI=TP+ϵTP+0.7 FN+0.3 FP+ϵ\mathrm{TI} = \frac{\mathrm{TP} + \epsilon}{\mathrm{TP} + 0.7\,\mathrm{FN} + 0.3\,\mathrm{FP} + \epsilon}
Asymmetric Tversky Index Formulation
α = 0.7
Heavy penalty on False Negatives
β = 0.3
Moderate penalty on False Positives
γ = 1.333
Focuses gradient on hard borders

Theoretical Guarantee: When γ > 1, easy confident predictions (TI → 1) generate near-zero loss, preventing background saturation and driving backpropagation updates into elusive firebreak margins.

Interactive Demonstration

Multispectral Inference & Comparison Slider

Drag the interactive divider to compare raw Sentinel-2 satellite inputs directly with Ilādṛṣṭi's predicted wildfire perimeter masks.

Raw Sentinel-2 RGB
Predicted Wildfire Mask Overlay
Raw Sentinel-2 RGB
Predicted Wildfire Mask Overlay
Mega-Fire Scar Complex (b__06_06.tif)

Boreal Pine Forest, Northern Mediterranean • Massive contiguous wildfire scar (>46,000 pixels burned) displaying near-perfect contour delineation and high model confidence.

Scene Dice
0.9317
Mean IoU
0.8747
Precision
95.3%
Recall
92.3%
Burned Scar
74.0%
Empirical Results & Spectral Ablation

Benchmark Evaluation Dashboard

Strict deterministic evaluation across an isolated held-out test split (Seed locked at 42). Comparing Mandatory Baseline (BCE) against Focal-Tversky across spectral configurations.

Model Architecture Benchmark (Held-out Test N=40)

Dice & IoU

Comparative test metrics comparing 4-Band BCE Baseline vs 3-Band and 4-Band Focal-Tversky configurations.

NIR Spectral Gain (3-Band vs 4-Band)

+7.69% Absolute Gain

Quantifying the physical penetration gain enabled by the Sentinel-2 Band 8 (NIR, 842nm) channel.

15-Epoch Training Convergence Progression

AdamW Optimizer • Cosine Annealing LR (1e-4 → 1e-6) • Checkpointed at Peak Val Dice

Best Checkpoint: Epoch 5
Scientific Integrity & Edge Cases

Limitations & Optical Failure Analysis

Honest accounting of physical optical ambiguities, false positive triggers under extreme atmospheric conditions, and proposed architectural mitigations.

Transparent Negative Results Audit

While Ilādṛṣṭi demonstrates high accuracy across standard vegetative wildfire regimes, optical satellite systems suffer from physical photon scattering ambiguities where non-fire phenomena exhibit spectral signatures resembling carbonized ash.

False Positive Commission

1. Dense Clouds & Edge Diffraction

Physical Mechanism:

High-albedo cloud tops reflect >85% of visible and NIR solar radiation. At cloud edges, sharp dropoffs create extreme spatial gradients that convolutional filters occasionally mistake for fresh fire perimeters.

Observed Impact:

Fringe false alarm rings bordering thick stratocumulus banks (demonstrated in Scene a1__07_13 with 13,269 FP pixels).

Phase 3 Mitigation: Ingestion of Sentinel-2 Scene Classification Layer (SCL) to pre-mask cloud and cirrus pixels prior to inference.
Ash Spectral Mimicry

2. Deep Mountain Ravine Shadows

Physical Mechanism:

Steep north-facing alpine ravines receive minimal direct solar irradiance. Low reflectance values (<0.05 across RGB and NIR) deceptively mimic the flat, dark optical absorption profile of charcoal ash.

Observed Impact:

Localized false positive clusters in deep mountain gorges (demonstrated in Scene k__4_6 with 7,211 FP pixels).

Phase 3 Mitigation: Fusing Short-Wave Infrared (SWIR Band 12, 2190nm) and Digital Elevation Model (DEM) hillshading layers.
Transient Omission

3. Low-Biomass Grassland Scars

Physical Mechanism:

Fast-moving savanna grass fires leave only ephemeral surface soot without deep canopy destruction. Wind rapidly scatters the light ash layer within 48 hours, causing spectral signatures to revert before satellite revisit.

Observed Impact:

Under-segmentation (False Negatives) for thin pasture fires compared to dense coniferous forest burns.

Phase 3 Mitigation: Bi-temporal Sentinel-2 pre/post difference mapping (dNBR) to detect transient vegetative biomass shifts.

Empirical Failure Showcase: Cloud Boundary & Shadow False Positives

Direct inspection of actual test samples where optical limitations triggered false alarms (Scene a1__07_13):

Cloud Tile Sentinel-2 RGB
1. Sentinel-2 RGB (Cloud Scene a1__07_13)
Prediction Overlay on Cloud Scene
2. False Alarms along Cloud Margin (Red Overlay)
Spatial Confusion Map
3. Spatial Confusion Map (Red = Commission Error)