FireRGBTNet

Official PyTorch implementation of FireRGBTNet: A Lightweight Forest Fire Detection Model Based on Efficient RGB–Thermal Fusion (Forests 2026, 17, 955).

Paper | Dataset (Google Drive) | Dataset (Quark, code hyx1)

FireRGBTNet is an end-to-end RGB-T detector for UAV forest fire monitoring. It uses a heterogeneous dual-stream backbone, an explicit cross-modal alignment branch and a lightweight gated fusion neck, followed by an RT-DETR style decoder.

Results on RGBT-3M

P	R	F1	mAP@0.5	mAP@0.75	mAP@0.5:0.95	FPS	Params	GFLOPs
91.8	94.7	93.2	95.9	68.9	62.8	44.7	4.13 M	13.50

The paper reports 4.13 M parameters and 13.50 GFLOPs with the MSAE branch removed, which is the configuration profiled by evaluate.py. FPS is measured on a Tesla V100 with a batch size of 1.

Method

Component	Role
RGBBlock	ELAN-style block with standard convolutions for dense RGB textures
ThermalBlock	ELAN-style block with large-kernel depthwise convolutions for sparse thermal features
TED	Target-Enhanced Downsampling: max-pool and stride-2 convolution branches concatenated to preserve small targets
MSAE	Multi-Scale Semantic Alignment Enhancement: semantic projection heads with channel-wise Wasserstein (CWA) and cosine constraints on C3/C4/C5; training only
MSGF	Multimodal Spatial Gated Fusion: SGCA for multi-scale gated cross-attention and MishGLU for global context compensation
RTDETRDecoder	300 queries, hidden dimension 128, 2 decoder layers, IoU-aware scoring

Repository structure

.
├── data/
│   ├── __init__.py
│   └── datasets.py          # paired RGB/thermal dataset and LightningDataModule
├── models/
│   ├── __init__.py
│   ├── firergbtnet.py       # backbones, fusion neck and the LightningModule
│   ├── head.py              # RT-DETR decoder
│   └── modules.py           # TED, RGB/Thermal blocks, SGCA, MishGLU, MSGF, MSAE
├── loss.py                  # Hungarian matching, focal/L1/GIoU/IoU-aware losses
├── metrics.py               # mAP, precision, recall, F1, confusion matrix
├── utils.py                 # box utilities
├── train.py                 # training entry point
├── evaluate.py              # params, GFLOPs, FPS and validation metrics
├── requirements.txt
└── LICENSE

Installation

The reference environment is Python 3.10, PyTorch 2.4.1 (CUDA 11.8), Lightning 2.5.5 and torchmetrics 1.8.2.

conda create -n firergbt python=3.10 -y
conda activate firergbt
pip install -r requirements.txt

requirements.txt points at the CUDA 11.8 wheels used in the paper. For a CPU-only or different CUDA build, install the matching torch and torchvision first, then the remaining packages.

Dataset

Download RGBT-3M from one of the links above and arrange the pairs as:

datasets/RGBT-3M/RGBT/
├── rgb/
│   ├── train/*.jpg
│   └── val/*.jpg
├── ir/
│   ├── train/*.jpg
│   └── val/*.jpg
└── labels/
    ├── train/*.txt
    └── val/*.txt

• rgb/ and ir/ must share file names for each pair.
• Labels use the YOLO format class cx cy w h with normalized coordinates.
• The three classes are 0: Smoke, 1: Fire, 2: Person.
• Images are padded to 640x640 as described in the paper. If your copy of the dataset uses different folder names, rename them or edit RGBIRDataset in data/datasets.py.
• The loader falls back to an all-zero thermal image if a pair is missing on the thermal side.

Training

Edit DATA_DIR at the bottom of train.py, then run:

python train.py

Default settings follow the paper:

• input size 640x640, batch size 32, 250 epochs, 16 dataloader workers
• AdamW, initial LR 1e-3, weight decay 1e-4, cosine schedule with 2% linear warm-up
• gradient clipping 0.1, 4-step gradient accumulation, 16-bit mixed precision
• mosaic/mixup disabled for the last 30 epochs, EMA weights used for validation
• checkpoints are selected by the best val_map

All weights are trained from scratch; no ImageNet or COCO pretraining is used.

Evaluation

Set ckpt_path and data_dir in evaluate.py, then run:

python evaluate.py

The script reports parameters, GFLOPs, FPS and validation P/R/F1/mAP. The MSAE branch is removed before profiling, matching the inference graph used in the paper.

Deployment

The inference graph is a plain nn.Module: remove model.msae and call model(rgb, thermal). Inputs are 1x3x640x640 RGB and thermal tensors normalized as in data/datasets.py; outputs are normalized cxcywh boxes, class logits and IoU scores. The module can be exported with torch.onnx or compiled with TensorRT.

Notes

• The training-only MSAE branch has no effect on the features passed to the neck, so it can be deleted after training without changing predictions.
• The released code uses the final paper configuration: 3 classes, 640x640 input, RT-DETR with 300 queries, hidden dimension 128 and 2 decoder layers.
• TED follows the implementation used for the reported results: the max-pool and stride-2 convolution branches are concatenated.

Citation

@article{ma2026firergbtnet,
  title   = {FireRGBTNet: A Lightweight Forest Fire Detection Model Based on Efficient RGB--Thermal Fusion},
  author  = {Ma, Yifan and Shan, Weifeng and Wang, Maofa and Sui, Yanwei and Wang, Mengyu},
  journal = {Forests},
  volume  = {17},
  number  = {8},
  pages   = {955},
  year    = {2026},
  doi     = {10.3390/f17080955}
}

License

MIT. See LICENSE.