ScanFocus

Official implementation of ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding (ECCV 2026).

ScanFocus is a coarse-to-fine STVG framework that decouples long-video grounding into a global spatio-temporal scan and a local boundary focus. The coarse stage efficiently scans sparse frames to produce a rough spatio-temporal tube, while the refinement stage densely samples around coarse temporal boundaries and uses semantic-guided temporal aggregation to recover fine-grained boundary cues.

[Paper] [Code]

The framework contains two stages:

• Global ST Scan: a unified vision-language encoder and semantic-motion fusion encoder align appearance, language, and motion features on sparsely sampled frames.
• Local Boundary Focus: dense boundary sampling around the coarse start/end predictions feeds a semantic-guided temporal aggregator and refine decoders for precise timestamp localization.

• ScanFocus has been accepted by ECCV 2026.
• This repository is being prepared for public release.

Main SOTA comparisons reported in the paper.

The code has been tested with PyTorch 2.0.1 and CUDA 11.7.

conda create -n scanfocus python=3.8 -y
conda activate scanfocus
pip install -r requirements.txt

This project uses mmcv.ops.MultiScaleDeformableAttention; please install an MMCV version compatible with your PyTorch/CUDA setup if it is not already available in your environment.

Prepare datasets under data/:

data/
  hc-stvg/
    all_images/
    annos/
    data_cache/
  hc-stvg2/
    all_images/
    annos/
    data_cache/
  vidstg/
    videos/ or all_images/
    vstg_annos/
    sent_annos/
    data_cache/

For HC-STVG training in this release, frames are expected at:

data/hc-stvg/all_images

If your data is stored elsewhere, update DATA_DIR and DATASET.FRAME_DIR in the corresponding file under experiments/.

Prepare the following pretrained weights:

pretrain_weights/GRES/beit3_base_patch16_224.zip
model_zoo/vit_b_k710_dl_from_giant.pth

For BEiT-3 weights, please follow the official BEiT-3 repository. For VideoMAE-style video backbones, please refer to the official VideoMAE and VideoMAEv2 repositories. If you place checkpoints at different paths, update the corresponding entries in experiments/*.yaml.

The coarse-stage configuration is:

experiments/hcstvg_coarse.yaml

Run:

bash scripts/train_hcstvg_coarse.sh

By default, the coarse checkpoint is saved under:

work-dir/hc-stvg/experiments/coarse_sota/

The refine-stage configuration is:

experiments/hcstvg-refine.yaml

Edit COARSE_CKPT in scripts/train_hcstvg_refine.sh to point to the trained coarse checkpoint, then run:

bash scripts/train_hcstvg_refine.sh

By default, the refine checkpoint is saved under:

work-dir/hc-stvg/experiments/refine_sota/

Set MODEL.WEIGHT to a trained ScanFocus checkpoint and run:

bash scripts/test.sh MODEL.WEIGHT path/to/model.pth

You can also call the evaluation entry directly:

python -m torch.distributed.launch \
  --nproc_per_node=4 \
  scripts/test_net_refine.py \
  --config-file experiments/hcstvg-refine.yaml \
  MODEL.WEIGHT path/to/model.pth

If this project is useful for your research, please consider citing ScanFocus:

@inproceedings{chen2026scanfocus,
  title={ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding},
  author={Chen, Kai and Dai, Ming and Cheng, Wenxuan and Yang, Wankou},
  booktitle={European Conference on Computer Vision},
  year={2026}
}

This codebase builds on common components from prior STVG and vision-language grounding projects. We thank the authors of HC-STVG, VidSTG, CG-STVG, BEiT-3, VideoMAE, and related open-source projects.