The code for TIGEr evaluation of our paper, Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning (WACV 2023).
The code is based on CapEval. We thank the authors of the repository.
git clone --recurse-submodules https://github.com/ukyh/tiger_cocout.git
cd tiger_cocout
conda create --name tiger python=2.7
conda activate tiger
conda install pytorch=0.4.1 cuda92 -c pytorch
conda install -c pytorch torchvision
conda install -c anaconda nltk
python -c "import nltk; nltk.download('punkt')"
conda install -c anaconda scipy
echo "" >> SCAN/__init__.pySee the following page for the details: https://github.com/SeleenaJM/CapEval#download-data-and-pretrained-scan-model
pip install gdown
mkdir -p data/precomp/usecase_coco_precomp
mkdir data/candidates
mkdir data/output
mkdir -p runs/coco_scan/log
cd ${HOME}/tiger_cocout/data/precomp/usecase_coco_precomp
gdown --fuzzy 'https://drive.google.com/file/d/1BCKOOZQLbxSIV9EwQFe77aM9McQsq-bH/view?usp=share_link'
gdown --fuzzy 'https://drive.google.com/file/d/1xuJsjR98nqJli9nldBhrbLm1nxe9ziGs/view?usp=share_link'
gdown --fuzzy 'https://drive.google.com/file/d/1gDe3G0gh_FbXFun3CPY1NT-PH761QtoJ/view?usp=share_link'
cd ${HOME}/tiger_cocout/runs/coco_scan/log
gdown --fuzzy 'https://drive.google.com/file/d/1MbTqWVsl5QGbjRL5ClmY3Ns7uC7lJkSt/view?usp=share_link'Copy the output files to evaluate from switch_disc_caption (the files under eval_results).
Then, run the following commands.
cd tiger_cocout
conda activate tiger
ID=sample_test
python -u preprocess_output.py eval_results/${ID}.json
python -u main.py --data_name usecase_coco_precomp --data_path $PWD/data/precomp/ --candidate_path $PWD/data/candidates --candidate_name ${ID} --output_path $PWD/data/output/If you find this code is useful, we appreicate it if you cite the following papers:
@inproceedings{honda2023switch,
title={Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning},
author={Honda, Ukyo and Taro, Watanabe and Yuji, Matsumoto},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year={2023}
}
@inproceedings{jiang-etal-2019-tiger,
title = "{TIGE}r: Text-to-Image Grounding for Image Caption Evaluation",
author = "Jiang, Ming and Huang, Qiuyuan and Zhang, Lei and Wang, Xin and Zhang, Pengchuan and Gan, Zhe and Diesner, Jana and Gao, Jianfeng",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)",
month = nov,
year = "2019",
address = "Hong Kong, China",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/D19-1220",
doi = "10.18653/v1/D19-1220",
pages = "2141--2152"
}