The code for improved BERTScore evaluation of our paper, Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning (WACV 2023).
The code is based on improved-bertscore-for-image-captioning-evaluation. We thank the authors of the repository.
git clone https://github.com/ukyh/bertspp_cocout.git
cd bertspp_cocout
conda create --name bertspp python=3.6
conda activate bertspp
pip install -r requirements.txt
# Test run
python -u run_metric_custom.py --file samples --dir example mkdir data; cd data
wget http://cs.stanford.edu/people/karpathy/deepimagesent/caption_datasets.zip
unzip caption_datasets.zip
rm -f caption_datasets.zipThen, download cocotalk_disc_text.zip and unzip it into data/:
unzip cocotalk_disc_text.zip -d data/
Copy the output files to evaluate from switch_disc_caption (the files under eval_results).
Then, run the following commands.
cd bertspp_cocout
conda activate bertspp
export PYTHONPATH=$PYTHONPATH:`pwd`
# NOTE: the end of the file name has to be "_val.json" or "_test.json"
ID=sample_test
python -u proc_bert_score_pp.py --hyp eval_results/${ID}.json
python -u run_metric_custom.py --file ${ID} --dir coco_capsIf you find the paper or this code useful, please consider citing:
@inproceedings{honda2023switch,
title={Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning},
author={Honda, Ukyo and Taro, Watanabe and Yuji, Matsumoto},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year={2023}
}
@inproceedings{yi2020improving,
title={Improving image captioning evaluation by considering inter references variance},
author={Yi, Yanzhi and Deng, Hangyu and Hu, Jinglu},
booktitle={ACL},
year={2020}
}