Repository navigation
TST: CI job frequently fails with cache damaged error #9765
Description
Description
After the series of cache refactoring PRs, starting with #9182 , the Windows job in CI often fails with an error that reports damaged cache, but often times it is unrelated to the affected PR. The current workaround is for someone with enough Travis CI admin access to go in and delete the cache for the PR, and then rerun the job. However, this is not a feasible solution for the long run.
@aarchiba , is there a way to mitigate this situation? Thanks!
Activity
bsipocz commentedon Dec 12, 2019
Not just the windows build, but I just needed to clear the cache for the remote-data/numpy-dev job on the 4.0.x branch as it was failing with various cache related issues even after restarting the job 3 times.
[-]TST: Windows CI job frequently fails with cache damaged error[/-][+]TST: CI job frequently fails with cache damaged error[/+]pllim commentedon Dec 13, 2019
Example logs from @bsipocz :
aarchiba commentedon Dec 13, 2019
There is definitely something wrong but I don't know what yet. I haven't seen it locally yet, which doesn't help debugging, but I haven't had a chance to sit down and hammer at it.
pllim commentedon Dec 13, 2019
The stop gap plan is to skip that flaky test so release can proceed.
After that, I will open a follow-up PR to undo the stop-gap and hopefully find the real cause. Stay tuned!aarchiba commentedon Mar 4, 2020
I wonder if maybe this is the result of the temporary-directory pytest fixture not always giving different temporary directories? Perhaps I can switch to a test fixture I trust and see if it fixes this - though as an erratic failure it's not going to be easy to tell. Perhaps I can replicate it locally with a looping parallel tox run?
pllim commentedon Jul 31, 2020
Haven't seen this in a while.