I keep having this error during the training using recent pytorch version. It was okay before.
Code:
result = comm.reduce_add(inputs)
results = []
for i in range(len(inputs)):
results.append(result.cuda(i))Error:
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/_utils.py", line 66, in _cuda
return new_type(self.size()).copy_(self, async)
RuntimeError: cuda runtime error (4) : unspecified launch failure at /home/ubuntu/pytorch/torch/lib/THC/THCTensorCopy.cu:85
terminate called after throwing an instance of 'std::runtime_error'
what(): cuda runtime error (4) : unspecified launch failure at /home/ubuntu/pytorch/torch/lib/THC/generic/THCStorage.c:182
I keep having this error during the training using recent pytorch version. It was okay before.
Code:
Error: