Issue & steps to reproduce
$ echo "Example text." > tmp
$ cat tmp
Example text.
$ s5cmd cp s3://nonexistentbucket/key tmp
ERROR "cp s3://nonexistentbucket/key tmp": InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records. status code: 403, request id: PAYWDH7HHVPAHC8G, host id: cnI/cBhYZKKJ159f8mMs4KHOn1+jYtvfpYnXgMuKMRd9pZ10FBdi/cGuVxyr+iwrKbk2kP7Opx4=
$ cat tmp
cat: tmp: No such file or directory
We would expect the local file to remain untouched since the download operation did not even started.
Proposed Solution
Instead of creating the file before the download request a custom WriterAt, which will create the file once the first write request came, can be used. Following snippets are going to change:
|
file, err := dstClient.Create(dsturl.Absolute()) |
|
if err != nil { |
|
return err |
|
} |
|
defer file.Close() |
|
|
|
size, err := srcClient.Get(ctx, srcurl, file, c.concurrency, c.partSize) |
|
func (f *Filesystem) Create(path string) (*os.File, error) { |
|
if f.dryRun { |
|
return &os.File{}, nil |
|
} |
|
|
|
return os.Create(path) |
|
} |
Issue & steps to reproduce
We would expect the local file to remain untouched since the download operation did not even started.
Proposed Solution
Instead of creating the file before the download request a custom WriterAt, which will create the file once the first write request came, can be used. Following snippets are going to change:
s5cmd/command/cp.go
Lines 511 to 517 in 123b1c7
s5cmd/storage/fs.go
Lines 208 to 214 in 0431f50