I think we should have a new log level which enables SDK verbose logging. When I debug an issue, I enable SDK verbose logging to see the request/response cycles between s5cmd and the remote endpoint.
Currently we have debug, info and error. --log=debug is used when the cp command skips overriding a file when certain criteria is met, such as cp -n -s -u.
I propose a new level because we can't use debug to print SDK verbose logs, which is not structured. That'd break our --json output.
--log=trace will be used to see what's really happening under the hood. It'll look like this:
$ s5cmd --log=trace ls
2021/09/23 10:43:47 DEBUG: Request s3/ListBuckets Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET / HTTP/1.1
Host: s3.amazonaws.com
User-Agent: aws-sdk-go/1.35.13 (go1.16.2; darwin; amd64)
Authorization: <REDACTED>
X-Amz-Content-Sha256: <REDACTED>
X-Amz-Date: 20210923T074347Z
X-Amz-Security-Token: <REDACTED>
Accept-Encoding: gzip
-----------------------------------------------------
2021/09/23 10:43:48 DEBUG: Response s3/ListBuckets Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 400 Bad Request
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 23 Sep 2021 07:43:48 GMT
Server: AmazonS3
X-Amz-Id-2: <REDACTED>
X-Amz-Request-Id: <REDACTED>
-----------------------------------------------------
ERROR "ls": ExpiredToken: The provided token has expired. status code: 400, request id: <REDACTED>, host id: <REDACTED>
I think we should have a new log level which enables SDK verbose logging. When I debug an issue, I enable SDK verbose logging to see the request/response cycles between s5cmd and the remote endpoint.
Currently we have
debug,infoanderror.--log=debugis used when thecpcommand skips overriding a file when certain criteria is met, such ascp -n -s -u.I propose a new level because we can't use
debugto print SDK verbose logs, which is not structured. That'd break our--jsonoutput.--log=tracewill be used to see what's really happening under the hood. It'll look like this: