Repository navigation
Investigate using Actions API to retrieve single job run logs #11118
Description
Activity
@williammartin and I looked into this to see if/how we can use the suggested API endpoint.
Limits of the new approach
It turned out that this new endpoint cannot replace our current approach (i.e. downloading ZIP archive of logs) due to the following reasons:
-
Missing step-wise logs: the log trail returned by the endpoint is an entire job run trail with no indication of individual steps (i.e. we cannot associate lines with their steps), while the ZIP archive (current approach) provides us with individual step logs (in addition to an entire job run log).
- This is an important issue, because when
ghdisplays the logs it prepends every line with the corresponding step name. - Also, the
--log-failedoption which is meant to show the logs of the failed steps, will not be useless if we don't have the step logs.
- This is an important issue, because when
-
Performance issue: the API endpoint is meant to be used to retrieve the logs of a single job run. However, when a user runs
gh run view --log <RUN-ID>,ghwill show the log of all jobs in the given run. Ifghwere to use this API endpoint, then it would have to make repeated API calls, which can be a remarkable number, like in matrix workflows. But in our current approach, only one API call to download the ZIP file is made. Also, despite the performance issue, the user might hit their rate-limit, avoiding which is complicated.
Use API as a fallback
Currently, it happens that the job/step log files are not found in the downloaded ZIP archive, and
ghdisplays nothing. Using this API endpoint as a fallback mechanism, can bring us value. We tried the approach and it works as it should. Coincidentally, this fallback mechanism also resolves other issues reported by our lovely community (#11059, #10868). However, we still have to improve our ZIP archive approach and fill in the gaps with the new Actions backend service (#10868).-
Context
We have had a good number of issues pointing out
gh run view (--log | --log-failed)command fails to display the requested job run logs. One of the reasons, for most of the cases, was the changes made to the ZIP file structure (i.e. file naming and sanitisation of special chars) thatghdownloads to extract the logs from.Suggested approach
As of our internal comms, @robherley, suggested using this API endpoint to fetch individual job run logs.
We need to investigate this and see how we can use this endpoint, and potentially replacing the code around handling downloaded ZIP archives.
Expected outcomes