Skip to content

gh run view --log does not display logs when job names are beyond a certain length #7642

Closed
#8684
Closed
gh run view --log does not display logs when job names are beyond a certain length#7642
#8684
Labels
bugSomething isn't workinggh-runrelating to the gh run commandhelp wantedContributions welcomepriority-2Affects more than a few users but doesn't prevent core functions

Description

@williammartin
williammartin
opened on Jun 29, 2023

Describe the bug

This is a spinoff from #5011

Given a job in a workflow with a long name, gh run view --log will not display logs for that name.

Steps to reproduce the behavior

Here is a workflow definition that includes a job with a long name. Note that I accidentally left the step name as duplicate but it's not relevant.

Running gh run view 5415184943 --log produces only the logs for the shortjob:

shortjob        Set up job      2023-06-29T18:15:54.4601154Z Current runner version: '2.305.0'
shortjob        Set up job      2023-06-29T18:15:54.4659852Z ##[group]Operating System
shortjob        Set up job      2023-06-29T18:15:54.4660607Z Ubuntu
shortjob        Set up job      2023-06-29T18:15:54.4660922Z 22.04.2
shortjob        Set up job      2023-06-29T18:15:54.4661172Z LTS
shortjob        Set up job      2023-06-29T18:15:54.4661557Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4661856Z ##[group]Runner Image
shortjob        Set up job      2023-06-29T18:15:54.4662253Z Image: ubuntu-22.04
shortjob        Set up job      2023-06-29T18:15:54.4662611Z Version: 20230625.1.0
shortjob        Set up job      2023-06-29T18:15:54.4663193Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230625.1.0/images/linux/Ubuntu2204-Readme.md
shortjob        Set up job      2023-06-29T18:15:54.4663834Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230625.1.0
shortjob        Set up job      2023-06-29T18:15:54.4664318Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4664681Z ##[group]Runner Image Provisioner
shortjob        Set up job      2023-06-29T18:15:54.4664992Z 2.0.238.1
shortjob        Set up job      2023-06-29T18:15:54.4665334Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4666082Z ##[group]GITHUB_TOKEN Permissions
shortjob        Set up job      2023-06-29T18:15:54.4666664Z Contents: read
shortjob        Set up job      2023-06-29T18:15:54.4666956Z Metadata: read
shortjob        Set up job      2023-06-29T18:15:54.4667704Z Packages: read
shortjob        Set up job      2023-06-29T18:15:54.4668042Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4671985Z Secret source: Actions
shortjob        Set up job      2023-06-29T18:15:54.4672585Z Prepare workflow directory
shortjob        Set up job      2023-06-29T18:15:54.5458534Z Prepare all required actions
shortjob        Set up job      2023-06-29T18:15:54.5759678Z Complete job name: shortjob
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.6997865Z ##[group]Run echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.6998447Z echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.7617853Z shell: /usr/bin/bash -e {0}
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.7618322Z ##[endgroup]
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.8244988Z You should not see me
shortjob        Complete job    2023-06-29T18:15:54.8549991Z Cleaning up orphan processes

Expected vs actual behavior

I would expect to see logs as in the Web UI for the job with the long name.

Why does this happen?

As described in my comment when the .zip file is produced, file paths are truncated due to intrinsic length limitations of .zip files. Our regexp matching logic does not apply the same behaviour, resulting in us not matching file names to their jobs.

See the truncated Loreum Ipsum dir name below:

➜  run-log-5415184943-1688062544 ls -lah
total 32
-rw-r--r--  1 williammartin  staff   2.2K Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types (1).txt
-rw-r--r--  1 williammartin  staff   452B Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types.txt
-rw-r--r--  1 williammartin  staff   2.1K Jun 29 18:25 1_shortjob.txt
-rw-r--r--  1 williammartin  staff   454B Jun 29 18:25 2_shortjob.txt
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 shortjob

Notes

Related issues: #7640, #7641

Activity

  1. williammartin
    added
    bugSomething isn't working
    needs-triageneeds to be reviewed
    on Jun 29, 2023
  2. samcoe
    added
    priority-2Affects more than a few users but doesn't prevent core functions
    and removed
    needs-triageneeds to be reviewed
    on Jun 29, 2023
  3. williammartin
    added
    gh-runrelating to the gh run command
    coreThis issue is not accepting PRs from outside contributors
    on Oct 3, 2023

  4. shayn-orca commented on Feb 1, 2024

    @shayn-orca
    shayn-orca
    on Feb 1, 2024
    Contributor

    This is reproducing for me as well. Any plans to apply a fix? Any good workarounds in the meanwhile?

    Thanks

  5. williammartin commented on Feb 1, 2024

    @williammartin
    williammartin
    on Feb 1, 2024
    MemberAuthor

    Hey @shayn-orca,

    Hadn't prioritised anything around this because it was just theoretical and no one had actually run into it yet.

    No workaround comes to mind other than reducing the length of your job names.

    I don't think a fix for this would be particularly complicated, just truncating some names based on length. Would you be interested in opening a PR if I pointed you in the right direction?

  6. shayn-orca commented on Feb 2, 2024

    @shayn-orca
    shayn-orca
    on Feb 2, 2024
    Contributor

    Sure @williammartin - please send me some directions, I'd love to contribute to cli/cli, I use it all the time :)

  7. williammartin commented on Feb 2, 2024

    @williammartin
    williammartin
    on Feb 2, 2024
    MemberAuthor

    Well, the code I believe we're interested in starts about here:

    cli/pkg/cmd/run/view/view.go

    Lines 516 to 528 in 5a0f892

    func attachRunLog(rlz *zip.Reader, jobs []shared.Job) {
    for i, job := range jobs {
    for j, step := range job.Steps {
    re := logFilenameRegexp(job, step)
    for _, file := range rlz.File {
    if re.MatchString(file.Name) {
    jobs[i].Steps[j].Log = file
    break
    }
    }
    }
    }
    }

    This function is is looping over files in the downloaded zip and comparing it to the workflow metadata. So for each step in each job, it tries to match the name of a file in the zip via regex. The thing is that the function that produces the regex doesn't account for the server side truncation:

    cli/pkg/cmd/run/view/view.go

    Lines 495 to 497 in 5a0f892

    sanitizedJobName := strings.ReplaceAll(job.Name, "/", "")
    re := fmt.Sprintf(`%s\/%d_.*\.txt`, regexp.QuoteMeta(sanitizedJobName), step.Number)
    return regexp.MustCompile(re)

    This looks at the name of the job in the workflow, replaces any / to make composite actions work, and then produces a regex to match against. This regex needs to be updated to account for the truncation, which looks to be 89 characters on the dir name.

    Below you will find an example output of the run log after unzip:

    ➜  run-log-5415184943-1688062544 ls -lah
total 32
-rw-r--r--  1 williammartin  staff   2.2K Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types (1).txt
-rw-r--r--  1 williammartin  staff   452B Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types.txt
-rw-r--r--  1 williammartin  staff   2.1K Jun 29 18:25 1_shortjob.txt
-rw-r--r--  1 williammartin  staff   454B Jun 29 18:25 2_shortjob.txt
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 shortjob

    The regex is kind of confusing so let's look closer:

    fmt.Sprintf(`%s\/%d_.*\.txt`, regexp.QuoteMeta(sanitizedJobName), step.Number)

    This is going to produce a regex that looks like:

    shortjob\/1_.*\.txt

    This doesn't seem to match anything right? Well, inside a zip there is just a flat file structure and / is used to indicate directory hierarchy. So the zip entry this matches will actually be like shortjob/1_<stepname> but since the number makes the step unique it's not required to match that last portion, hence the .*.

    Ok so anyway, I would bet that if you added a line:

    func logFilenameRegexp(job shared.Job, step shared.Step) *regexp.Regexp {
	sanitizedJobName := strings.ReplaceAll(job.Name, "/", "")
	// sanitizedJobName = truncateString(sanitizedJobName, 89)
	re := fmt.Sprintf(`%s\/%d_.*\.txt`, regexp.QuoteMeta(sanitizedJobName), step.Number)
	return regexp.MustCompile(re)
}

    That things would start working for you again. The question is, does the server do truncation before or after / (I'm going to bet after, but we can confirm after proving our theory).

    Finally, we'll need to add a test for this. It looks like we have a test that opens up a fixture and checks how attachRunLog works, so we should add a file in there with a truncated filename as compared to a new entry in the table driven test that has a long job name:

    cli/pkg/cmd/run/view/view_test.go

    Line 1471 in 201842d

    rlz, err := zip.OpenReader("./fixtures/run_log.zip")

    Hope that makes sense and if not please let me know!

  8. williammartin
    added
    help wantedContributions welcome
    and removed
    coreThis issue is not accepting PRs from outside contributors
    on Feb 2, 2024

  9. shayn-orca commented on Feb 11, 2024

    @shayn-orca
    shayn-orca
    on Feb 11, 2024
    Contributor

    Hey @williammartin, check out #8684 please :) We've tested manually and added a UT and it seems to work OK.

  10. williammartin
    closed this as completedin #8684on May 6, 2024

  11. williammartin commented on May 6, 2024

    @williammartin
    williammartin
    on May 6, 2024
    MemberAuthor

    Using the same workflow definition as above, we now get logs from the long job name:

    shortjob        Set up job      2024-05-06T09:50:14.1206542Z Current runner version: '2.316.0'
shortjob        Set up job      2024-05-06T09:50:14.1230364Z ##[group]Operating System
shortjob        Set up job      2024-05-06T09:50:14.1231104Z Ubuntu
shortjob        Set up job      2024-05-06T09:50:14.1231498Z 22.04.4
shortjob        Set up job      2024-05-06T09:50:14.1231862Z LTS
shortjob        Set up job      2024-05-06T09:50:14.1232253Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1232669Z ##[group]Runner Image
shortjob        Set up job      2024-05-06T09:50:14.1233110Z Image: ubuntu-22.04
shortjob        Set up job      2024-05-06T09:50:14.1233548Z Version: 20240422.1.0
shortjob        Set up job      2024-05-06T09:50:14.1234563Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240422.1/images/ubuntu/Ubuntu2204-Readme.md
shortjob        Set up job      2024-05-06T09:50:14.1236013Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240422.1
shortjob        Set up job      2024-05-06T09:50:14.1236916Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1237361Z ##[group]Runner Image Provisioner
shortjob        Set up job      2024-05-06T09:50:14.1237854Z 2.0.369.1
shortjob        Set up job      2024-05-06T09:50:14.1238245Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1239254Z ##[group]GITHUB_TOKEN Permissions
shortjob        Set up job      2024-05-06T09:50:14.1240771Z Contents: read
shortjob        Set up job      2024-05-06T09:50:14.1241352Z Metadata: read
shortjob        Set up job      2024-05-06T09:50:14.1241764Z Packages: read
shortjob        Set up job      2024-05-06T09:50:14.1242457Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1245399Z Secret source: Actions
shortjob        Set up job      2024-05-06T09:50:14.1246114Z Prepare workflow directory
shortjob        Set up job      2024-05-06T09:50:14.1862882Z Prepare all required actions
shortjob        Set up job      2024-05-06T09:50:14.2091965Z Complete job name: shortjob
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3122787Z ##[group]Run echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3123535Z echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3682154Z shell: /usr/bin/bash -e {0}
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3682755Z ##[endgroup]
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.4167284Z You should not see me
shortjob        Complete job    2024-05-06T09:50:14.4439332Z Cleaning up orphan processes
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3250337Z Current runner version: '2.316.0'
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3275062Z ##[group]Operating System
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3275970Z Ubuntu
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3276328Z 22.04.4
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3276760Z LTS
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3277061Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3277456Z ##[group]Runner Image
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3277968Z Image: ubuntu-22.04
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3278330Z Version: 20240422.1.0
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3279325Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240422.1/images/ubuntu/Ubuntu2204-Readme.md
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3280815Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240422.1
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3281645Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3282063Z ##[group]Runner Image Provisioner
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3282604Z 2.0.369.1
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3282925Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3283870Z ##[group]GITHUB_TOKEN Permissions
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3285501Z Contents: read
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3285935Z Metadata: read
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3286331Z Packages: read
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3287065Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3290362Z Secret source: Actions
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3290923Z Prepare workflow directory
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3913572Z Prepare all required actions
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.4153383Z Complete job name: Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5244563Z ##[group]Run echo "You should not see me"
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5245295Z echo "You should not see me"
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5851988Z shell: /usr/bin/bash -e {0}
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5852558Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.6258664Z You should not see me
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Complete job    2024-05-06T09:50:15.6533322Z Cleaning up orphan processes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggh-runrelating to the gh run commandhelp wantedContributions welcomepriority-2Affects more than a few users but doesn't prevent core functions

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions