Skip to content

Exporting gh secret list should always provide number of selected repositories #8679

Description

@nvincent-vossloh

Describe the bug

When using gh secret list -o <org> --json 'name,numSelectedRepos' and sending the output to a file or to another process, the integers from numSelectedRepos are set to 0. However if the standard output is the interactive terminal, then the values are correct

gh version: 2.43.1, happens all the way back to 2.36.0 when --json option appeared.

I have only seen this behavior with gh secret list, I could not reproduce it with gh issue -R zephyrproject-rtos/zephyr list --json 'number' | cat - for instance.

Steps to reproduce the behavior

  1. Create a github organisation dummyOrg
  2. Create at least one repository inside dummyOrg
  3. Create a secret at the organisation level and share that secret with the repository created in previous step
  4. List secret at organisation level with json formatting: gh secret list -o dummyOrg --json 'name,numSelectedRepos' | cat -
  5. the numSelectedRepos field is set to zero instead of 1.

Expected vs actual behavior

I am expecting the numSelectedRepos field to have a consistent value when piping the output of gh secret list command or sending it to the terminal

Logs

Sending output to terminal (expected behavior)

gh secret list -o dummyOrg --json 'name,numSelectedRepos'
[
  {
    "name": "MYSECRET",
    "numSelectedRepos": 1
  }
]

Sending output to another process (unexpected behavior)

gh secret list -o dummyOrg --json 'name,numSelectedRepos' | cat -
[{"name":"MYSECRET","numSelectedRepos":0}]

The numSelectedRepos has been converted to 0, the same unexpected behavior occurs with jq instead of cat - or sending the output to a file ( > file)

Activity

  1. andyfeller commented on Feb 12, 2024

    @andyfeller
    Contributor

    @nvincent-vossloh : thank you for opening up this issue! 🙇

    After digging into the code a bit, I think I know where this is happening, which dates back to the origin of the command in #4714:

    var secrets []Secret
    showSelectedRepoInfo := opts.IO.IsStdoutTTY()
    switch secretEntity {
    case shared.Repository:
    secrets, err = getRepoSecrets(client, baseRepo, secretApp)
    case shared.Environment:
    secrets, err = getEnvSecrets(client, baseRepo, envName)
    case shared.Organization, shared.User:
    var cfg config.Config
    var host string
    cfg, err = opts.Config()
    if err != nil {
    return err
    }
    host, _ = cfg.Authentication().DefaultHost()
    if secretEntity == shared.User {
    secrets, err = getUserSecrets(client, host, showSelectedRepoInfo)
    } else {
    secrets, err = getOrgSecrets(client, host, orgName, showSelectedRepoInfo, secretApp)
    }
    }

    The code currently uses whether a TTY to show selected repository information or not. I'm going to follow up regarding the concern and if there are any issues with relaxing that requirement. 👍

    Update

    TLDR: We could relax this requirement, but it will increase API requests used. Additionally, it appears gh secret list will only pull up to 100 secrets.

    Once secrets are retrieved, secondary enrichment of secrets will attempt to add number of repositories selected if relevant and depending on TTY. Depending on whether you are using a PAT versus a GitHub App installation access token, that will just quicken the request per hour used.

    func populateSelectedRepositoryInformation(client *http.Client, host string, secrets []Secret) error {
    apiClient := api.NewClientFromHTTP(client)
    for i, secret := range secrets {
    if secret.SelectedReposURL == "" {
    continue
    }
    response := struct {
    TotalCount int `json:"total_count"`
    }{}
    if err := apiClient.REST(host, "GET", secret.SelectedReposURL, nil, &response); err != nil {
    return fmt.Errorf("failed determining selected repositories for %s: %w", secret.Name, err)
    }
    secrets[i].NumSelectedRepos = response.TotalCount
    }
    return nil
    }

    Lastly, it appears gh secret list is hard coded to pull a single 100 page of secrets from the API. In the case of limits of GitHub Actions secrets, an organization can have up to 1,000.

    func getSecrets(client *http.Client, host, path string) ([]Secret, error) {
    var results []Secret
    apiClient := api.NewClientFromHTTP(client)
    path = fmt.Sprintf("%s?per_page=100", path)
    for path != "" {
    response := struct {
    Secrets []Secret
    }{}
    var err error
    path, err = apiClient.RESTWithNext(host, "GET", path, nil, &response)
    if err != nil {
    return nil, err
    }
    results = append(results, response.Secrets...)
    }
    return results, nil
    }

  2. added
    priority-3Affects a small number of users or is largely cosmetic
    gh-secretrelating to the gh secret command
    and removed on Feb 12, 2024
  3. changed the title [-]inconsistent json output for integer values when stdout sent to a pipe[/-] [+]Exporting `gh secret list` should always provide number of selected repositories[/+] on Feb 12, 2024
  4. babakks commented on Apr 1, 2024

    @babakks
    Member

    @andyfeller I just submitted a PR to fix this. Thanks for the hint.

  5. babakks commented on Apr 1, 2024

    @babakks
    Member

    Lastly, it appears gh secret list is hard coded to pull a single 100 page of secrets from the API. In the case of limits of GitHub Actions secrets, an organization can have up to 1,000.

    @andyfeller But that happens in a loop, so it'll fetch all secrets, right? Or I'm missing something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggh-secretrelating to the gh secret commandhelp wantedContributions welcomepriority-3Affects a small number of users or is largely cosmetic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions