Skip to content

Windows mangles double quotes within single quotes in arguments #3425

Description

@jetersen

Describe the bug

gh version 1.8.1 (2021-04-02)
https://github.com/cli/cli/releases/tag/v1.8.1

The example for gh api --jq shows that you should use single quotes for the string. '
Take this command for instance.

gh api orgs/anorg/repos --paginate --jq '[.[] | select(.default_branch == "master")] | map({ name: .full_name, branch: .default_branch })'

We spent a few minutes trying different ways of selecting and ended up figuring out that the " inside the jq string needed backlashes.
In both powershell and bash or zsh the single quotes should ensure that everything inside is not interpreted and in powershell you usually escape with backticks.

Expected vs actual behavior

The expected would be the command above but the actual command needs backslashes in powershell

gh api orgs/anorg/repos --paginate --jq '[.[] | select(.default_branch == \"master\")] | map({ name: .full_name, branch: .default_branch })'

Activity

  1. mislav commented on Apr 15, 2021

    @mislav
    Contributor

    @jetersen For me this works without backslashes in both zsh and bash.

    Would you care to elaborate why do you think double quotes need to be escaped? Within single quotes, a double quote is a literal character without any special meaning, so I do not understand why escaping would be necessary.

    Additionally, if you find one of our examples to not be working, please point us to the example from our docs and the shell in which you've found it to not be working.

    Note that not all shells escape strings the same. We are aware that not all our examples work on cmd.exe, for example. However, it's hard (or impossible) to produce examples that work in every possible shell. We focus on POSIX-compatible shells, and Powershell when possible, for our examples.

  2. jetersen commented on Apr 15, 2021

    @jetersen
    Author

    Okay, I tested it, and it seems to be a PowerShell terminal thing.
    Tested on both PowerShell 5 and 7.

  3. jetersen commented on Apr 15, 2021

    @jetersen
    Author

    Actually CMD does something completely different.

    Powershell:

    gh api orgs/anorg/repos --paginate --jq '[.[] | select(.default_branch == "master")] | map({ name: .full_name })' > test.json
    function not defined: master/0
    

    CMD:

    gh api orgs/anorg/repos --paginate --jq '[.[] | select(.default_branch == "master")] | map({ name: .full_name })' > test.json
    'select' is not recognized as an internal or external command,
    operable program or batch file.
    
  4. jetersen commented on Apr 15, 2021

    @jetersen
    Author

    Powershell and github cli is behavior is not aligned.
    Based on https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7.1
    None of the quoting works as expected.

    This includes powershell 7 on both Windows and Linux

    gh api orgs/anorg/repos --paginate --jq "[.[] | select(.default_branch == ""master"")] | map({ name: .full_name, branch: .default_branch })" > test.json
    function not defined: master/0
    gh api orgs/anorg/repos --paginate --jq "[.[] | select(.default_branch == `"master`")] | map({ name: .full_name, branch: .default_branch })" > test.json
    function not defined: master/0
    $str = @"
    [.[] | select(.default_branch == "master")] | map({ name: .full_name, branch: .default_branch })
    "@
    gh api orgs/anorg/repos --paginate --jq $str > test.json
    function not defined: master/0
    $str = @'
    [.[] | select(.default_branch == "master")] | map({ name: .full_name, branch: .default_branch })
    '@
    gh api orgs/anorg/repos --paginate --jq $str > test.json
    function not defined: master/0
  5. mislav commented on Apr 15, 2021

    @mislav
    Contributor

    Ah, thanks for bringing this to our attention.

    The docs that you've linked to explicitly state that this is valid quoting style in powershell: 'hello "beautiful" soul'. The double quotes don't need to be escaped since they are within single quotes.

    However, when powershell runs an executable (gh being one) and passes arguments to it, it apparently mangles double quotes. Apparently this was known for a long time 🤦

    So I'm not sure what to do. How should we rewrite this example so that it works both for a POSIX shell and in Powershell with the same syntax?

  6. added and removed
    more-info-neededMore info needed from user/contributor
    on Apr 15, 2021
  7. jetersen commented on Apr 15, 2021

    @jetersen
    Author

    I was not aware of this issue in PowerShell 🤔

  8. changed the title [-]quotes inside jq need to be escaped with backslashes[/-] [+]Windows mangles double quotes within single quotes in arguments[/+] on May 25, 2021
  9. jetersen commented on Jul 23, 2021

    @jetersen
    Author

    Seems this is getting fixed in powershell though most likely only for powershell 7.2 and onward

    PowerShell/PowerShell#15408

  10. added
    priority-3Affects a small number of users or is largely cosmetic
    on Nov 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocsgh-apirelating to the gh api commandpriority-3Affects a small number of users or is largely cosmetic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions