gh tries to use hosts.yml after initial boot and fails, then tries keyring and succeeds

Describe the bug

Immediately after boot, any attempt to use the gh cli commands fails as unauthorized. Any second attempt always works. This is because gh tries to use hosts.yml initiallity for authorization, and it says the (nonexistent) token is invalid. On second attempts it correctly looks in the system keyring, finds the credentials, and succeeds.

gh version 2.45.0 (1980-01-01)
https://github.com/cli/cli/releases/tag/v2.45.0

OS: NixOS 24.05
DE: KDE Plasma 6.0.1

Steps to reproduce the behavior

Boot system
Run gh auth status once:

❯ gh auth status
github.com
  X Failed to log in to github.com account XXXXXXX (/home/XXXXXXX/.config/gh/hosts.yml)
  - Active account: true
  - The token in /home/XXXXXXX/.config/gh/hosts.yml is invalid.
  - To re-authenticate, run: gh auth login -h github.com
  - To forget about this account, run: gh auth logout -h github.com -u tdhooten

Run gh auth status again:

❯ gh auth status
github.com
  ✓ Logged in to github.com account XXXXXXX (keyring)
  - Active account: true
  - Git operations protocol: https
  - Token: gho_************************************
  - Token scopes: 'gist', 'read:org', 'repo', 'workflow'

Expected vs actual behavior

Expected: it should work the first time.
Actual: it only works the second time.

This has had some discussion over in #8729 (comment) but I'm not 100% sure these issues are the same.

I don't think the title is quite correct, I think the error message is an artifact of gh falling back to hosts.yml after failing to use the keyring but this is still to be confirmed. It's interesting that this has come up a few times in the past few weeks, I don't think we changed anything around the secret handling, and I don't think we updated any dependencies there but I'll have to check.

Would you be willing to try a few older versions to see whether this issue persists? I wouldn't go back before 2.40.0 unless you're ok with running into some strangeness around the multi-account switchover (which might require you to delete your config.yml and hosts.yml file to start afresh).

In either case I'm definitely going to look into this a bit more today, I wasn't able to reproduce it on a VM last week but that could have been user error 😅

It looks to me like it is probably the same issue. FWIW, I am using libsecret as the backend for git-credential-helper. I don't know if gh relies on git's configuration or not.

gh uses https://github.com/zalando/go-keyring to interact with the system keyring directly, beyond that I'll need to do more digging.

I am having the exact same issue as OP.

If this is of any help, the issue happens for me on a Fedora linux machine using KDE plasma (and thus the keyring is managed by KDE wallet I think?).

I have a second Fedora linux machine using GNOME and this is not an issue there. I thought maybe the GNOME keyring and KDE wallet do something slightly different and that causes these issues?

Yes it seems like everyone affected is running KDE.

If y'all run busctl --user | grep secrets what do you see?

I created a VM today running Debian and using KDE, I configured the secret service to be accessible via dbus (but I'm not sure if I did it totally in line with your configuration, hence my question above ^). I was not able to replicate this.

In the mean time, another idea I had was to create a debug version of the CLI that would expose the error that I believe is currently being hidden. If you clone https://github.com/cli/cli/tree/wm/debug-8802 and run make, you should then be able to run ./bin/gh auth debug and it will either print the error or say that everything is good i.e.:

➜  cli git:(wm/debug-8802) ./bin/gh auth debug
everything seems good :)
...
➜  cli git:(wm/debug-8802) ./bin/gh auth debug
secret not found in keyring

If you are unable to build the CLI locally, let me know your architecture and I can build a binary for you.

If y'all run busctl --user | grep secrets what do you see?

org.freedesktop.secrets 1989 .kwalletd6-wrap USERNAME :1.41 [email protected] -

I created a VM today running Debian and using KDE, I configured the secret service to be accessible via dbus (but I'm not sure if I did it totally in line with your configuration, hence my question above ^). I was not able to replicate this.

In the mean time, another idea I had was to create a debug version of the CLI that would expose the error that I believe is currently being hidden. If you clone https://github.com/cli/cli/tree/wm/debug-8802 and run make, you should then be able to run ./bin/gh auth debug and it will either print the error or say that everything is good i.e.:
➜  cli git:(wm/debug-8802) ./bin/gh auth debug
everything seems good :)
...
➜  cli git:(wm/debug-8802) ./bin/gh auth debug
secret not found in keyring
If you are unable to build the CLI locally, let me know your architecture and I can build a binary for you.

I get a repository not found error when trying to clone. I'm on amd64.

I get a repository not found error when trying to clone. I'm on amd64.

gh repo clone cli/cli && cd cli && git checkout wm/debug-8802 && make

I get a repository not found error when trying to clone. I'm on amd64.
gh repo clone cli/cli && cd cli && git checkout wm/debug-8802 && make

First run: timeout while trying to get secret from keyring

Second run: everything seems good :)

Interesting. I pushed a commit that bumps the timeout to a ridiculous amount if you can pull, make and try again.

Specifically, I'd like to know whether it's hanging forever.

@tdhooten

@tdhooten Please pull, build and and try again, I had one wild idea before giving up for this evening. Looking at the code for the zalando secret service I see a request to prompt, then registration of a rule to match on the response. However, in the dbus output you sent me we see that the prompt response comes back, and then the rule match is added:

method call time=1710441095.362970 sender=:1.77 -> destination=org.freedesktop.DBus serial=5 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch
   string "type='signal',path='/org/freedesktop/secrets/prompt/p0',interface='org.freedesktop.Secret.Prompt'"

signal time=1710441095.362967 sender=:1.18 -> destination=:1.77 serial=44 path=/org/freedesktop/secrets/prompt/p0; interface=org.freedesktop.Secret.Prompt; member=Completed
   boolean false
   variant       array [
         object path "/org/freedesktop/secrets/aliases/default"
      ]

I don't know enough about dbus rule matching to know the exact semantics but it does look kind of racy to me. I made an attempt to swap the ordering in a fork: williammartin/go-keyring@928f5e8

Since I'm not sure how to get my own system to request a prompt, I haven't been able to test this this evening so it may just fall over or deadlock in some other way but I thought it was worth an attempt in case you are around.

I added a couple of prints that you may see in the output.

@tdhooten

@tdhooten Please pull, build and and try again, I had one wild idea before giving up for this evening. Looking at the code for the zalando secret service I see a request to prompt, then registration of a rule to match on the response. However, in the dbus output you sent me we see that the prompt response comes back, and then the rule match is added:
method call time=1710441095.362970 sender=:1.77 -> destination=org.freedesktop.DBus serial=5 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch
   string "type='signal',path='/org/freedesktop/secrets/prompt/p0',interface='org.freedesktop.Secret.Prompt'"

signal time=1710441095.362967 sender=:1.18 -> destination=:1.77 serial=44 path=/org/freedesktop/secrets/prompt/p0; interface=org.freedesktop.Secret.Prompt; member=Completed
   boolean false
   variant       array [
         object path "/org/freedesktop/secrets/aliases/default"
      ]
I don't know enough about dbus rule matching to know the exact semantics but it does look kind of racy to me. I made an attempt to swap the ordering in a fork: williammartin/go-keyring@928f5e8

Since I'm not sure how to get my own system to request a prompt, I haven't been able to test this this evening so it may just fall over or deadlock in some other way but I thought it was worth an attempt in case you are around.

I added a couple of prints that you may see in the output.

That did it!

unlocking
Adding match signal
Prompting
everything seems good :)

@tdhooten

@tdhooten wow. That would be quite a solve if we got it. Thank you for your active involvement here, I could not have done it. I suspect what's happening is you have some config that auto unlocks instead of offering the prompt and it happens so fast it's racy. Perhaps you have something like PAM configured: https://wiki.archlinux.org/title/KDE_Wallet

@yutanagano @siniarskimar @avivace @glenfletcher would any of you be willing to build and test this change out as described here: #8802 (comment)

gh repo clone cli/cli && cd cli && git checkout wm/debug-8802 && make

Then after restart try ./bin/gh auth status from the cli directory. Where you would expect gh auth status to have previously failed, I would now expect it to succeed.

I suspect what's happening is you have some config that auto unlocks instead of offering the prompt and it happens so fast it's racy.

Yes, I do indeed have the wallet set to auto-unlock. The weird thing is that I have tried manually opening the wallet to make sure it's ready before first running gh commands, and yet it would still fail the first time.

Anyway, thank you so much for your work on this.

Seems to be fixed on my end. Did three reboots to make sure and gh auth status works first-time every time.

@williammartin

I also gave it a try and can confirm that this new build works where the current version does not. Thank you so much @williammartin for the very speedy response and fix, and @tdhooten for active testing :) you guys are awesome

@siniarskimar

@siniarskimar @yutanagano thank you very much for taking the time to try this out.

I've created an issue and PR to the zalando/go-keyring library that we depend upon. I'm not sure what their maintenance approach is for this library. Hopefully the maintainers will be responsive as they have accepted contributions from us in the past. If we don't get a timely response then we have two options:

Fork the library into cli/go-keyring
Proceed with an open PR that moves away from this library

I'm hopeful that we'll get a review because I have limited confidence that I didn't accidentally break something else with this change! 😅

I'll update this issue when we know more.

The folks over at zalando have agreed with our assessment and have merged the PR. I've asked whether they will cut a release or if we should pin to the sha. Ideally we could get this fixed in our next release which on our usual schedule would be on Tuesday.

The folks over at zalando have agreed with our assessment and have merged the PR. I've asked whether they will cut a release or if we should pin to the sha. Ideally we could get this fixed in our next release which on our usual schedule would be on Tuesday.

Looks like they already pushed a new release!

Great job finding this fix so quickly, I really appreciate your work.

I've opened a PR here to bump to the new release of go-keyring: #8833

We released v2.46.0 today which includes the fix. If you run into issues please re-open.

Navigation Menu

Repository navigation

gh tries to use hosts.yml after initial boot and fails, then tries keyring and succeeds #8802

Describe the bug

Steps to reproduce the behavior

Expected vs actual behavior

6 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

gh tries to use hosts.yml after initial boot and fails, then tries keyring and succeeds #8802

Description

Describe the bug

Steps to reproduce the behavior

Expected vs actual behavior

Activity

williammartin commented on Mar 11, 2024

williammartin commented on Mar 11, 2024

tdhooten commented on Mar 11, 2024

williammartin commented on Mar 11, 2024

yutanagano commented on Mar 13, 2024

tdhooten commented on Mar 13, 2024

williammartin commented on Mar 13, 2024

williammartin commented on Mar 13, 2024

tdhooten commented on Mar 13, 2024

tdhooten commented on Mar 13, 2024

williammartin commented on Mar 14, 2024

tdhooten commented on Mar 14, 2024

williammartin commented on Mar 14, 2024

6 remaining items

williammartin commented on Mar 14, 2024

tdhooten commented on Mar 14, 2024

williammartin commented on Mar 14, 2024

tdhooten commented on Mar 14, 2024

siniarskimar commented on Mar 14, 2024

yutanagano commented on Mar 15, 2024

williammartin commented on Mar 15, 2024

williammartin commented on Mar 16, 2024

tdhooten commented on Mar 16, 2024

williammartin commented on Mar 18, 2024

williammartin commented on Mar 20, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions