Skip to content

Update GitHub CLI release process to generate artifact attestations #9041

Closed
#9087
Closed
Update GitHub CLI release process to generate artifact attestations#9041
#9087
Assignees
malancas
Labels
gh-attestationrelated to the gh attestation commandtech-debtA chore that addresses technical debt

Description

@thepwagner
thepwagner
opened on May 3, 2024

Describe the feature or problem you’d like to solve

Integrate the features announced in https://github.com/cli/cli/releases/tag/v2.49.0 , to attest the artifacts attached to cli/cli releases.

The only usage of id-token: from Actions I can see looks like a regression test, https://github.com/search?q=repo%3Acli%2Fcli+%2Fattest%2F+language%3AYAML&type=code&l=YAML

Proposed solution

Call actions/attest-build-provenance with each artifact produced.
If you'd prefer to sign the digest file (1 operation vs N operations for each platform/arch), that works for me too!

  • Users of CLI will have a "live" example of a verifiiable artifact to play with.
  • Users of CLI will be able to verify the authentication of the CLI that they just downloaded.
  • Consumers of actions/attest-build-provenance data will have an ~official pattern (e.g. sign each artifact vs the digest file), that other repos may follow.

Additional context

Assumption: I'm doing this right:

$ gh attestation verify gh_2.49.0_macOS_arm64.zip -o cli
Loaded digest sha256:4c589dfabdf92d33df8bb4af474f6a5dc1945e2a67247469260104fbfd58978c for file://gh_2.49.0_macOS_arm64.zip
✗ Loading attestations from GitHub API failed

Error: failed to fetch attestations from cli: HTTP 404: Not Found (https://api.github.com/orgs/cli/attestations/sha256:4c589dfabdf92d33df8bb4af474f6a5dc1945e2a67247469260104fbfd58978c?per_page=30)

$ gh attestation verify gh_2.49.0_checksums.txt -o cli
Loaded digest sha256:ee1fa24e93d70cde0676321c57fe888dc41238b352a14aff3da2a2f7576c71a2 for file://gh_2.49.0_checksums.txt
✗ Loading attestations from GitHub API failed

Error: failed to fetch attestations from cli: HTTP 404: Not Found (https://api.github.com/orgs/cli/attestations/sha256:ee1fa24e93d70cde0676321c57fe888dc41238b352a14aff3da2a2f7576c71a2?per_page=30)

Activity

  1. thepwagner
    added
    enhancementa request to improve CLI
    on May 3, 2024
  2. cliAutomation
    added
    needs-triageneeds to be reviewed
    on May 3, 2024
  3. williammartin
    added
    gh-attestationrelated to the gh attestation command
    on May 4, 2024
  4. andyfeller
    changed the title [-]Attest release artifacts[/-] [+]Update GitHub CLI release process to generate artifact attestations[/+] on May 7, 2024
  5. andyfeller
    removed
    needs-triageneeds to be reviewed
    on May 7, 2024

  6. andyfeller commented on May 7, 2024

    @andyfeller
    andyfeller
    on May 7, 2024
    Contributor

    @malancas @phillmv : Any concerns or considerations for the various release assets we generate in order to generate artifact attestations?

    Looking at v2.49.0 release will show the various Debian and RPM packages, compiled source code, Windows exe and MSI installer:

    • GitHub CLI 2.49.0 linux 386 deb
    • GitHub CLI 2.49.0 linux 386 RPM
    • GitHub CLI 2.49.0 linux 386
    • GitHub CLI 2.49.0 linux amd64 deb
    • GitHub CLI 2.49.0 linux amd64 RPM
    • GitHub CLI 2.49.0 linux amd64
    • GitHub CLI 2.49.0 linux arm64 deb
    • GitHub CLI 2.49.0 linux arm64 RPM
    • GitHub CLI 2.49.0 linux arm64
    • GitHub CLI 2.49.0 linux armv6 deb
    • GitHub CLI 2.49.0 linux armv6 RPM
    • GitHub CLI 2.49.0 linux armv6
    • GitHub CLI 2.49.0 macOS amd64
    • GitHub CLI 2.49.0 macOS arm64
    • GitHub CLI 2.49.0 windows 386 installer
    • GitHub CLI 2.49.0 windows 386
    • GitHub CLI 2.49.0 windows amd64 installer
    • GitHub CLI 2.49.0 windows amd64
    • GitHub CLI 2.49.0 windows arm64

    Looking at the deployment workflow, I assume this would be incorporated into the release job for production environments similar to the GPG signing logic below:

    cli/.github/workflows/deployment.yml

    Lines 224 to 278 in 4896546

    release:
    runs-on: ubuntu-latest
    needs: [linux, macos, windows]
    environment: ${{ inputs.environment }}
    if: inputs.release
    steps:
    - name: Checkout cli/cli
    uses: actions/checkout@v4
    - name: Merge built artifacts
    uses: actions/download-artifact@v4
    - name: Checkout documentation site
    uses: actions/checkout@v4
    with:
    repository: github/cli.github.com
    path: site
    fetch-depth: 0
    token: ${{ secrets.SITE_DEPLOY_PAT }}
    - name: Update site man pages
    env:
    GIT_COMMITTER_NAME: cli automation
    GIT_AUTHOR_NAME: cli automation
    GIT_COMMITTER_EMAIL: [email protected]
    GIT_AUTHOR_EMAIL: [email protected]
    TAG_NAME: ${{ inputs.tag_name }}
    run: |
    git -C site rm 'manual/gh*.md' 2>/dev/null || true
    tar -xzvf linux/manual.tar.gz -C site
    git -C site add 'manual/gh*.md'
    sed -i.bak -E "s/(assign version = )\".+\"/\1\"${TAG_NAME#v}\"/" site/index.html
    rm -f site/index.html.bak
    git -C site add index.html
    git -C site diff --quiet --cached || git -C site commit -m "gh ${TAG_NAME#v}"
    - name: Prepare release assets
    env:
    TAG_NAME: ${{ inputs.tag_name }}
    run: |
    shopt -s failglob
    rm -rf dist
    mkdir dist
    mv -v {linux,macos,windows}/gh_* dist/
    - name: Install packaging dependencies
    run: sudo apt-get install -y rpm reprepro
    - name: Set up GPG
    if: inputs.environment == 'production'
    env:
    GPG_PUBKEY: ${{ secrets.GPG_PUBKEY }}
    GPG_KEY: ${{ secrets.GPG_KEY }}
    GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
    GPG_KEYGRIP: ${{ secrets.GPG_KEYGRIP }}
    run: |
    base64 -d <<<"$GPG_PUBKEY" | gpg --import --no-tty --batch --yes
    base64 -d <<<"$GPG_KEY" | gpg --import --no-tty --batch --yes
    echo "allow-preset-passphrase" > ~/.gnupg/gpg-agent.conf
    gpg-connect-agent RELOADAGENT /bye
    /usr/lib/gnupg2/gpg-preset-passphrase --preset "$GPG_KEYGRIP" <<<"$GPG_PASSPHRASE"

  7. andyfeller
    added
    tech-debtA chore that addresses technical debt
    and removed
    enhancementa request to improve CLI
    on May 7, 2024
  8. malancas
    self-assigned this
    on May 15, 2024

  9. malancas commented on May 24, 2024

    @malancas
    malancas
    on May 24, 2024
    Contributor

    @malancas @phillmv : Any concerns or considerations for the various release assets we generate in order to generate artifact attestations?

    Looking at v2.49.0 release will show the various Debian and RPM packages, compiled source code, Windows exe and MSI installer:

    • GitHub CLI 2.49.0 linux 386 deb
    • GitHub CLI 2.49.0 linux 386 RPM
    • GitHub CLI 2.49.0 linux 386
    • GitHub CLI 2.49.0 linux amd64 deb
    • GitHub CLI 2.49.0 linux amd64 RPM
    • GitHub CLI 2.49.0 linux amd64
    • GitHub CLI 2.49.0 linux arm64 deb
    • GitHub CLI 2.49.0 linux arm64 RPM
    • GitHub CLI 2.49.0 linux arm64
    • GitHub CLI 2.49.0 linux armv6 deb
    • GitHub CLI 2.49.0 linux armv6 RPM
    • GitHub CLI 2.49.0 linux armv6
    • GitHub CLI 2.49.0 macOS amd64
    • GitHub CLI 2.49.0 macOS arm64
    • GitHub CLI 2.49.0 windows 386 installer
    • GitHub CLI 2.49.0 windows 386
    • GitHub CLI 2.49.0 windows amd64 installer
    • GitHub CLI 2.49.0 windows amd64
    • GitHub CLI 2.49.0 windows arm64

    Looking at the deployment workflow, I assume this would be incorporated into the release job for production environments similar to the GPG signing logic below:

    cli/.github/workflows/deployment.yml

    Lines 224 to 278 in 4896546

    release:
    runs-on: ubuntu-latest
    needs: [linux, macos, windows]
    environment: ${{ inputs.environment }}
    if: inputs.release
    steps:
    - name: Checkout cli/cli
    uses: actions/checkout@v4
    - name: Merge built artifacts
    uses: actions/download-artifact@v4
    - name: Checkout documentation site
    uses: actions/checkout@v4
    with:
    repository: github/cli.github.com
    path: site
    fetch-depth: 0
    token: ${{ secrets.SITE_DEPLOY_PAT }}
    - name: Update site man pages
    env:
    GIT_COMMITTER_NAME: cli automation
    GIT_AUTHOR_NAME: cli automation
    GIT_COMMITTER_EMAIL: [email protected]
    GIT_AUTHOR_EMAIL: [email protected]
    TAG_NAME: ${{ inputs.tag_name }}
    run: |
    git -C site rm 'manual/gh*.md' 2>/dev/null || true
    tar -xzvf linux/manual.tar.gz -C site
    git -C site add 'manual/gh*.md'
    sed -i.bak -E "s/(assign version = )".+"/\1"${TAG_NAME#v}"/" site/index.html
    rm -f site/index.html.bak
    git -C site add index.html
    git -C site diff --quiet --cached || git -C site commit -m "gh ${TAG_NAME#v}"
    - name: Prepare release assets
    env:
    TAG_NAME: ${{ inputs.tag_name }}
    run: |
    shopt -s failglob
    rm -rf dist
    mkdir dist
    mv -v {linux,macos,windows}/gh_* dist/
    - name: Install packaging dependencies
    run: sudo apt-get install -y rpm reprepro
    - name: Set up GPG
    if: inputs.environment == 'production'
    env:
    GPG_PUBKEY: ${{ secrets.GPG_PUBKEY }}
    GPG_KEY: ${{ secrets.GPG_KEY }}
    GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
    GPG_KEYGRIP: ${{ secrets.GPG_KEYGRIP }}
    run: |
    base64 -d <<<"$GPG_PUBKEY" | gpg --import --no-tty --batch --yes
    base64 -d <<<"$GPG_KEY" | gpg --import --no-tty --batch --yes
    echo "allow-preset-passphrase" > ~/.gnupg/gpg-agent.conf
    gpg-connect-agent RELOADAGENT /bye
    /usr/lib/gnupg2/gpg-preset-passphrase --preset "$GPG_KEYGRIP" <<<"$GPG_PASSPHRASE"

    Yep, we can incorporate the provenance attestation generation in the release job for production environments.

    The generate-build-provenance Action supports attestation generation for multiple artifacts. When the Action is added, I can set the subject-path input to dist/gh*. This means the Action will generate an attestation for each artifact in the dist directory prefixed with gh.

  10. thepwagner commented on May 24, 2024

    @thepwagner
    thepwagner
    on May 24, 2024
    Author

    I can set the subject-path input to dist/gh*

    FWIW that's what we did: https://github.com/Shopify/ejson/pull/146/files
    I appreciate GitHub encoding it as a pattern for goreleaser projects 🚀 .

  11. williammartin
    closed this as completedin #9087on May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

gh-attestationrelated to the gh attestation commandtech-debtA chore that addresses technical debt

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions