Skip to content

Conclusion comment shows success when safe_outputs fails to submit PR review #30662

Closed

Description

@jonathanpeppers

Summary

When the safe_outputs job fails to submit a PR review (e.g. GitHub API returns 422), the conclusion job still updates the PR comment with "✅ completed successfully!" because it only checks Agent Conclusion and Detection Conclusion, not whether safe outputs were actually delivered.

This makes it look like the reviewer ran and found nothing, when in reality it generated comments that failed to post.

Repro

  1. Trigger an agentic workflow that uses safe-outputs with submit_pull_request_review
  2. If the GitHub API returns a transient 422 error during PR review submission, the safe_outputs job logs ##[error] but the conclusion job still reports success

Example run: https://github.com/dotnet/android/actions/runs/25446317518

Logs from safe_outputs job

Submitting PR review on dotnet/android#11254: event=COMMENT, comments=3, bodyLength=1962
POST /repos/dotnet/android/pulls/11254/reviews - 422 with id 11C0:1C2012:7B8F57:1D76664:69FB6710 in 626ms
##[error]Failed to submit PR review: Unprocessable Entity: "An internal error occurred, please try again."

Logs from conclusion job

Agent Conclusion: success
Detection Conclusion: success
Successfully updated comment

The conclusion job never checks the safe_outputs job result.

Expected behavior

The conclusion comment should show an error (e.g. ❌ or ⚠️) when safe_outputs fails to deliver its outputs, so the user knows to re-trigger the workflow.

Actual behavior

The comment shows "✅ completed successfully!" even though no review comments were posted to the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions