Repository navigation
flutter/packages release sometimes fails due to slow task check-in #137299
Description
Activity
- addedc: contributor-productivityTeam-specific productivity, code health, technical debt.Team-specific productivity, code health, technical debt.packageflutter/packages repository. See also p: labels.flutter/packages repository. See also p: labels.team-ecosystemOwned by Ecosystem teamOwned by Ecosystem teamfyi-infraFor the attention of Infrastructure teamFor the attention of Infrastructure team
on Oct 25, 2023 - addedtriaged-infraTriaged by Infrastructure teamTriaged by Infrastructure team
on Oct 26, 2023 - removedfyi-infraFor the attention of Infrastructure teamFor the attention of Infrastructure teamtriaged-infraTriaged by Infrastructure teamTriaged by Infrastructure team
on Oct 26, 2023 - addedP1High-priority issues at the top of the work listHigh-priority issues at the top of the work listtriaged-ecosystemTriaged by Ecosystem teamTriaged by Ecosystem team
on Oct 31, 2023 - removedtriaged-ecosystemTriaged by Ecosystem teamTriaged by Ecosystem team
on Mar 5, 2024 This issue is marked P1 but has had no recent status updates.
The P1 label indicates high-priority issues that are at the top of the work list. This is the highest priority level a bug can have if it isn't affecting a top-tier customer or breaking the build. Bugs marked P1 are generally actively being worked on unless the assignee is dealing with a P0 bug (or another P1 bug). Issues at this level should be resolved in a matter of months and should have monthly updates on GitHub.
Please consider where this bug really falls in our current priorities, and label it or assign it accordingly. This allows people to have a clearer picture of what work is actually planned. Thanks!
stuartmorgan-g commented
on Mar 5, 2024 on Mar 5, 2024 · Hidden as resolvedAuthorshow commentMore actionsstuartmorgan-g commented
on Mar 11, 2024 on Mar 11, 2024 · Hidden as resolvedAuthorshow commentMore actions7 remaining items
- removedtriaged-infraTriaged by Infrastructure teamTriaged by Infrastructure team
on Mar 14, 2024 How about adding a sleep step to the action: https://cardinalby.github.io/blog/post/github-actions/implementing-deferred-steps? This is not a real fix, but could help mitigate the issue when github webhook regression happens (before we have a long term fix).
Yes, we could do that. Do we have historical stats on how long checkin generally takes when it has these slowdown regressions? I don't want to make it sleep for too long, because while we wouldn't notice for standard triggers (since we have to wait for other tests to run anyway), it could make re-runs annoying if we sleep for too long.
Do we have historical stats on how long checkin generally takes when it has these slowdown regressions?
That really depends on how long GitHub issue gets resolved. Whenever the issue is mitigated, the task status should be posted to GitHub in seconds. We can start with a small number like 1 min to mitigate the too-fast github action. If working well, we can readjust the number further.
I just saw this happen again, I'll try adding a 60s delay to mitigate as suggested by @keyonghan!
- added a commit that references this issue
on Mar 28, 2024 I landed the timeout, but should we keep this open to track the fine-tuning of the timeout, or landing a more proper fix?
(The extra minute gets completely lost in the CI run; it's barely noticeable)
Let's leave it closed for now, and we can re-open if we have more issues later.
Reacted by David IglesiasThis thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.- locked as resolved and limited conversation to collaborators
on Apr 15, 2024 - added a commit that references this issue
on Jun 10, 2026 - added a commit that references this issue
on Jun 19, 2026
This code in the flutter/packages CI
releasestep was added to fix a problem where the following section would run before any of the LUCI checks had checked in (especially when there was an incident causing slow webhooks), causingreleaseto run before any of the tests. It's fixed that problem, but the same basic problem now manifests in a different way: sometimes the new code runs beforeLinux ci_yaml packages rollerhas checked in, causing the step to immediately fail.This is a lot better than publishing plugins that haven't passed any tests, but it's now a flaky failure, which is not ideal since it then requires manual intervention to re-run to open the tree. Ideally what we want is a step that waits until that step both exits and completes, rather than failing if it's not there yet. I don't see a way to do that with the wait action we're using though, so we may well need to build our own replacement to get that behavior.