Skip to content

Infra: Reduce build time to perform CLR up-to-date check #47022

Open
Open
Infra: Reduce build time to perform CLR up-to-date check#47022
Labels
area-Infrastructurehelp wanted[up-for-grabs] Good issue for external contributors
Milestone
Future

Description

@agocke
agocke
opened on Jan 15, 2021

The time being measured is the time needed to build the clr subset, after the subset has been completely built already.

If this time were low enough, under 5 seconds or so, it would be feasible to add it as a dependency to other subsets. As it is, it takes roughly 51s on my machine.

Ongoing list of work items, to be updated until we're at 5 secons or less:

  • Incrementally crossgen System.Private.CoreLib
  • clretwrc appears to always be rebuilt
  • Switch to Ninja in Windows by default
    • This will require a workflow for building and preserving VS project files
  • Consider changing the default meaning of clr partition
    • Linux DAC is built by default on Windows (this is expensive. We could try to make it cheaper, or move it out of dev build)
    • CLR managed tools are built as part of CLR

Activity

  2. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Jan 15, 2021
  3. agocke
    added
    area-Infrastructure-coreclrOnly use for closed issues
    and removed
    untriagedNew issue has not been triaged by the area owner
    on Jan 15, 2021

  4. hoyosjs commented on Feb 9, 2021

    @hoyosjs
    hoyosjs
    on Feb 9, 2021
    Member

    After building with #48044, a release build up-to-date-check using build-runtime.cmd writes no files other than the CMake cache.

    Some rough numbers with the MSBuild/VS generator on my 8 core machine for an up-to-date check on release x64 fornative builds:

    • With ./build.cmd clr.runtime -rc release it takes somewhere between 40 and 50 seconds. There's a small overhead on restore of tools here.
    • With direct call into build-runtime (src/coreclr/build-runtime.cmd -release -x64 -enforcepgo) Roughly 40 seconds.
    • With direct call into build-runtime.cmd and skipping CMake setup (src/coreclr/build-runtime.cmd -release -x64 -enforcepgo -skipconfigure) 25-30 seconds. This option is not bubbled up to the root build script.
    • MSBuild takes 20s-25s of this.

    With ninja:

    • root script and build-runtime (src/coreclr/build-runtime.cmd -release -x64 -enforcepgo) take roughly 18-20 seconds.
    • build-runtime.cmd and skipping CMake setup gives 10-12 seconds.

  5. hoyosjs commented on Feb 9, 2021

    @hoyosjs
    hoyosjs
    on Feb 9, 2021
    Member

    Largely 10 seconds are used in invoking MSBuild in 3 invocations:

    • Generate a version header.
    • Restore the pgo package
    • Generate a PGO path file.

    These could arguably be munged into 1 (max two) calls.

  6. ViktorHofer commented on Feb 9, 2021

    @ViktorHofer
    ViktorHofer
    on Feb 9, 2021
    Member

    Largely 10 seconds are used in invoking MSBuild in 3 invocations:

    Are those steps running in incremental builds as well? The version header could presumably have inputs and outputs applied. Restoring via nuget is already incremental so nothing to do there.

  7. trylek
    added
    help wanted[up-for-grabs] Good issue for external contributors
    on Mar 2, 2021

  8. agocke commented on Mar 11, 2021

    @agocke
    agocke
    on Mar 11, 2021
    MemberAuthor

    @hoyosjs Looking at the log, when I run build clr -ninja I still see

    Read file version from native version header at 'C:/Users/andy/Code/runtime/src/coreclr/../../artifacts/obj/coreclr/_version.h'.
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/andy/Code/runtime/artifacts/nmakeobj/windows.x64.Debug
[1/7] Linking RC shared library dlls\clretwrc\clretwrc.dll
[2/7] Linking RC shared library dlls\mscorrc\mscorrc.dll

    I wasn't expecting us to have to re-link clretwrc. Any ideas?

  9. hoyosjs commented on Mar 11, 2021

    @hoyosjs
    hoyosjs
    on Mar 11, 2021
    Member

    Tracked it down I think. Let me test if it's that.

  10. hoyosjs commented on Mar 12, 2021

    @hoyosjs
    hoyosjs
    on Mar 12, 2021
    Member

    Hmm. Looks like what I was seeing will only affect non-Windows system. Will try to see where this is coming from.

  11. ViktorHofer
    added this to the 7.0.0 milestone on Sep 7, 2021

  12. ericstj commented on Apr 7, 2022

    @ericstj
    ericstj
    on Apr 7, 2022
    Member

    Just imaged a brand new machine and was measuring build performance. On a repo with zero changes rebuild of CLR subset is taking over a minute and a half, while build of libs is about 20s.

    It looks to me like CLR is rebuilding a bunch of stuff. I put up a log at \bvtsrv2\team\ericstj\incremental-build-clr

  13. hoyosjs commented on Apr 8, 2022

    @hoyosjs
    hoyosjs
    on Apr 8, 2022
    Member

    Looks like this took a 1:15
    "C:\src\dotnet\runtime\src\coreclr\build-runtime.cmd" -x64 -release -pgodatapath "C:\Users\ericstj\.nuget\packages\optimization.windows_nt-x64.pgo.coreclr\1.0.0-prerelease.22121.2"

  14. agocke
    added this to Runtime Infraon Jul 19, 2022
  15. agocke
    modified the milestones: 7.0.0, Future on Jul 26, 2022
  16. sbomer
    added a commit that references this issue on Jun 9, 2023

    Incremental mibc merge (#87336)

    454057d
  17. agocke
    added
    area-Infrastructure
    and removed
    area-Infrastructure-coreclrOnly use for closed issues
    on Apr 13, 2026

  18. dotnet-policy-service commented on Apr 13, 2026

    @dotnet-policy-service
    dotnet-policy-servicebot
    on Apr 13, 2026 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @dotnet/runtime-infrastructure
    See info in area-owners.md if you want to be subscribed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-Infrastructurehelp wanted[up-for-grabs] Good issue for external contributors

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions