Skip to content

Make iOS CPU/GPU measurement tool work for Xcode 11+ and iOS 13+ #45435

Description

@liyuqian

Such tool is essential for us to track and improve the Fluter power/energy usage. (Search for "percentage" on https://flutter-dashboard.appspot.com/benchmarks.html.) For example, it provides us clear metrics on how to solve the following issue that's blocking one of our very important customers: #31865.

Unfortunately, that tool currently relies on TraceUtility which doesn't work properly on Xcode 11. Moreover, we've seen some flakiness: #45194

It would be nice if we can either improve TraceUtility, or create our own version of that tool, to parse Instrument trace documents reliably with Xcode 11. That way, we'll

  1. unblock the infra team to upgrade all our Macs to Xcode 11, and reduce the maintenance cost
  2. reduce test flakiness and make the whole team happier
  3. enlighten our customers by providing better and more reliable power usage metrics for improvements.

EDIT: we recently also found that TraceUtility doesn't work well with iOS 12.4 even if Xcode is still in 10.3. As a workaround, we've switched the iOS version to 12.3 to keep the test working.

Activity

  1. added
    c: contributor-productivityTeam-specific productivity, code health, technical debt.
    c: performanceRelates to speed or footprint issues (see "perf:" labels)
    team-infraOwned by Infrastructure team
    on Nov 22, 2019
  2. dnfield commented on Nov 26, 2019

    @dnfield
    Contributor

    We're going to have this issue every time we want to upgrade Xcode. The utility we're using is relying on private API that can change at any time.

    I wonder if we could utilize something like https://github.com/rickytan/GPUUtilization?

  3. dnfield commented on Nov 26, 2019

    @dnfield
    Contributor

    My other suggestion also uses private APIs though so.. shrug

  4. added
    perf: energyPerformance issues related to energy use (power, watts)
    on Dec 2, 2019
  5. gaaclarke commented on Dec 12, 2019

    @gaaclarke
    Member
  6. gaaclarke commented on Dec 12, 2019

    @gaaclarke
    Member

    I considered the option where we created our own Instruments template and wrote the data to disk in a format of our choosing. I don't think that will work because it looks like the templates you are able to create rely on dtrace which isn't going to give us the time profiler data we'd like for example.
    https://medium.com/appspector/building-custom-instruments-package-9d84fd9339b6

  7. liyuqian commented on Dec 13, 2019

    @liyuqian
    ContributorAuthor

    Discussed with @gaaclarke yesterday, and here's the summary:

    1. We'll try to see if there's a non-hacky way of creating our own instruments template to capture the CPU/GPU percentages without relying on any private APIs.
    2. If 1. is proved to be impossible, then we'll try to come up with a patch for https://github.com/Qusic/TraceUtility to support parsing Xcode 11 trace.
    3. If 2. turns out to be too costly, we may find a way to change the instruments binary so it can output the result in a format defined by us, so we can easily parse it.
  8. bitsydarel commented on Dec 13, 2019

    @bitsydarel
  9. gaaclarke commented on Dec 16, 2019

    @gaaclarke
  10. bitsydarel commented on Dec 16, 2019

    @bitsydarel
  11. gaaclarke commented on Dec 16, 2019

    @gaaclarke
  12. gaaclarke commented on Dec 16, 2019

    @gaaclarke
    Member

    @liyuqian Can you be more specific about doesn't work with Xcode 11 and TraceUtility? I just ran it and I get output. There isn't an obvious error to me.

  13. changed the title [-]Make iOS CPU/GPU measurement tool work for Xcode 11[/-] [+]Make iOS CPU/GPU measurement tool work for Xcode 11 or iOS 12.4[/+] on Dec 16, 2019
  14. 23 remaining items

  15. itaybre commented on Jun 24, 2020

    @itaybre

    Xcode 12 is bringing a tool to export traces to xml, so no more tricks will be needed when it is released

  16. gaaclarke commented on Jun 24, 2020

    @gaaclarke
    Member

    @itaybre Is there a link available for more information yet?

  17. itaybre commented on Jun 24, 2020

    @itaybre

    @gaaclarke you can see it here: https://developer.apple.com/documentation/xcode-release-notes/xcode-12-beta-release-notes
    You will need a developer account to download the beta

  18. jmagman commented on Jun 24, 2020

    @jmagman
    Member
    xctrace(1)                BSD General Commands Manual               xctrace(1)
    
    NAME
         xctrace -- Record, import and export Instruments .trace files.
    
    SYNOPSIS
         xctrace [command [options]]
    
         xctrace help [command]
    
         xctrace record [--output path] [--append-run] [--template path | name]
                 [--device name | UDID] [--time-limit time [ms|s|m|h]]
                 [--all-processes | --attach pid | name | --launch command [arguments]]
                 [--target-stdin - | file] [--target-stdout - | file]
                 [--env VAR=value] [--package file]
    
    
         xctrace import [--input file] [--output path] [--template path | name]
                 [--package file]
    
         xctrace export [--input file] [--output path]
                 [--toc | --xpath expression]
    
         xctrace list [devices | templates]
    
    DESCRIPTION
         xctrace is used to record new Instruments.app .trace files using the
         specified recording template.  It also allows for converting supported
         converting files of supported input types  into .trace files and export-
         ing data from .trace files to parseable formats like xml.
    
         Available commands and their options:
    
         help        General help or help specific to command argument
    
         record      Perform a new recording on the specified device and target
                     with the given template. Resulting .trace file can be viewed
                     in Instruments.app or exported using xctrace.  Recording can
                     target all processes, attach to an existing process or launch
                     a new one.
    
                     --output path     Save the trace to the specified path.  A
                                       path to an existing trace file can be used
                                       if --append-run is specified. In this case
                                       the provided template will be ignored and
                                       the template in the existing trace will be
                                       used.  If the passed path is a directory, a
                                       uniquely named file will be created in it.
                                       If the path contains the extension .trace,
                                       a new trace will be created with that name
                                       at the specified path or working directory.
    
                     --append-run      Allows to target an existing trace file.
                                       Specifying it causes new run to be
                                       appended. New run is recorded with the tem-
                                       plate that this existing trace file was
                                       initially recorded with.
    
                     --template path | name
                                       Records using given trace template name or
                                       path. If name is specified, template needs
                                       to be either bundled with an installed
                                       Instruments Package or located in the
                                       Application Support directory for Instru-
                                       ments. To retrieve available templates, run
                                       xctrace list templates.
    
                     --device name | UDID
                                       Record on a device with the given name or
                                       UDID. If not specified, then local device
                                       is chosen for the recording.  To get list
                                       of available devices, run xctrace list
                                       devices.
    
                     --time-limit time [ms|s|m|h]
                                       Limits the recording to the specified time
                                       period. Example: "--time-limit 5s" or
                                       "--time-limit 1m"
    
                     --all-processes   Record all processes running on the system.
    
                     --attach pid | name
                                       Attach and record running process with the
                                       given name or pid.
    
                     --launch -- command [arguments]
                                       Launch and record process with the given
                                       name or path and provided arguments list.
                                       This option should be passed as the last
                                       one in the xctrace command invocation.
    
                     --package file    Install the given Instruments Package tem-
                                       porarily for the duration of a command.
    
                     --target-stdin - | file
                                       Redirects standard input stream of a
                                       launched process in case if target device
                                       is set to the local Mac.  Can be pointing
                                       to a file or be specified as - , which
                                       causes standard input of xctrace to be
                                       passed into the standard input of a target
                                       process.
    
                     --target-stdout - | file
                                       Redirects standard output stream of a
                                       launched process.  Can be pointing to a
                                       file or be specified as - , which causes
                                       standard output of a target process to be
                                       to be redirected into the standard output
                                       of xctrace.
    
                     --env VAR=value   Sets environment variable for a launched
                                       process. Example: "--env PATH=/tmp"
    
    
         import      If the specified file is of a support input format, imports
                     the file into an Instruments .trace file using the specified
                     template. This file can later be viewed using Instruments.app
                     or exported by using xctrace export command.
    
                     --input file      Import data from a supported file format at
                                       the given path.
    
                     --output path     Save the trace to the specified path.
                                       Existing trace files cannot be targeted.
                                       If the passed path is a directory, a
                                       uniquely named file will be created in it.
                                       If the path contains the extension .trace,
                                       a new trace will be created with that name
                                       at the specified path or working directory.
    
                     --template path | name
                                       Imported using given trace template name or
                                       path. Only data that is needed by Instru-
                                       ments contained in the template will be
                                       imported. If name is specified, template
                                       needs to be either bundled with an
                                       installed Instruments Package or located in
                                       the Application Support directory for
                                       Instruments. To retrieve available tem-
                                       plates, run xctrace list templates.
    
                     --package file    Install the given Instruments Package tem-
                                       porarily for the duration of a command.
    
         export      Perform a new recording on the specified device and target
                     with the given template. Resulting .trace file can be viewed
                     in Instruments.app or exported using xctrace.  Recording can
                     target all processes, attach to an existing process or launch
                     a new one.
    
                     --input file      Export data from .trace file at the given
                                       path.
    
                     --output path     Save output of the export operation to the
                                       given path. If not specified, then textual
                                       output is being written to the xctrace
                                       standard output.
    
                     --toc             Export the table of contents of a trace
                                       file. This will contain overview of content
                                       and exportable entities from the given
                                       trace file.
    
                     --xpath expression
                                       Perform given XPath query on the table of
                                       contents to select entities to export from
                                       the given trace file. Selected entities
                                       will be present in the export command out-
                                       put.
    
         list devices
                     Lists all of non-host devices that can be used as a target
                     for a new recording.
    
         list templates
                     Lists all of templates that are known to These include
                     Instruments.app bundled templates, templates from installed
                     Instruments Packages and custom templates installed into
                     Application Support directory for Instruments.app.
    
    SEE ALSO
         Instruments.app may be used to perform trace recordings in a graphical
         environment and may also be used to open trace documents created by
         xctrace.
    
    EXAMPLES
         import --input system_logs.logarchive --template 'Custom.template'
    
         import --input ktrace001.trace --template 'Custom.template' --package
         '/tmp/PackageToLoad.instrdst' --output system.trace
    
         export --input input.trace --toc
    
         export --input input.trace --xpath
         '/trace-toc/run[@number="1"]/data/table[@schema="my-table-schema"]'
    
         record --template 'Time Profiler' --all-processes --time-limit 5s
    
         record --template 'Time Profiler' --device-name Chad's iPhone --attach
         'Trailblazer'
    
         record --template 'Time Profiler' --target-stdout - --launch -- /tmp/tool
         arg1 arg2
    
  19. gaaclarke commented on Jun 24, 2020

    @gaaclarke
    Member

    I had a feature request open for this. They never responded but... you're welcome 😁

  20. itaybre commented on Jun 24, 2020

    @itaybre

    I had to create my own parser after TraceUtility broke (now public here), but it is much better to have native support

  21. kf6gpe commented on Aug 3, 2020

    @kf6gpe
    Contributor

    As I understand it, still blocked on #45435.

  22. chinmaygarde commented on Aug 10, 2020

    @chinmaygarde
    Contributor

    As I understand it, still blocked on #45435.

    That is this issue :D Did you mean to link to something else?

  23. liyuqian commented on Aug 10, 2020

    @liyuqian
    ContributorAuthor

    #63124 has verified that our CPU/GPU measurement tool is working again and our devicelab started recording valid CPU/GPU percentages since the merge of that PR.

  24. liyuqian commented on Aug 10, 2020

    @liyuqian
    ContributorAuthor

    For the record, the actual fix of this issue happened in #58803, flutter-team-archive/engine#18087, flutter-team-archive/engine#18516, flutter-team-archive/engine#18752. As we've now had them for iOS, our next step is to add them for Android (#61567).

  25. github-actions commented on Aug 17, 2021

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  26. locked as resolved and limited conversation to collaborators on Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    c: contributor-productivityTeam-specific productivity, code health, technical debt.c: performanceRelates to speed or footprint issues (see "perf:" labels)customer: alibabaperf: energyPerformance issues related to energy use (power, watts)team-infraOwned by Infrastructure team

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions