Make iOS CPU/GPU measurement tool work for Xcode 11+ and iOS 13+

Such tool is essential for us to track and improve the Fluter power/energy usage. (Search for "percentage" on https://flutter-dashboard.appspot.com/benchmarks.html.) For example, it provides us clear metrics on how to solve the following issue that's blocking one of our very important customers: #31865.

Unfortunately, that tool currently relies on TraceUtility which doesn't work properly on Xcode 11. Moreover, we've seen some flakiness: #45194

It would be nice if we can either improve TraceUtility, or create our own version of that tool, to parse Instrument trace documents reliably with Xcode 11. That way, we'll

unblock the infra team to upgrade all our Macs to Xcode 11, and reduce the maintenance cost
reduce test flakiness and make the whole team happier
enlighten our customers by providing better and more reliable power usage metrics for improvements.

EDIT: we recently also found that TraceUtility doesn't work well with iOS 12.4 even if Xcode is still in 10.3. As a workaround, we've switched the iOS version to 12.3 to keep the test working.

We're going to have this issue every time we want to upgrade Xcode. The utility we're using is relying on private API that can change at any time.

I wonder if we could utilize something like https://github.com/rickytan/GPUUtilization?

My other suggestion also uses private APIs though so.. shrug

Some resources talking about extracting the data:

I considered the option where we created our own Instruments template and wrote the data to disk in a format of our choosing. I don't think that will work because it looks like the templates you are able to create rely on dtrace which isn't going to give us the time profiler data we'd like for example.
https://medium.com/appspector/building-custom-instruments-package-9d84fd9339b6

@gaaclarke

Discussed with @gaaclarke yesterday, and here's the summary:

We'll try to see if there's a non-hacky way of creating our own instruments template to capture the CPU/GPU percentages without relying on any private APIs.
If 1. is proved to be impossible, then we'll try to come up with a patch for https://github.com/Qusic/TraceUtility to support parsing Xcode 11 trace.
If 2. turns out to be too costly, we may find a way to change the instruments binary so it can output the result in a format defined by us, so we can easily parse it.

@liyuqian

@liyuqian Can you be more specific about doesn't work with Xcode 11 and TraceUtility? I just ran it and I get output. There isn't an obvious error to me.

Xcode 12 is bringing a tool to export traces to xml, so no more tricks will be needed when it is released

@itaybre

@itaybre Is there a link available for more information yet?

@gaaclarke

@gaaclarke you can see it here: https://developer.apple.com/documentation/xcode-release-notes/xcode-12-beta-release-notes
You will need a developer account to download the beta

xctrace(1)                BSD General Commands Manual               xctrace(1)

NAME
     xctrace -- Record, import and export Instruments .trace files.

SYNOPSIS
     xctrace [command [options]]

     xctrace help [command]

     xctrace record [--output path] [--append-run] [--template path | name]
             [--device name | UDID] [--time-limit time [ms|s|m|h]]
             [--all-processes | --attach pid | name | --launch command [arguments]]
             [--target-stdin - | file] [--target-stdout - | file]
             [--env VAR=value] [--package file]


     xctrace import [--input file] [--output path] [--template path | name]
             [--package file]

     xctrace export [--input file] [--output path]
             [--toc | --xpath expression]

     xctrace list [devices | templates]

DESCRIPTION
     xctrace is used to record new Instruments.app .trace files using the
     specified recording template.  It also allows for converting supported
     converting files of supported input types  into .trace files and export-
     ing data from .trace files to parseable formats like xml.

     Available commands and their options:

     help        General help or help specific to command argument

     record      Perform a new recording on the specified device and target
                 with the given template. Resulting .trace file can be viewed
                 in Instruments.app or exported using xctrace.  Recording can
                 target all processes, attach to an existing process or launch
                 a new one.

                 --output path     Save the trace to the specified path.  A
                                   path to an existing trace file can be used
                                   if --append-run is specified. In this case
                                   the provided template will be ignored and
                                   the template in the existing trace will be
                                   used.  If the passed path is a directory, a
                                   uniquely named file will be created in it.
                                   If the path contains the extension .trace,
                                   a new trace will be created with that name
                                   at the specified path or working directory.

                 --append-run      Allows to target an existing trace file.
                                   Specifying it causes new run to be
                                   appended. New run is recorded with the tem-
                                   plate that this existing trace file was
                                   initially recorded with.

                 --template path | name
                                   Records using given trace template name or
                                   path. If name is specified, template needs
                                   to be either bundled with an installed
                                   Instruments Package or located in the
                                   Application Support directory for Instru-
                                   ments. To retrieve available templates, run
                                   xctrace list templates.

                 --device name | UDID
                                   Record on a device with the given name or
                                   UDID. If not specified, then local device
                                   is chosen for the recording.  To get list
                                   of available devices, run xctrace list
                                   devices.

                 --time-limit time [ms|s|m|h]
                                   Limits the recording to the specified time
                                   period. Example: "--time-limit 5s" or
                                   "--time-limit 1m"

                 --all-processes   Record all processes running on the system.

                 --attach pid | name
                                   Attach and record running process with the
                                   given name or pid.

                 --launch -- command [arguments]
                                   Launch and record process with the given
                                   name or path and provided arguments list.
                                   This option should be passed as the last
                                   one in the xctrace command invocation.

                 --package file    Install the given Instruments Package tem-
                                   porarily for the duration of a command.

                 --target-stdin - | file
                                   Redirects standard input stream of a
                                   launched process in case if target device
                                   is set to the local Mac.  Can be pointing
                                   to a file or be specified as - , which
                                   causes standard input of xctrace to be
                                   passed into the standard input of a target
                                   process.

                 --target-stdout - | file
                                   Redirects standard output stream of a
                                   launched process.  Can be pointing to a
                                   file or be specified as - , which causes
                                   standard output of a target process to be
                                   to be redirected into the standard output
                                   of xctrace.

                 --env VAR=value   Sets environment variable for a launched
                                   process. Example: "--env PATH=/tmp"


     import      If the specified file is of a support input format, imports
                 the file into an Instruments .trace file using the specified
                 template. This file can later be viewed using Instruments.app
                 or exported by using xctrace export command.

                 --input file      Import data from a supported file format at
                                   the given path.

                 --output path     Save the trace to the specified path.
                                   Existing trace files cannot be targeted.
                                   If the passed path is a directory, a
                                   uniquely named file will be created in it.
                                   If the path contains the extension .trace,
                                   a new trace will be created with that name
                                   at the specified path or working directory.

                 --template path | name
                                   Imported using given trace template name or
                                   path. Only data that is needed by Instru-
                                   ments contained in the template will be
                                   imported. If name is specified, template
                                   needs to be either bundled with an
                                   installed Instruments Package or located in
                                   the Application Support directory for
                                   Instruments. To retrieve available tem-
                                   plates, run xctrace list templates.

                 --package file    Install the given Instruments Package tem-
                                   porarily for the duration of a command.

     export      Perform a new recording on the specified device and target
                 with the given template. Resulting .trace file can be viewed
                 in Instruments.app or exported using xctrace.  Recording can
                 target all processes, attach to an existing process or launch
                 a new one.

                 --input file      Export data from .trace file at the given
                                   path.

                 --output path     Save output of the export operation to the
                                   given path. If not specified, then textual
                                   output is being written to the xctrace
                                   standard output.

                 --toc             Export the table of contents of a trace
                                   file. This will contain overview of content
                                   and exportable entities from the given
                                   trace file.

                 --xpath expression
                                   Perform given XPath query on the table of
                                   contents to select entities to export from
                                   the given trace file. Selected entities
                                   will be present in the export command out-
                                   put.

     list devices
                 Lists all of non-host devices that can be used as a target
                 for a new recording.

     list templates
                 Lists all of templates that are known to These include
                 Instruments.app bundled templates, templates from installed
                 Instruments Packages and custom templates installed into
                 Application Support directory for Instruments.app.

SEE ALSO
     Instruments.app may be used to perform trace recordings in a graphical
     environment and may also be used to open trace documents created by
     xctrace.

EXAMPLES
     import --input system_logs.logarchive --template 'Custom.template'

     import --input ktrace001.trace --template 'Custom.template' --package
     '/tmp/PackageToLoad.instrdst' --output system.trace

     export --input input.trace --toc

     export --input input.trace --xpath
     '/trace-toc/run[@number="1"]/data/table[@schema="my-table-schema"]'

     record --template 'Time Profiler' --all-processes --time-limit 5s

     record --template 'Time Profiler' --device-name Chad's iPhone --attach
     'Trailblazer'

     record --template 'Time Profiler' --target-stdout - --launch -- /tmp/tool
     arg1 arg2

I had a feature request open for this. They never responded but... you're welcome 😁

I had to create my own parser after TraceUtility broke (now public here), but it is much better to have native support

As I understand it, still blocked on #45435.

As I understand it, still blocked on #45435.

That is this issue :D Did you mean to link to something else?

#63124 has verified that our CPU/GPU measurement tool is working again and our devicelab started recording valid CPU/GPU percentages since the merge of that PR.

For the record, the actual fix of this issue happened in #58803, flutter-team-archive/engine#18087, flutter-team-archive/engine#18516, flutter-team-archive/engine#18752. As we've now had them for iOS, our next step is to add them for Android (#61567).

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Make iOS CPU/GPU measurement tool work for Xcode 11+ and iOS 13+ #45435

23 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Make iOS CPU/GPU measurement tool work for Xcode 11+ and iOS 13+ #45435

Description

Activity

dnfield commented on Nov 26, 2019

dnfield commented on Nov 26, 2019

gaaclarke commented on Dec 12, 2019

gaaclarke commented on Dec 12, 2019

liyuqian commented on Dec 13, 2019

bitsydarel commented on Dec 13, 2019

gaaclarke commented on Dec 16, 2019

bitsydarel commented on Dec 16, 2019

gaaclarke commented on Dec 16, 2019

gaaclarke commented on Dec 16, 2019

23 remaining items

itaybre commented on Jun 24, 2020

gaaclarke commented on Jun 24, 2020

itaybre commented on Jun 24, 2020

jmagman commented on Jun 24, 2020

gaaclarke commented on Jun 24, 2020

itaybre commented on Jun 24, 2020

kf6gpe commented on Aug 3, 2020

chinmaygarde commented on Aug 10, 2020

liyuqian commented on Aug 10, 2020

liyuqian commented on Aug 10, 2020

github-actions commented on Aug 17, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions