Add an option to "flutter test/drive/analyze" to make output more machine readable #14134

Activity

1. mjohnsullivan

   added

   customer: alibaba

   on Jan 17, 2018
2. Hixie commented on Jan 17, 2018

   

   Hixie

   on Jan 17, 2018

   Contributor

   More actions

   Ideally not JSON...

   The current output is machine readable (in fact, it's machine-read; that's why it's still using [hint] and similar syntax instead of the bullets and such that the dartanalyzer uses by default). There's also a mode for dartanalyzer (which you can invoke directly) that is intended specifically to be read by machines. Are either of those sufficient? If not, can you elaborate on the use case?
3. mjohnsullivan commented on Jan 17, 2018

   

   mjohnsullivan

   on Jan 17, 2018

   ContributorAuthor

   More actions

   The ask is to have the output in an easier format/structure so that it's easier to integrate into a CI system. The current output (as far as I can see, unless there's other ways to format via flags, etc.) is not the easiest to parse.
4. Hixie commented on Jan 18, 2018

   

   Hixie

   on Jan 18, 2018

   Contributor

   More actions

   Can you elaborate on "integrate into a CI system"? What are you going to do with the data? What fields do you need? It's hard to know how to address the request without knowing the full use case.
5. kangwang1988 commented on Jan 18, 2018

   

   kangwang1988

   on Jan 18, 2018

   Last edited by devoncarew

   Contributor

   More actions

   @Hixie Now when I run flutter analyze for flutter package of the master branch, the console output is like below:

   [warning] The method 'launch' isn't defined for the class 'UpdaterState'. (/Users/kylewong/Codes/Flutter/flutter_master/flutter/examples/flutter_gallery/lib/gallery/updates.dart, line 46, col 9)
   [warning] The method 'launch' isn't defined for the class 'GalleryDrawer'. (/Users/kylewong/Codes/Flutter/flutter_master/flutter/examples/flutter_gallery/lib/gallery/drawer.dart, line 224, col 9)
   [error] Target of URI doesn't exist: 'package:url_launcher/url_launcher.dart'. (/Users/kylewong/Codes/Flutter/flutter_master/flutter/examples/flutter_gallery/lib/gallery/drawer.dart, line 11, col 8)
   [warning] The method 'launch' isn't defined for the class 'LinkTextSpan'. (/Users/kylewong/Codes/Flutter/flutter_master/flutter/examples/flutter_gallery/lib/gallery/drawer.dart, line 32, col 7)
   [lint] 2 public members lack documentation (ran in 37.7s)
   

   Do you mean the right approach for us to know how much passed/failed is to analyze the formated message line by line？ Extract type(warning/error), message, file and line num from each line?

   As to the flutter test, the message as below:

   00:02 +22 -1: 00:01 +24 -1: /Users/kylewong/Codes/Flutter/flutter_master/flutter/packages/flutter/test/animation/animation_controller_test.dart: velocity test - linear
   02:17 +2177 ~16 -2: Some tests failed.
   

   Should we use the same approach to extract the type(passed/failed) and file from each line?
6. Hixie commented on Jan 18, 2018

   

   Hixie

   on Jan 18, 2018

   Contributor

   More actions

   Do you mean the right approach for us to know how much passed/failed is to analyze the formated message line by line?

   I'm not sure exactly what you mean by "how much passed/failed". The analyzer only tells you if there was a problem; the output cannot reliably tell you how much of a problem it is or how many problems there are. For example, remove a semicolon or add a comma somewhere bogus, and you'll see the analyzer will start to report thousands of errors, some in entirely different libraries. You can tell if there were any errors by looking at the exit code (0 = no errors, anything else = errors).

   Extract type(warning/error), message, file and line num from each line?

   Well, again, I'm not sure exactly what you're trying to do here so it's hard to evaluate. If you are trying to write an IDE-like feature where it highlights all the lines that had errors, then the output above is easy to parse: it follows the form [...] ... (..., line ..., col ...); ignore lines that don't match.

   flutter test

   For flutter test, the output is not machine-readable. The exit code is 0 on success, 1 on failure. Without knowing exactly what you want to do with the output it's hard to give further advice.
7. Hixie commented on Jan 18, 2018

   

   Hixie

   on Jan 18, 2018

   Contributor

   More actions

   Side note: the classification of errors as warnings, errors, lints, or hints, is mostly arbitrary and I would strongly recommend ignoring it. Some of the lints are more serious than some of the errors, in my experience.
8. kangwang1988 commented on Jan 23, 2018

   

   kangwang1988

   on Jan 23, 2018

   Contributor

   More actions

   @Hixie Got it.
   Thanks.
9. zoechi

   added

   a: tests"flutter test", flutter_test, or one of our tests"flutter test", flutter_test, or one of our tests

   will need additional triageThis issue or PR needs attention before being routed to secondary triageThis issue or PR needs attention before being routed to secondary triage

   and removed

   will need additional triageThis issue or PR needs attention before being routed to secondary triageThis issue or PR needs attention before being routed to secondary triage

   on Dec 3, 2018
10. zoechi

    added

    toolAffects the "flutter" command-line tool. See also t: labels.Affects the "flutter" command-line tool. See also t: labels.

    t: flutter driver"flutter driver", flutter_drive, or a driver test"flutter driver", flutter_drive, or a driver test

    on Feb 22, 2019
11. iapicca commented on Nov 1, 2019

    

    iapicca

    on Nov 1, 2019

    More actions

    Hi @mjohnsullivan is this issue still meant to be open?

    👍React with 👍1Reacted by Matt Sullivan
12. iapicca

    added

    waiting for responseThe Flutter team cannot make further progress on this issue until the original reporter respondsThe Flutter team cannot make further progress on this issue until the original reporter responds

    on Nov 1, 2019
13. 2 remaining items

    Load more
14. iskakaushik

    added 2 commits that reference this issue on Dec 5, 2019

    Manual roll engine fdaa7cf..ee4c2a5 (8 commits)

    ...

    56d50c8

    Manual roll engine fdaa7cf..ee4c2a5 (8 commits) (#46174)

    ...

    6705d08
15. engine-flutter-autoroll

    added a commit that references this issue on Dec 5, 2019

    Roll engine fdaa7cf..ee4c2a5 (8 commits) (#46178)

    ...

    cb0dcbe
16. jmagman commented on Feb 20, 2020

    

    jmagman

    on Feb 20, 2020

    Member

    More actions

    The flutter drive part was split out to #51135.
17. llucax commented on Jan 27, 2021

    

    llucax

    on Jan 27, 2021

    More actions

    Another use case is parsing the output to integrate in editors/IDEs. For example, I would like to use these commands as makeprg in VIM so I can run :make and it will run them and parse the output to get a list of errors so I can quickly jump to them. Since plugin-ins like coc-flutter already do some on-the-fly analyzing, it is more important for test for me, although it would be nice to also be able to run an analyze for the whole project (as the on-the-fly option only shows errors in the current file).

    In particular the VIM plugin Neomake complains about issues adding dart support because of this: neomake/neomake#1803
18. llucax commented on Jan 27, 2021

    

    llucax

    on Jan 27, 2021

    Last edited by llucax

    More actions

    BTW, I guess the suggestion to use dartanalyzer is outdated, as the tendency is to move away from dartanalyzer according to the dart tool documentation:

    Over time, we expect the dart command to replace most existing commands in the Dart SDK.

    But another reason for me, is flutter analyze does some extra magic, like running inside the example directory by changing to it, as it is a separate project.
19. ueman commented on May 4, 2021

    

    ueman

    on May 4, 2021

    Contributor

    More actions

    I want to add support for reporting linter errors to Code Climate. Sot his would be beneficial for it.
20. christopherfujino

    added

    c: new featureNothing broken; request for a new capabilityNothing broken; request for a new capability

    c: proposalA detailed proposal for a change to FlutterA detailed proposal for a change to Flutter

    P3Issues that are less important to the Flutter projectIssues that are less important to the Flutter project

    on Jun 1, 2022
21. flutter-triage-bot

    added

    team-toolOwned by Flutter Tool teamOwned by Flutter Tool team

    triaged-toolTriaged by Flutter Tool teamTriaged by Flutter Tool team

    on Jul 8, 2023
22. bkonyi

    closed this as not plannedon Jun 27, 2025
23. github-actions commented on Jul 11, 2025

    

    github-actionsbot

    on Jul 11, 2025 – with GitHub Actions

    More actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.
24. github-actions

    locked as resolved and limited conversation to collaborators on Jul 11, 2025