Skip to content

Flutter Test should support multiple reporters #69425

Description

@danilofuchs

Use case

When running flutter test on a CI, output is very verbose and hard to follow:

image
(this goes on for 200+ lines)

The current alternative is to output to json with flutter test --machine. This is even more verbose, but can be converted to JUnit XML so we can use a specialized test visualization tools, such as Bitrise Test Reports.

This does the job, but requires a lot of configuration and there is currently no way of having two reporters at the same time.
If I want both CLI output and JSON, I need to run tests twice.
I want to be able to output to any format while still having a console output (customizable as well).

Proposal

Flutter should have a --reporter CLI option, inspired by Mocha Reporters or Jest Reporters.

# Current output if not in CI environment
flutter test

# Simple, CI friendly output
flutter test --reporter spec

# Minimal output with number of passes AND json output to test_results.json file
flutter test --reporter min --reporter json --reporter-option output=test_results.json

# JUnit XML report out of the box
flutter test --reporter xunit  --reporter-option output=test_results.xml

By default Flutter will use the current reporter. If env CI=true, flutter will use spec automatically.

This is an idea we can take from the very well established and mature JS ecosystem.

Activity

  1. added
    a: tests"flutter test", flutter_test, or one of our tests
    c: proposalA detailed proposal for a change to Flutter
    toolAffects the "flutter" command-line tool. See also t: labels.
    on Nov 2, 2020
  2. ogezue commented on Aug 8, 2022

    @ogezue

    is there something on the market that can be used instead?

  3. nico-famedly commented on Aug 8, 2022

    @nico-famedly

    Dart supports the --file-reporter option: https://github.com/dart-lang/test/blob/master/pkgs/test/doc/json_reporter.md#usage

    It would be nice if flutter supported that too.

  4. bartekpacia commented on Oct 22, 2022

    @bartekpacia
    Member

    flutter test does have a --reporter argument, which accepts compact, expanded, and json. Is this what you're looking for?

  5. danilofuchs commented on Oct 22, 2022

    @danilofuchs
    Author

    flutter test does have a --reporter argument, which accepts compact, expanded, and json. Is this what you're looking for?

    The main goal is outputting to more than one reporter at the same time. So JSON + expanded in a single run.

    Optionally, allow custom reporters, for developing plugins for jUnit XML, ...

  6. github-actions commented on Mar 7, 2023

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  7. locked as resolved and limited conversation to collaborators on Mar 7, 2023
  8. added
    P3Issues that are less important to the Flutter project
    and removed on Jun 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issues that are less important to the Flutter projecta: tests"flutter test", flutter_test, or one of our testsc: proposalA detailed proposal for a change to FluttertoolAffects the "flutter" command-line tool. See also t: labels.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions