[flutter-web] Add support for classes or testID attribute that can be used to identify html tags

@yjbanov

I'm desperately trying to get Cypress to run on top of the html renderer. Out of all possible testing tools I am strongly favouring Cypress for many reasons. All I need is some sort of an id/class/attribute that can help me find the desired divs. I was looking for ids but I found this answer. Please don't force us to misuse Semantics for the sake of testing in Cypress. And no I don't desire to use the official testing tooling. I've had many experience in the past to reach the conclusion that Cypress is the way forward.

We cannot support adding stable IDs to HTML nodes generated by the engine because the DOM structure is not stable. There's no 1-to-1 correspondence between widgets and DOM nodes. However, you can use platform views where you can create your own DOM nodes and assign arbitrary attributes, including id.

Originally posted by @yjbanov in #95624 (comment)

I'm going to share some extra info just to clarify that I really hope to see support for testing ids or plain old ids and classes. As long as the semantics can be tracked in the widgets tree, I suppose there's a way to mirror ids as well.

Testing with Cypress
Thanks to the HTML renderer in Flutter we had the possibility to test using Cypress. The beauty of this approach is that we can use the comfort of the browser developer tools to run the tests as close to real conditions as possible. Another benefit is the way Cypress deals with change detection in SPA frameworks. Cypress by default waits a certain amount of period until the change detection does its job. This spares us of the major pain of having to orchestrate change detection events inside the framework testing environment (Angular, React, Flutter, testing tools all have this drawback).

Another consideration is the severity of the bugs and the time spent on fixing them. From practice and experience I observed that the serious bugs show up in integration scenarios. These integration scenarios tend to be a lot harder to test in a unit testing fashion. When testing, our focus is on finding integration behaviours that don't work as expected. Thankfully, Cypress can be used to emulate both unit testing and integration testing. That means you have one tool that handle both scenarios if properly configured.

During the lifespan of the app the components themselves are changed quite often. Therefore more than half of the development effort for unit testing is spent on upgrading the tests. The difficult part of refactoring unit tests is that it requires careful analysis of the code that rigs the component for testing, the code that triggers the change and the code that evaluates the results. Testing code built with the unit testing provided by the framework has to deal with:

Setting up a testbed (declaring the dependencies)
Precise invocation of the methods that control the change detection mechanism
Blindly reading virtual dom and using super technical abstractions to reach to the template
Interpreting the results and checks to make sure we really testing what we think we are testing

All these operations are far easier to setup and maintain when dealing with the component as it is in the UI. This is where Cypress shines. Cypress is able to handle the components like any other regular DOM. It has built in async support which automatically waits until the change detection settles on the final state. It has short and concise syntax. Overall the life testing in Cypress is a lot easier. With the exception induced by Flutter DOM rendered, of not so easy to query the DOM. I'll explain down bellow.

Summary:

Access to browser dev tools
No need to hassle with change detection
Unit testing eats up a lot of valuable time at refactoring
Most bugs happen in integration
Framework agnostic testing (we are using only the browser APIs).

Cypress Caveats
In an ideal world, Flutter would provide a means of tagging elements with testingIds. For whatever internal reason the dev team decided not do so. This makes the job of composing queries to find elements in page is a bit harder. Since the flutter framework generates a super verbose dom tree we need to rely on other method for spotting elements in the hierarchy.

Search by text
Search by aria label (semantics for accessibility)

Search by text however has another caveat. For whatever reason, the Flutter framework often times outputs small canvas elements for small text fragments. This means sometimes you might not be able to find your element by using cy.contains(). A fix around this issue is to add the following property in the text fields that refuse to render as paragraphs.

#74986 - Forcing Flutter to render <p> instead of <canvas>

    style: TextStyle(
      fontFeatures: const [FontFeature.proportionalFigures()],
    ),

Whenever you can't use text search, you can use Aria tags. Make sure to label your ids in such a way that they make sense for screen readers and users with disabilities. After all we will need to consider this demographic sooner or later. So we better start earlier than later. There are currently 2 ways available to setup these tags:

StackOverflow - Using semantics as makeshift ids

    Semantics(
      label: 'Article title',
      child: Container(...),
    ),


Text(
      widget.article.author?.name ?? '',
      semanticsLabel: 'Author name',
    ),

These fixes are not a great solution but they do the job. Hopefully in the future the Flutter dev team will address the text rendering issue. And hopefully they will add support for ids and classes. Until that happens we have to work around these issues. Ruins a bit the fun of Cypress but it still is not a sufficient reason to abandon this approach. The benefits far outweigh the drawbacks.

I think this code sample explains better why I'm willing to cause so much trouble for getting Cypress and Flutter to play nice together:

describe.only('Article Editor', () => {

  before(() => cy.goTo('/s/space-brains/all/article/create'));

  it('Displays keywords field placeholder', () => {
    cy.app().contains('Enter Keywords')
      .should('exist');
  });

  it('Displays author avatar', () => {
    cy.app().get('[aria-label="Author avatar"]')
      .should('exist');
  });

  it('Displays author name', () => {
    cy.app().contains('Space Instructor')
      .should('exist');
  });
  
  // ...

So to get a better experience out of it we need to be able to define test-id instead of aria-label

@yjbanov

/cc @yjbanov

I'm still of the opinion that looking into <flt-glass-pane> is the wrong method for this. However, being able to integrate with other testing tools is a reasonable feature request.

@adrian-moisa

@adrian-moisa @yjbanov
I'm successfully using integration testing based on semantics, it works fine with both html and canvaskit renderers. This post on stackoverflow (by @adrian-moisa) helped me to start: https://stackoverflow.com/a/70918545/714136
Semantics generated "by default" is sufficiently enough, I don't have many "custom" Semantics() in my large project.
In some cases it would be nice to be able to add classes or ids to DOM elements, but it will be limited to html renderer only.

// First activate semantics:
document.querySelector('flt-glass-pane').shadowRoot.querySelector('flt-semantics-placeholder').click({force: true});

document.querySelector('flt-glass-pane').shadowRoot.querySelector('[aria-label="Search"]').click();

document.querySelector('flt-glass-pane').shadowRoot.querySelector('[aria-label="First Name *"]').focus();
document.querySelector('flt-glass-pane').shadowRoot.querySelector('[aria-label="First Name *"]').value = 'John';

document.querySelector('flt-glass-pane').shadowRoot.querySelector('[aria-label^="HOLD"]').click();

@slavap

@slavap I'd recommend you alias these long queries for readability. I see that the Flutter team marked it as new feature. I hope that we will see testing ids soon. Until then we can alias the querying to make it easier to digest and hide this little nastiness from the tests.

@bartekpacia

@bartekpacia is right. This take lots of time. In meanwhile we can use the integration_test package.

Of course using patrol for web would be super amazing. So please keep pushing you guys at LeanCode!

I've been writing lately quite a few tests in Patrol. I can see it does the job nicely. Plus the maintainers team seems super active and responsive to tickets. There are some things that could be improved but overall I'm super pleased. The most important part is that it hides away the pumpAndSettle. So I'd say don't bother making a fish climb a tree. Use native dedicated tooling. With enough community build up we might see even a test runner resembling Cypress.

As for the web only limitation, ain't such a big limitation after all. I've been able to setup an emulator device that has a 1080p resolution and pixel density. Thus I can test on mobile the GUI as if it is rendered on the web. No difference at all. Plus I made a couple of testing utils that can force the viewport size to resize to tablet or mobile without leaving the emulator. This enables me to test all behaviours in one emulator. So, overall, I'm really happy. Even more, I managed to run the entire app, including auth. No major compromise made.

@adrian-moisa

@adrian-moisa Please don't close it - this functionality is still needed by many.

@ricardolb In Flutter 3.13.6 it is working again, but now it is implemented differently than before!
document.querySelector('flt-glass-pane').shadowRoot.querySelector('flt-semantics-placeholder').click({force: true});

document.querySelector('flt-semantics-host').querySelector('[aria-label="Search"]').click();

document.querySelector('flt-semantics-host').querySelector('[aria-label="First Name *"]').focus();
document.querySelector('flt-semantics-host').querySelector('[aria-label="First Name *"]').value = 'John';
Pay attention that if you enter these commands in browser development tools console, then you have to activate tab with application after sending focus(), then return to console and enter .value =

It would be really nice if someone finds how to eliminate or replace focus() to something else, because it is pretty inconvenient and unstable.

The input field value doesn't show up in form. And click() on submit button does nothing. Only thing is that using document.querySelector('flt-glass-pane').shadowRoot.querySelector('flt-semantics-placeholder').click({force: true}); shows up the DOM element. But form input and submission doesn't work. I tried with focus and then setting the value, still it doesn't work. Any help on this please.

We are soon going to make Semantics.identifier useful for DOM lookup. You'll be able to do something like this in your app:

Semantics(
  identifier: 'my-test-id',
  child: ...
)

and then, you can lookup the DOM element by doing:

cy.get('[flt-semantics-identifier="my-test-id"]')

Does this help with your use case?

@mdebbar

@mdebbar @yjbanov
And how this identifier: 'my-test-id' will be working in case of canvaskit renderer?
Currently
document.querySelector('flt-semantics-host').querySelector('[aria-label="Search"]').click();
works for both html and canvaskit renderers.

OR do you mean that my-test-id will be available through flt-semantics-host ?

@mdebbar

@mdebbar I've long given up on Cypress for flutter and moved away to another testing library. I'm using Patrol and I'm satisfied enough to not consider ditching it. Would have been nice to benefit from a mature framework like Cypress but Patrol works as is and is good enough. Also it's confusing, I see that there are plans to removed the html renderer. I don't see how Cypress would work at all in that case.

[web] Add 'flt-semantics-identifier' attribute to semantics nodes (fl…

And how this identifier: 'my-test-id' will be working in case of canvaskit renderer?

Semantics work the same way in both the html or canvaskit renderer.

Also it's confusing, I see that there are plans to removed the html renderer. I don't see how Cypress would work at all in that case.

If you are relying on semantics, they will continue to work the same way with the canvaskit renderer. The semantics system is independent from the renderer you use.

And how this identifier: 'my-test-id' will be working in case of canvaskit renderer?

Semantics work the same way in both the html or canvaskit renderer.

Also it's confusing, I see that there are plans to removed the html renderer. I don't see how Cypress would work at all in that case.

If you are relying on semantics, they will continue to work the same way with the canvaskit renderer. The semantics system is independent from the renderer you use.

is there anyway we can test this already? I see this has been merged now

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

[web] Add 'flt-semantics-identifier' attribute to semantics nodes (fl…

Navigation Menu

Repository navigation

[flutter-web] Add support for classes or testID attribute that can be used to identify html tags #97455

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[flutter-web] Add support for classes or testID attribute that can be used to identify html tags #97455

Description

Activity

adrian-moisa commented on Jan 30, 2022

adrian-moisa commented on Jan 30, 2022

darshankawar commented on Jan 31, 2022

yjbanov commented on Feb 3, 2022

slavap commented on Mar 4, 2022

adrian-moisa commented on May 2, 2022

14 remaining items

carlosfarfangalindoNL222 commented on Nov 7, 2023

adrian-moisa commented on Nov 11, 2023

jaseemtan commented on Nov 30, 2023

mdebbar commented on Jun 7, 2024

slavap commented on Jun 7, 2024

adrian-moisa commented on Jun 16, 2024

mdebbar commented on Jun 24, 2024

ricardolb commented on Jun 24, 2024

github-actions commented on Jul 9, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions