Skip to content

Enable testWidgets to track leaks. - #124435

Closed
polina-c wants to merge 24 commits into
flutter:masterfrom
polina-c:testWidgets
Closed

polina-c wants to merge 24 commits into
flutter:masterfrom
polina-c:testWidgets

Conversation

@polina-c

@polina-c polina-c commented Apr 8, 2023 •

Copy link
Copy Markdown
Contributor

Contributes to flutter/devtools#3951

@flutter-dashboard flutter-dashboard Bot added a: tests "flutter test", flutter_test, or one of our tests p: material_ui material_ui package in flutter/packages framework flutter/packages/flutter repository. See also f: labels. c: contributor-productivity Team-specific productivity, code health, technical debt. labels Apr 8, 2023
// ignore: deprecated_member_use
import 'package:test_api/test_api.dart' as test_package;

import '_leak_tracking.dart';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the leading underscore?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is internal library, not exposed in API.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything under lib/src is private unless it's exported by something in lib

@polina-c polina-c Apr 13, 2023 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some repos use leading underscore to make visibility level visible without checking if the file is exported.

And I see such files exist in this repo too:

Screenshot 2023-04-13 at 10 43 04 AM

Did I misinterpret something?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our general policy is to export everything. Those underscore libs there are a somewhat special case, they contain the platform-specific implementation (see the io vs web suffix) for certain features and are used with conditional imports. The library that imports these files conditionally do exports the full API.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to keep this private, why not just add it as a private method to the file where it is used?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasoning to have code separately can be size of the library widget_tester.dart. We want to keep it reasonable and collect everything related to leak tracking, separately.

Merged 'leak_tracking.dart' and '_leak_tracking.dart'

Comment thread packages/flutter_test/test/utils/leak_tracking_utils.dart Outdated
Comment thread packages/flutter_test/lib/src/widget_tester.dart Outdated
Comment thread packages/flutter_test/lib/src/leak_tracking.dart Outdated
Comment thread packages/flutter_test/lib/src/leak_tracking.dart Outdated
Comment thread packages/flutter_test/lib/src/_leak_tracking.dart Outdated
/// The Flutter related enhancements are:
/// 1. Listens to [MemoryAllocations] events.
/// 2. Uses `asyncCodeRunner` for async call for leak detection.
Future<void> withFlutterLeakTracking(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this method duplicated from elsewhere in the framework now? Can we remove it from there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this file will be removed in next PR. Added TODO to that method under flutter/test/foundation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just fix this up in this PR?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to do it, but I think it is not possible, because the usage of the new method is in different package. So, it needs the method to be checked in, in order to use it.

Comment thread packages/flutter_test/lib/src/_leak_tracking.dart Outdated
Comment thread packages/flutter_test/test/widget_tester_test.dart
Comment thread packages/flutter_test/test/widget_tester_test.dart Outdated
@goderbauer

goderbauer commented Apr 12, 2023 •

Copy link
Copy Markdown
Member

One disadvantage of adding this functionality to the testing framework is that we are going to lock people into a particular version of the leak_tracking package. If the leak tracking package evolves, people will not be able to benefit from it until we have upgraded the leak tracking version in Flutter and developers have updated to that new version of Flutter.

This burden is not to be underestimated. Some people have regretted that their package is actually that closely tied to Flutter releases.

cc @dnfield

vector_math: 2.1.4

# Used by testWidgets.
leak_tracker: 2.0.1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what will become a bit challenging that @goderbauer is referring to.

One option would be to start with this as a dev_dependency until it's been vetted a bit more with the framework tests.

At the same time, we should have a good cycle of time to let this get ready before another stable...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to reference something that is imported in lib/src with dev_dependency?

According to this doc, no: https://dart.dev/tools/pub/dependencies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To evaluate this with the framework tests we'd ideally keep it as a dev_dependency on the framework. Not super sure how we can do that with how things are currently setup, though. May need some extra thought.

@polina-c

polina-c commented Apr 13, 2023 •

Copy link
Copy Markdown
Contributor Author

One disadvantage of adding this functionality to the testing framework is that we are going to lock people into a particular version of the leak_tracking package. If the leak tracking package evolves, people will not be able to benefit from it until we have upgraded the leak tracking version in Flutter and developers have updated to that new version of Flutter.

This burden is not to be underestimated. Some people have regretted that their package is actually that closely tied to Flutter releases.

cc @dnfield

In future, if it starts hurting, we will copy minimal subset of the leak_tracker code to the flutter framework and remove the dependency.

Yes, this means one time the users will need to shift to new version of flutter that does not contain the dependency, to use leak tracker.

@polina-c
polina-c marked this pull request as ready for review April 13, 2023 22:57
/// When to collect stack trace information.
///
/// You may need to know call stack to troubleshoot memory leaks.
/// Custonize this parameter to collect stack traces when needed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sp: Customize

Comment on lines +22 to +25
@Deprecated(_nonBackwardCompatibleApiWarning)
class LeakTrackingFlutterTestConfig {
/// Creates a new instance of [LeakTrackingFlutterTestConfig].
@Deprecated(_nonBackwardCompatibleApiWarning)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels odd to check in deprecated code intentionally. @goderbauer @dnfield is there anywhere else in the framework where we do this / is this an acceptable pattern? I know in other places we have _kDebug* flags that can be flipped manually or via a service extension.

I would worry that users would ignore the deprecation warnings and check in usages of LeakTrackingFlutterTestConfig to their repos anyway.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a pattern we have elsewhere and we shouldn't establish it. I am afraid, this is just teaching people to ignore deprecation warnings, which is not in our interest. It is also not in line with our breaking change policy: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General practice would be to evolve the API as a package and then pull it into the framework when it is reasonably stable.


typedef LeaksObtainer = void Function(Leaks foundLeaks);

// TODO(polina-c): remove this file once https://github.com/flutter/flutter/pull/124435

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just do this now?

// ignore: deprecated_member_use
import 'package:test_api/test_api.dart' as test_package;

import '_leak_tracking.dart';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to keep this private, why not just add it as a private method to the file where it is used?

Comment on lines +22 to +25
@Deprecated(_nonBackwardCompatibleApiWarning)
class LeakTrackingFlutterTestConfig {
/// Creates a new instance of [LeakTrackingFlutterTestConfig].
@Deprecated(_nonBackwardCompatibleApiWarning)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a pattern we have elsewhere and we shouldn't establish it. I am afraid, this is just teaching people to ignore deprecation warnings, which is not in our interest. It is also not in line with our breaking change policy: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes

Comment on lines +22 to +25
@Deprecated(_nonBackwardCompatibleApiWarning)
class LeakTrackingFlutterTestConfig {
/// Creates a new instance of [LeakTrackingFlutterTestConfig].
@Deprecated(_nonBackwardCompatibleApiWarning)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General practice would be to evolve the API as a package and then pull it into the framework when it is reasonably stable.

Comment thread packages/flutter_test/lib/src/widget_tester.dart Outdated

class StatelessLeakingWidget extends StatelessWidget {
StatelessLeakingWidget({super.key}) {
// ignore: unused_local_variable

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could just be part of widget_tester_test.dart?

vector_math: 2.1.4

# Used by testWidgets.
leak_tracker: 2.0.1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To evaluate this with the framework tests we'd ideally keep it as a dev_dependency on the framework. Not super sure how we can do that with how things are currently setup, though. May need some extra thought.

@flutter-dashboard

Copy link
Copy Markdown

This pull request has been changed to a draft. The currently pending flutter-gold status will not be able to resolve until a new commit is pushed or the change is marked ready for review again.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@polina-c polina-c closed this Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: tests "flutter test", flutter_test, or one of our tests c: contributor-productivity Team-specific productivity, code health, technical debt. framework flutter/packages/flutter repository. See also f: labels. p: material_ui material_ui package in flutter/packages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants