InteractiveViewer Widget by justinmc · Pull Request #56409 · flutter/flutter

justinmc · 2020-05-05T23:57:35Z

Description

This PR introduces a new widget called InteractiveViewer. Any widget tree passed as its child will be able to be panned and zoomed by user gestures.

Closes #8272
Closes #12878
Closes #20175

Future: rotation #57698
Future: overscroll #58042
Future: direction locking: #58043
Future: builder #58603

Table Example

InteractiveViewer(
  scaleEnabled: false,
  child: Table(
    columnWidths: <int, TableColumnWidth>{
      for (int column = 0; column < columnCount; column++)
        column: const FixedColumnWidth(300.0),
    },
    children: <TableRow>[
      for (int row = 0; row < rowCount; row++)
        TableRow(
          children: <Widget>[
            for (int column = 0; column < columnCount; column++)
              Container(
                height: 100,
                color: row % 2 + column % 2 == 1 ? Colors.red.withOpacity(0.5) : Colors.green.withOpacity(0.5),
              ),
          ],
        ),
    ],
  );
),

Image viewer example

This example was modified from the Flutter gallery.

InteractiveViewer(
  child: Center(
    child: Image.asset(
      photo.assetName,
      fit: BoxFit.cover,
      package: photo.assetPackage,
    ),
  ),
),

Game board example

This example was also modified from the Flutter gallery.

return InteractiveViewer(
  // The transformationController is used to animate back to the "home"
  // position, as seen in the gif.
  transformationController: _transformationController,
  minScale: 0.1,
  onScaleStart: _onScaleStart,
  onTapUp: _onTapUp,
  child: CustomPaint(
    size: _board.size,
    painter: _BoardPainter(
      board: _board,
    ),
  ),
);

API Details

boundaryMargin

The boundary is the invisible perimeter around the child that prevents the viewport from viewing beyond it. It's specified with EdgeInsets. The default is EdgeInsets.zero, which prevents the viewport from viewing beyond the child. A positive value extends the boundary by the given amount. The example below allows the viewport to move 20px beyond the edge of the child in all directions.

  boundaryMargin: EdgeInsets.all(20.0),

I used this API because I anticipate that users will most often want the boundary to perfectly frame the child, and otherwise will want to specify it in relation to the edges of the child. In these cases, the user doesn't have to know the size of the child or the current constraints in order to specify the boundary.

transformationController

transformationController is a ValueNotifier on the Matrix4 that defines the current transformation. It enables both listening to changes and setting the value of the matrix. This provides a generic way of implementing lots of new capabilities, like:

Set an initial position/orientation for the child by setting the initial value.
Listen for and respond to certain transformations.
Animate a transformation, such as the "undo" button in the game board demo above.

pan/zoom/(rotate)Enabled

Booleans that enable/disable their corresponding interaction.

min/maxScale

Doubles that limit zoom. Defaults to 0.8 and 2.5, respectively, just because I thought those values worked out well.

Question: Should I default them both to 1.0 so that it doesn't zoom at all, or is that unnecessarily difficult for the user?

GestureDetector listeners

I have passed through all callbacks for GestureDetector events, like onDoubleTap, etc. so that the user can receive events transformed back to the original child's coordinates. For example, without this, tapping the top left corner of the child is detected as 0,0 and translating the child to the bottom right of the viewport at tapping the child's top left corner is detected as viewportWidth,viewportHeight. My built-in onTapUp will do the inverse transform and return 0,0 for both taps.

Question: Should I remove all of these listeners and document that the user needs to handle this themselves? They could use a transformationController to get the current matrix and do the inverse transform themselves. Done, and added toScene helper method on transformationController.

Tests

A bunch of unit tests for my math util functions.
Test pan and zoom on a child that's the same size as the boundary, smaller, and larger.
Test inertia panning.
Test sliding boundaries.
Test zooming adjusts translation to stay in boundaries.
Test zooming adjusts translation to keep the focal point in the same location in the viewport.

More questions

~~Mouse zoom support? (PointerScrollEvent)~~ Done
~~Drag targets inside of the child don't work if they are initially outside of the viewport.~~ Fixed
- Update: I've decided to not add further support for this in this PR. Original notes follow.
- Flutter's hit testing purposely excludes events outside of the original widget. This has come up before with the Transform widget. The solution was to make the Transform widget bigger. See GestureDetector doesn't respond correctly after transition with Transform widget #27587.
- Users of InteractiveViewer may not be able to just make InteractiveViewer big enough.
- I can fix this inside of InteractiveViewer by wrapping everything in SingleChildScrollViews, which allows the child to be as big as it wants (even bigger than the screen), so drop targets work anywhere. However, this means the child is sized with no constraints, which is usually not what users will want.
- I can't leave it to the users to wrap their child in SingleChildScrollViews because then InteractiveViewer can't know the size of the viewport.
- Should I add a parameter to conditionally add the SingleChildScrollViews?

justinmc · 2020-05-18T18:12:40Z

Should I even bother using InteractiveViewer in these two gallery demos anymore now that the main gallery is a separate repo? I guess these are just for integration tests.

justinmc · 2020-05-18T19:00:29Z

Notes for reviewers at this time:

I'm still adding tests.
~~I'm also investigating if it's possible to add mouse scrollwheel zooming at this time.~~ Added.
Rotation is currently disabled. I did this by asserting that disableRotation is set to true in order to avoid API changes in the future. The reason is that I likely need some time to get boundaries correct after rotation (there is a TODO in the code with more about this).
My API decision to pass-through all the GestureDetector listeners is very verbose. Is there a more concise way to achieve the same? Document for the user how to transform the coordinates themselves? I wrote some notes about this in the PR description above. Hans helped me improve this API. See onInteraction.
There is an underlying Flutter problem with using drag targets inside of InteractiveViewer, see the PR description above.
~~I think the tests are failing due to the weird version error, and I need to squash all the commits. Will do that soon.~~ Done and fixed.
Let me know if the API could be improved at all!

…ilder inside of IV

HansMuller · 2020-05-19T15:50:05Z

@justinmc - please assume that I've inserted "IMHO" below

Question: Should I default them both to 1.0 so that it doesn't zoom at all, or is that unnecessarily difficult for the user?

Making the "out of the box" use case work nicely sounds like the right way to go.

Question: Should I remove all of these listeners and document that the user needs to handle this themselves? They could use a transformationController to get the current matrix and do the inverse transform themselves.

Although I'm a big fan of low-cal APIs, providing these listeners seems like a useful convenience. On the other hand, they don't seem necessary for most apps? If they're typically not needed, since the InteractiveViewer's child will handle events directly, I think it would be best to just provide an easy way to do the transforms with the transformationController.

HansMuller · 2020-05-19T15:53:21Z

Rotation is currently disabled. I did this by asserting that disableRotation is set to true in order to avoid API changes in the future. The reason is that I likely need some time to get boundaries correct after rotation (there is a TODO in the code with more about this).

I think it would be reasonable to ship the first version of this widget without rotation support. The docs will need to be updated to reflect that.

HansMuller

I haven't really made it into the depth of this PR. Just submitting a few comments I made this morning, since it will be a little while before I return to it.

HansMuller · 2020-05-19T15:54:37Z

+/// that the callbacks are called with are untransformed and in relation to the
+/// original position of the child.
+///
+/// [child] must not be null.


The [child]

HansMuller · 2020-05-19T15:55:18Z

+import 'layout_builder.dart';
+import 'ticker_provider.dart';
+
+/// A widget that enables pan, zoom, and rotate interactions with its child.


Here and elsewhere: just leave rotation out of the API and docs.

HansMuller · 2020-05-19T15:56:02Z

+/// available.
+///
+/// {@tool dartpad --template=stateful_widget_material_ticker}
+/// This example shows how to use InteractiveViewer in the simple case of


This doesn't seem like a simple case because DragTarget. The examples of the PR's description seem like what you'd want here.

Whoops I forgot I had left DragTarget in here. I'll simplify it to be like the first example in the description.

HansMuller · 2020-05-19T15:56:44Z

+///           column: const FixedColumnWidth(300.0),
+///       },
+///       children: <TableRow>[
+///         for (int row = 0; row < rowCount; row++)


+= 1 instead of ++ because Flutter style

HansMuller · 2020-05-19T15:59:32Z

+/// panning over a large widget representing a table.
+///
+/// ```dart
+///   static Widget _getTable(int rowCount, int columnCount) {


With a few more lines of code this could be a stateless widget. Widgets are preferable to static methods.

HansMuller · 2020-05-19T16:01:40Z

+    this.onVerticalDragEnd,
+    this.onVerticalDragStart,
+    this.onVerticalDragUpdate,
+    ValueNotifier<Matrix4> transformationController,


If this was its own type it would be easier to add functionality in the future. Or convenience methods for doing the coordinate transforms now.

HansMuller · 2020-05-19T16:03:17Z

+
+  /// The Widget to perform the transformations on.
+  ///
+  /// [child] should usually have an intrinsic size. This is used to calculate


This needs some elaboration. When can child not have an intrinsic size? What's an example of a child with an intrinsic size? What's an example of a child that does not?

I've since updated the widget so that it handles both cases just like any other widget would. For child: Container(width: 100, height: 100), you'll get a 100x100 box. For child: Container(), it can expand to the size of the parent, like it would if there was no InteractiveViewer. Things like boundary work intuitively in both cases.

I'll update these docs.

…e fromViewport public.

…et implemented

justinmc · 2020-06-29T17:29:32Z

@not-holar You seem to be right about both of these.

Here's a PR to reverse scroll zoom direction: #60523

And here's a PR to update the boundaries when child size changes: #60532

That second PR seems to greatly improve behavior after the screen size changes. I imagine there could still be edge cases that don't feel right though. Let me know if you see anything else.

MichealReed · 2020-06-29T18:56:12Z

@justinmc would you mind taking a look at scale and pan performance on web / mobile web, especially when text is used? May even be an opportunity for a test/benchmark. I used statsfl (https://pub.dev/packages/statsfl) (to easily benchmark across mobile and desktop web browsers), your above column/row example, and added an appbar to increase the row/column count and disable/enable text.

On chrome mobile, LG G7 ThinQ, issues are immediately apparent with text enabled and a crash can instantly be induced by scaling. FPS issues with scroll and scale start around 75x75 if text is disabled, jank is severe around 150x150, and crash prone around 300x300.

On chrome with my decently specced PC, an FPS drop can be seen immediately when scrolling with text enable, but scaling can maintain 60fps. At 10x10 scaling performance is impacted, severe at 20x20, and crash prone on anything above 30x30. Without text performance issues start at 20x20, but the issue does not become severe until over 100x100.

bool _text = true

appBar: AppBar(
        actions: [
          IconButton(
              icon: Icon(Icons.add),
              onPressed: () {
                setState(() {
                  _rowCount += 5;
                  _columnCount += 5;
                });
              }),
          IconButton(
              icon: Icon(Icons.text_fields),
              onPressed: () {
                setState(() {
                  _text = !_text;
                });
              })
        ],
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),

                      Container(
                        child: _text
                            ? Text("Column: $_columnCount\nRow: $_rowCount")
                            : null,
                        height: 100,
                        color: row % 2 + column % 2 == 1
                            ? Colors.red.withOpacity(0.5)
                            : Colors.green.withOpacity(0.5),
                      ),

This may be related to deeper issues with web, but even testing this with skia, you will still notice performance issues. If it is a deeper issue with web, maybe InteractiveViewer can be a better window to diagnose than previous widgets.

justinmc · 2020-06-29T20:16:19Z

@michaelreed Would you be able to open a separate issue with these details and the full code for your example?

I appreciate you taking the time to dig into the performance. I haven't done much optimization for web so there could be some easy improvements to make. Long term though, I think we're going to need something like and InteractiveViewer.builder constructor (similar to ListView.builder) that builds visible content on demand. That will be the only way to handle very large tables.

Edit: Or maybe this is covered by #59775

MichealReed · 2020-06-29T20:47:21Z

Opened #60550 to cover.

The planned builder will help when the user is zoomed, the general concept of a builder here reminds me of Occlusion Culling (https://docs.unity3d.com/Manual/OcclusionCulling.html). It may not help when the user is completely zoomed out though as it will still need to render a significant number of cells and the viewer can be customized to allow for large views, but prevent max scale thus still allowing scroll/pan. ListViews and GridViews have it easier and don't have to tackle the scale aspect.

Although skia fixes the crashing related to text and increases performance significantly, I am still seeing fps drop and jank when using it, especially on mobile. If there's anything low hanging for performance on the widget side that you could change, that would be awesome.

using
flutter run -d chrome --web-hostname=lanIP --web-port=8080 --dart-define=FLUTTER_WEB_USE_SKIA=true --release
to deploy and access via mobile browsers.

edit: I believe the repaint issue to be separate. I can't think of any other widget that impacts scale like the InteractiveViewer, and it may have uncovered/be connected to issues with how flutter handles scaling. A bit over my head currently, but I have a hunch.

justinmc · 2020-06-30T20:45:15Z

Thanks for opening that issue, I'll try to see if there are any quick improvements to make.

One thing I've been thinking about recently related to the scale problem. Currently, as a user of InteractiveViewer, you could listen to the current zoom level by adding a listener to transformationController and looking at transformationController.value.getMainScaleOnAxis(). If the zoom level is below a certain value (meaning zoomed out), you could pass a simplified version of your child to InteractiveViewer. I wrote a demo sort of like that recently.

MichealReed · 2020-07-01T13:23:46Z

Ah very cool, I can use that to pause animations. I could also implement an LOD system based on this to downscale assets. Great idea, let me know if any others come to mind.

czaku · 2020-07-01T18:25:19Z

I am really looking forward to using this widget in one of my projects, do you know when it would be available in beta or stable?

justinmc · 2020-07-01T22:12:22Z

This made it into v1.19.0-5.0.pre, so it's only in master right now. It looks like it should be in the next beta release, though.

InteractiveViewer Widget

TheOneWithTheBraid · 2020-08-06T13:18:23Z

Is it somehow possible to set the current scale or offset of the InteractiveViewer from outside the widget? Eg. by a GlobalKey etc.

This is useful if you would like to implement a scroll bar or a zoom slider outside of the widget.

justinmc · 2020-08-06T15:51:57Z

Yes, using transformationController. There's an example in the docs there that should get you started, but you can set the matrix like transformationController.value = myMatrix. The Matrix4 class has some useful methods for working with the matrix.

TheOneWithTheBraid · 2020-08-06T17:03:22Z

Thank you, @justinmc. For future people having the same issue:

transformationController.value = transformationController.value.scaled(myCustomScale);

aytunch · 2020-08-07T15:57:47Z

Will there be a way to allow other widgets to stay on top of the InteractiveViewer widget while it is being interacted with? What I mean is something like a z-index. A boolean property named isAboveRest which defaults to true could solve this. If it is set to false by the dev, the InteractiveViewer widget's z-index does not get changed. If it was inside of a Stack, it would still behave same way in terms of z-index while being interacted with.

justinmc · 2020-08-07T23:31:21Z

@aytunch What do you mean exactly? Here I can put something on top of InteractiveViewer using a Stack, but I think you're asking for something more complicated: https://dartpad.dartlang.org/fc0095e229e2ea1f8df7d06cd8e76dbe

aytunch · 2020-08-08T18:19:22Z

@justinmc I thought that once the InteractiveViewer widget is interacted with, it comes to the front most z index. But from your example I see that it keeps it place z-wise even while interacted with. Thanks for the example (btw dartpad gives an error regarding Stack)

Edit: One confusion I have with naming is why do we have only ScaleStartDetails for onInteractionStart?
Afaik there are 3 gestures/interaction that are allowed: Scale, Pan, Rotate(in the future). If we have only one callback for all 3 of them, why is the name ScaleXXXDetails instead of InteractionXXXDetails? Am I missing something?

              onInteractionStart: (ScaleStartDetails det){
                print(det.focalPoint.dx);
              },
              onInteractionUpdate: (ScaleUpdateDetails det){
                print(det.scale);
              },

Also this example does not print anything when I am pinch zooming on the dartweb by the touchpad.

justinmc · 2020-08-10T16:03:14Z

@aytunch Glad that helped. I'm not sure about the error, hopefully it's just a problem with dartpad. I don't get the error when running locally in Chrome.

Internally, scale, pan, and rotation are all implemented using GestureDetector's onScale callbacks. The given details should provide all the information you need about the gesture (though you may need to do a little bit of calculation, see #43833).

Pinch to zoom on a trackpad doesn't work right now, see #63060. Two finger scroll to zoom should work.

TerrenceAddison · 2020-09-03T05:05:04Z

@justinmc Hello! so I filed a bug issue regarding InteractiveViewer see #64852 . I was planning on attempting to solve the bug issue if it's possible and I wonder if you have any lead or guide I can follow to start of? Thank you!

justinmc · 2020-09-03T22:38:12Z

@TerrenceAddison Thank you for the issue! I will respond over there.

devteam-tchs · 2020-09-21T12:27:19Z

I think a method for panning to a certain point is going to be required if you're making a game like myself. If I have a large map and start panning around, how do I get back to where let's say my character is currently standing? I don't see any way of currently doing this in the widget's API. @justinmc

MichealReed · 2020-09-21T14:03:54Z

@devteam-tchs just assign a new, or update the existing transformation controller.

    _transformController = new TransformationController(
        Matrix4.translation(vm.Vector3(-2200, -500, 0)));```

justinmc · 2020-09-21T17:10:08Z

+1 to @MichealReed's answer. You can also animate that by using Flutter's existing animation framework, which I have an example of in the gallery.

devteam-tchs · 2020-10-04T11:57:39Z

@justinmc Where can I find the source code for your demos? The snippets provided aren't very useful without their full context. I've been trying to get the Table version to pan but it's just stuck, zooming works however.

MichealReed · 2020-10-04T19:28:03Z

@justinmc Where can I find the source code for your demos? The snippets provided aren't very useful without their full context. I've been trying to get the Table version to pan but it's just stuck, zooming works however.

constrained: false should fix your pan issue. Docs might need to be updated to show this argument, the example might have been made before the parameter was introduced.

devteam-tchs · 2020-10-05T08:29:58Z

@MichealReed That seemed to have done it. Weird I thought I added that argument yesterday when I copied over this example https://github.com/justinmc/flutter-interactive-viewer-demos/blob/master/lib/table/table_page.dart. You are right that it needs to be added to the docs.

justinmc · 2020-10-05T16:30:00Z

I've opened a PR to improve the docs a bit around this, let me know what you guys think: #67314

And it seems like you've found my examples repository. I did a little bit of clean up and modernization there as well just now.

justinmc added framework flutter/packages/flutter repository. See also f: labels. work in progress; do not review labels May 5, 2020

justinmc self-assigned this May 5, 2020

googlebot added the cla: yes label May 5, 2020

justinmc force-pushed the gesture-transformable-2 branch from 5cc1994 to 4701c6e Compare May 7, 2020 17:46

fluttergithubbot added p: material_ui material_ui package in flutter/packages c: contributor-productivity Team-specific productivity, code health, technical debt. labels May 8, 2020

justinmc commented May 18, 2020

View reviewed changes

justinmc requested a review from HansMuller May 18, 2020 19:00

Applying diff of many commits in order to fix version errors

d407d46

justinmc force-pushed the gesture-transformable-2 branch from fb9f624 to d407d46 Compare May 18, 2020 21:54

justinmc added 3 commits May 18, 2020 14:58

Add disableRotation in test since it's now required

83af255

Fix an edge case and test it, along with another widget test

939fbdb

Remove the complicated IVSized abstraction in favor of using LayoutBu…

e29be50

…ilder inside of IV

sensuikan1973 mentioned this pull request May 19, 2020

flexible life road size sensuikan1973/HumanLifeGame#113

Merged

Fix sample code from analyzer

1663b8d

HansMuller reviewed May 19, 2020

View reviewed changes

justinmc added 3 commits May 20, 2020 14:31

Remove all passed through event listeners, add onInteraction, and mak…

5430f61

…e fromViewport public.

_getOffset removed, since it isn't needed with details.localPosition

5baddeb

Docs improved per code review

74ffb3a

justinmc mentioned this pull request May 20, 2020

InteractiveViewer should support rotation #57698

Open

justinmc added 6 commits May 20, 2020 15:39

Rotation is no longer mentioned in the docs, except to say it's not y…

fdcf088

…et implemented

Mousewheel zooming

c20636b

Analyzer fix in the sample

b3dc34c

TransformationController class

070dce6

fromViewport renamed to toScene

908815d

Move TransformationController

f3e840b

justinmc mentioned this pull request Jun 29, 2020

InteractiveViewer with a changing screen size #60532

Merged

iapicca mentioned this pull request Jul 3, 2020

GestureDetector hitTesting not working when used with Transform.translate #39656

Closed

vasilich6107 added a commit to artflutter/whatsup_flutter_june2020 that referenced this pull request Jul 8, 2020

https://github.com/flutter/flutter/pull/56409

1f1bba7

InteractiveViewer Widget

bobagold mentioned this pull request Aug 4, 2020

FR use InteractiveViewer bobagold/minesweeper#11

Open

justinmc mentioned this pull request Oct 5, 2020

InteractiveViewer table example improvements #67314

Merged

TzviPM mentioned this pull request Jul 20, 2022

WIP InteractiveViewer Widget #44015

Closed

Conversation

justinmc commented May 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Table Example

Image viewer example

Game board example

API Details

boundaryMargin

transformationController

pan/zoom/(rotate)Enabled

min/maxScale

GestureDetector listeners

Tests

More questions

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justinmc commented May 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HansMuller commented May 19, 2020

Uh oh!

HansMuller commented May 19, 2020

Uh oh!

HansMuller left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justinmc commented Jun 29, 2020

Uh oh!

MichealReed commented Jun 29, 2020

Uh oh!

justinmc commented Jun 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MichealReed commented Jun 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinmc commented Jun 30, 2020

Uh oh!

MichealReed commented Jul 1, 2020

Uh oh!

czaku commented Jul 1, 2020

Uh oh!

justinmc commented Jul 1, 2020

Uh oh!

TheOneWithTheBraid commented Aug 6, 2020

Uh oh!

justinmc commented Aug 6, 2020

Uh oh!

TheOneWithTheBraid commented Aug 6, 2020

Uh oh!

aytunch commented Aug 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinmc commented Aug 7, 2020

Uh oh!

aytunch commented Aug 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinmc commented Aug 10, 2020

Uh oh!

justinmc commented May 5, 2020 •

edited

Loading

justinmc commented May 18, 2020 •

edited

Loading

justinmc commented Jun 29, 2020 •

edited

Loading

MichealReed commented Jun 29, 2020 •

edited

Loading

aytunch commented Aug 7, 2020 •

edited

Loading

aytunch commented Aug 8, 2020 •

edited

Loading