Expanded hitTest area

It would be useful to be able to expand the hit test area of a widget. Often, the size of a widget can't be increased without ugly hacks or at all, while the current size is too small to make it easily tappable.

The code below contains two examples where this would be useful (and a failed attempt to achieve this). In the first, the red container might be the thumb of a slider at the bottom of an app bar or a handle at the top of a panel for resizing it. In the second case, the button with "<" is inside several widgets to align it with "Title" below it. (In our code, there are several more layout widgets between the padding and "<" to handle animations) In both cases, increasing the size of the widget inside the gesture detector, while keeping the layout visually the same, would lead to a awkward layout.

The ExpandedHitTestArea widget is an attempt to achieve an expansion without affecting the size. However, it doesn't work because all parent widgets don't hit test their children when the hit point is outside the parent.

import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:flutter/widgets.dart';
import 'package:flutter/rendering.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) => MaterialApp(home: HitTest());
}

class HitTest extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Material(
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.stretch,
        mainAxisAlignment: MainAxisAlignment.spaceAround,
        children: [
          SizedBox(
            width: 100,
            height: 100,
            child: Container(
              alignment: Alignment.bottomCenter,
              color: Colors.yellow,
              height: 100,
              width: 100,
              child: GestureDetector(
                onTap: () => print("I'm hit! I'm hit!"),
                child: ExpandedHitTestArea(
                  child: Container(width: 20, height: 10, color: Colors.red),
                ),
              ),
            ),
          ),
          Container(
            height: 100,
            color: Colors.black12,
            child: Padding(
              padding: const EdgeInsets.all(8.0),
              child: Column(
                crossAxisAlignment: CrossAxisAlignment.start,
                children: <Widget>[
                  GestureDetector(
                    onTap: () => print("Tapped"),
                    child: ExpandedHitTestArea(child: Text("<")),
                  ),
                  SizedBox(height: 20),
                  Text("Title"),
                ],
              ),
            ),
          ),
        ],
      ),
    );
  }
}

class ExpandedHitTestArea extends SingleChildRenderObjectWidget {
  const ExpandedHitTestArea({
    Key key,
    Widget child,
  }) : super(key: key, child: child);

  @override
  RenderObject createRenderObject(BuildContext context) => RenderExpandedHitTestArea();
}

class RenderExpandedHitTestArea extends RenderBox with RenderObjectWithChildMixin<RenderBox> {
  
  // trivial implementations left out to save space: computeMinIntrinsicWidth, computeMaxIntrinsicWidth, computeMinIntrinsicHeight, computeMaxIntrinsicHeight

  @override
  void performLayout() {
    child.layout(constraints, parentUsesSize: true);
    size = child.size;
  }

  @override
  void paint(PaintingContext context, Offset offset) {
    if (child != null) {
      final BoxParentData childParentData = child.parentData;
      context.paintChild(child, childParentData.offset + offset);
    }
  }

  @override
  bool hitTest(HitTestResult result, {Offset position}) {
    const minimalSize = 44;
    final deltaX = (minimalSize - size.width).clamp(0, double.infinity) / 2;
    final deltaY = (minimalSize - size.height).clamp(0, double.infinity) / 2;
    if (Rect.fromLTRB(-deltaX, -deltaY, size.width + deltaX, size.height + deltaY).contains(position)) {
      result.add(BoxHitTestEntry(this, position));
      return true;
    }
    return false;
  }
}

[✓] Flutter (Channel unknown, v1.4.18, on Mac OS X 10.14.4 18E226, locale en-NL)
    • Flutter version 1.4.18 at /Users/spkersten/Development/.../flutter
    • Framework revision 8bea3fb2eb (2 weeks ago), 2019-04-11 13:11:22 -0700
    • Engine revision 72986c39ea
    • Dart version 2.2.1 (build 2.2.1-dev.3.1 None)

This is something I looked at fairly extensively when we refreshed the material widget catalog to meet android accessibility guidelines. Specifically, Android recommends at least 48 by 48 minimum hit-testable area (and iOS 44 by 44).

One of the first solutions I looked at was something similar to what you've proposed above. The problem is - how do you adjust hit testing to work without requiring a single gesture to hit test every single render box? Once the layout bounds and the hit testable bounds don't line up you always need to walk the tree.

Ultimately, what we found is that making a visual element's hit testable bounds larger without increasing the size of the element is actually the hack, and not the other way around. While some wiggle room is okay, It's not intuitive if the two don't agree. The correct solution is to adjust the designs and UX.

If the widget needs small adjustments, use padding. This is the approach we've taken with many of the material widgets. This can be configured via https://docs.flutter.dev/flutter/material/MaterialTapTargetSize-class.html in the theme, (so that we can remove the padding on desktop platforms).
(Caution) you may increase the size of a child hit testable region to at least the size of a parent. This is done for the dismissible section of the material chips. The render object that wraps the chip redirects taps from the right third of the chip to make it easier to tap.

flutter/packages/flutter/lib/src/material/chip.dart

Line 1719 in 0bf0f5c

class _ChipRedirectingHitDetectionWidget extends SingleChildRenderObjectWidget {

Oh, and the other issue with increasing hit test area: it's not possible to guarantee that these hit test regions do not overlap, unless we did another layout pass

@jonahwilliams

@jonahwilliams In the cases we would need this, the parent is limiting the size the child can be expanded to, so your enumerated solutions don't work there. Saying the problem is the UI design sounds a bit too easy for me. There are cases where element are far enough away in the design, but where the natural structure of widgets make it hard to expand tappable widgets. For example cards in a sliver grid with axis spacing: Making cards invisibly bigger and reducing the spacing is an awkward solution.

how do you adjust hit testing to work without requiring a single gesture to hit test every single render box? Once the layout bounds and the hit testable bounds don't line up you always need to walk the tree.

Widgets could hitTest their children when the event happens within their bounds extended by a small margin (44/2 or 48/2). Or widgets could report a sizeForHitTesting upstream besides the normal size.

Being able to modify the hitTestArea of a widget would also solve the issues with OverflowBox, where anything in the "overflow" area can't receive any hitTest events, since it's outside of the parent's boundaries.

Kind of hack, but works well. The main idea is: add a Member into GestureArena.

import 'package:flutter/gestures.dart';
import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';
import 'package:flutter/widgets.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) => MaterialApp(home: HitTest());
}

class HitTest extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Material(
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.stretch,
        mainAxisAlignment: MainAxisAlignment.spaceAround,
        children: [
          SizedBox(
            width: 100,
            height: 100,
            child: GestureDetector(
              onTap: () => print('Tap Yellow'),
              child: Container(
                alignment: Alignment.bottomCenter,
                color: Colors.yellow,
                height: 100,
                width: 100,
                child: ExpandedHitTestArea(
                  child: Container(width: 20, height: 10, color: Colors.red),
                  onTap: () => print("I'm hit! I'm hit!"),
                ),
              ),
            ),
          ),
          Container(
            height: 100,
            color: Colors.black12,
            child: Padding(
              padding: const EdgeInsets.all(8.0),
              child: Column(
                crossAxisAlignment: CrossAxisAlignment.start,
                children: <Widget>[
                  ExpandedHitTestArea(
                    child: Text("<"),
                    onTap: () => print("Tapped"),
                  ),
                  SizedBox(height: 20),
                  Text("Title"),
                ],
              ),
            ),
          ),
        ],
      ),
    );
  }
}

class ExpandedHitTestArea extends SingleChildRenderObjectWidget {
  final VoidCallback onTap;
  ExpandedHitTestArea({
    Key key,
    Widget child,
    this.onTap,
  }) : super(key: key, child: child);

  @override
  RenderObject createRenderObject(BuildContext context) => RenderExpandedHitTestArea(onTap: onTap);
}

class TestGestureArenaMember extends GestureArenaMember {
  TestGestureArenaMember({
    this.onTap,
  });
  final VoidCallback onTap;
  @override
  void acceptGesture(int key) {
    if (this.onTap != null) {
      this.onTap();
    }
  }

  @override
  void rejectGesture(int key) {}
}

class RenderExpandedHitTestArea extends RenderBox with RenderObjectWithChildMixin<RenderBox> {
  RenderExpandedHitTestArea({
    @required this.onTap,
  });
  final VoidCallback onTap;
  @override
  void performLayout() {
    child.layout(constraints, parentUsesSize: true);
    size = child.size;
  }

  @override
  void paint(PaintingContext context, Offset offset) {
    if (child != null) {
      final BoxParentData childParentData = child.parentData;
      context.paintChild(child, childParentData.offset + offset);
    }
  }

  @override
  bool hitTestSelf(Offset position) {
    return true;
  }

  @override
  void handleEvent(PointerEvent event, BoxHitTestEntry entry) {
    if (event is PointerDownEvent) {
      TestGestureArenaMember member = TestGestureArenaMember(onTap: onTap);
      GestureBinding.instance.gestureArena.add(event.pointer, member);
    } else if (event is PointerUpEvent) {
      GestureBinding.instance.gestureArena.sweep(event.pointer);
    }
  }

  @override
  bool hitTest(HitTestResult result, {Offset position}) {
    const minimalSize = 44;
    final deltaX = (minimalSize - size.width).clamp(0, double.infinity) / 2;
    final deltaY = (minimalSize - size.height).clamp(0, double.infinity) / 2;
    if (Rect.fromLTRB(-deltaX, -deltaY, size.width + deltaX, size.height + deltaY).contains(position)) {
      result.add(BoxHitTestEntry(this, position));
      return true;
    }
    return false;
  }
}

Kind of hack, but works well. The main idea is: add a Member into GestureArena.

import 'package:flutter/gestures.dart';
import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';
import 'package:flutter/widgets.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) => MaterialApp(home: HitTest());
}

class HitTest extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Material(
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.stretch,
        mainAxisAlignment: MainAxisAlignment.spaceAround,
        children: [
          SizedBox(
            width: 100,
            height: 100,
            child: GestureDetector(
              onTap: () => print('Tap Yellow'),
              child: Container(
                alignment: Alignment.bottomCenter,
                color: Colors.yellow,
                height: 100,
                width: 100,
                child: ExpandedHitTestArea(
                  child: Container(width: 20, height: 10, color: Colors.red),
                  onTap: () => print("I'm hit! I'm hit!"),
                ),
              ),
            ),
          ),
          Container(
            height: 100,
            color: Colors.black12,
            child: Padding(
              padding: const EdgeInsets.all(8.0),
              child: Column(
                crossAxisAlignment: CrossAxisAlignment.start,
                children: <Widget>[
                  ExpandedHitTestArea(
                    child: Text("<"),
                    onTap: () => print("Tapped"),
                  ),
                  SizedBox(height: 20),
                  Text("Title"),
                ],
              ),
            ),
          ),
        ],
      ),
    );
  }
}

class ExpandedHitTestArea extends SingleChildRenderObjectWidget {
  final VoidCallback onTap;
  ExpandedHitTestArea({
    Key key,
    Widget child,
    this.onTap,
  }) : super(key: key, child: child);

  @override
  RenderObject createRenderObject(BuildContext context) => RenderExpandedHitTestArea(onTap: onTap);
}

class TestGestureArenaMember extends GestureArenaMember {
  TestGestureArenaMember({
    this.onTap,
  });
  final VoidCallback onTap;
  @override
  void acceptGesture(int key) {
    if (this.onTap != null) {
      this.onTap();
    }
  }

  @override
  void rejectGesture(int key) {}
}

class RenderExpandedHitTestArea extends RenderBox with RenderObjectWithChildMixin<RenderBox> {
  RenderExpandedHitTestArea({
    @required this.onTap,
  });
  final VoidCallback onTap;
  @override
  void performLayout() {
    child.layout(constraints, parentUsesSize: true);
    size = child.size;
  }

  @override
  void paint(PaintingContext context, Offset offset) {
    if (child != null) {
      final BoxParentData childParentData = child.parentData;
      context.paintChild(child, childParentData.offset + offset);
    }
  }

  @override
  bool hitTestSelf(Offset position) {
    return true;
  }

  @override
  void handleEvent(PointerEvent event, BoxHitTestEntry entry) {
    if (event is PointerDownEvent) {
      TestGestureArenaMember member = TestGestureArenaMember(onTap: onTap);
      GestureBinding.instance.gestureArena.add(event.pointer, member);
    } else if (event is PointerUpEvent) {
      GestureBinding.instance.gestureArena.sweep(event.pointer);
    }
  }

  @override
  bool hitTest(HitTestResult result, {Offset position}) {
    const minimalSize = 44;
    final deltaX = (minimalSize - size.width).clamp(0, double.infinity) / 2;
    final deltaY = (minimalSize - size.height).clamp(0, double.infinity) / 2;
    if (Rect.fromLTRB(-deltaX, -deltaY, size.width + deltaX, size.height + deltaY).contains(position)) {
      result.add(BoxHitTestEntry(this, position));
      return true;
    }
    return false;
  }
}

I build a new package: expand_tap_area

increasing the size of the element is actually the hack

If something is less code, easier to change, easier to understand and more resilient, I don't really understand how it can be called a "hack". For better or worse, Material is a boxy ui, with paradigms that lend themselves well to this approach, but this assumption is being pushed too deep into the framework itself.

It's biased against fun playful UI's that break rules, and overlap content, and have things peek from behind other things, etc... which is a shame, cause Flutter has all the tools to make these sorts of UI's easily, but is handcuffed by this "in the box" thinking wrt hit detection.

Adobe AIR did this performantly, at 60fps, on an iPad 2, 10 years ago. I know that might not be completely fair comparison but it's hard to justify this limitation for performance reasons, in 2021, after 10 cycles of moores law.

Adobe AIR did this performantly, at 60fps, on an iPad 2, 10 years ago.

I would be interested in an apples-to-apples comparison here. What exactly did Adobe AIR do? What exactly was the performance of hit-testing in that environment? What's the performance of the equivalent in Flutter today on the same hardware?

Well, simply put: AIR allowed pixel perfect UI hit detection in at a reasonable cost, without caring about parent bounds or negative margins. Hit areas were as big as they looked visually (though this was easily overridden), if it was on-screen and visible you could touch it, and it didn't matter where you where in your parent's coordinates.

This is super flexible and robust, and really lets the author express themselves in whatever way they see fit, without battling the framework.

Here's a quick demo I put together, close to 1000 buttons in each.
https://user-images.githubusercontent.com/736973/124080755-f7af0200-da07-11eb-92e2-40fa704a738b.mp4

Both AIR and Flutter use about 3% of my CPU at peak when doing mouse-overs for 10 seconds. The AS3 buttons are offset in negative space in their parent, just to show it does not matter.

If I drop the button count to a more reasonable 40 buttons, we see about 0.5% CPU usage, and again both mechanisms seem to have similar performance.

AS3:
https://github.com/esDotDev/air-flutter-hit-tests/blob/master/air_hit_tests/src/Main.as
https://github.com/esDotDev/air-flutter-hit-tests/blob/master/air_hit_tests/src/Button.as

Flutter:
https://github.com/esDotDev/air-flutter-hit-tests/blob/master/flutter_hit_tests/lib/main.dart

How deep is the tree in AS3? What was its hit testing algorithm? How did it determine sibling order (e.g. when two "cousin" buttons overlap, which one gets the events)? Were all objects able to get events or only those who registered as event targets? How efficient were tree updates (e.g. reordering buttons, or moving them about such that the hit order was different)?

https://pub.dev/packages/extra_hittest_area

This works for me

Since flutter framework has more or less decided not to fix this, i've come up with my own solution which is a hacky way of doing things but for now this is the best approach for me

import 'package:flutter/material.dart';

//
// Adapted from
// https://medium.com/@diegoveloper/flutter-widget-size-and-position-b0a9ffed9407
// https://medium.com/saugo360/https-medium-com-saugo360-flutter-using-overlay-to-display-floating-widgets-2e6d0e8decb9
//
// used to increase toucharea of components
// example usage:
// CustomOverLayComponent(
//                   padSize: Size(20, 20),
//                   onTap: () {
//                     print('on tap received');
//                   },
//                   onPanUpdate: (details) {
//                     print('pan update');
//                   },
//                   child: Container(
//                     width: 1,
//                     color: Colors.blue,
//                   )),
//
// The container will be padded to width size of 21 instead of just 1
// allowing for a larger touch area
//
class CustomOverLayComponent extends StatefulWidget {
  final Size padSize;
  final Function? onTap;
  final Function? onPanUpdate;
  final bool debugMode;
  final Widget child;
  const CustomOverLayComponent(
      {super.key, required this.child, required this.padSize, this.onTap, this.onPanUpdate, this.debugMode = false});

  @override
  State<CustomOverLayComponent> createState() => _CustomOverLayState();
}

class _CustomOverLayState extends State<CustomOverLayComponent> {
  final GlobalKey _globalKey = GlobalKey();
  final LayerLink _layerLink = LayerLink();
  OverlayEntry? _overlayEntry;
  bool get isOverlayShown => _overlayEntry != null;
  double top = 0;
  double left = 0;
  double height = 0;
  double width = 0;

  _getSizes() {
    final RenderBox renderBoxRed = _globalKey.currentContext!.findRenderObject() as RenderBox;
    final sizeRed = renderBoxRed.size;

    height = sizeRed.height + widget.padSize.height / 2;
    width = sizeRed.width + widget.padSize.width / 2;
  }

  _getPositions() {
    final RenderBox renderBoxRed = _globalKey.currentContext!.findRenderObject() as RenderBox;
    final positionRed = renderBoxRed.localToGlobal(Offset.zero);
    top = positionRed.dy - widget.padSize.height;
    left = positionRed.dx - widget.padSize.width;
  }

  void removeOverlay() {
    _overlayEntry?.remove();
    _overlayEntry = null;
  }

  void toggleOverlay(Widget child) => isOverlayShown ? removeOverlay() : _insertOverlay(child);

  void _insertOverlay(Widget child) {
    _overlayEntry = OverlayEntry(
      builder: (_) => _dismissibleOverlay(child),
    );
    Overlay.of(context)?.insert(_overlayEntry!);
  }

  Widget _dismissibleOverlay(Widget child) => Stack(
        children: [
          Positioned(
            top: top,
            left: left,
            child: CompositedTransformFollower(
              link: _layerLink,
              showWhenUnlinked: false,
              offset: Offset(-(widget.padSize.width / 4), -(widget.padSize.height / 4)),
              child: Container(
                height: height,
                width: width,
                color: widget.debugMode ? Colors.red.withOpacity(0.5) : Colors.transparent,
                child: GestureDetector(
                  // onTap: removeOverlay,
                  onTap: (() {
                    if (widget.onTap != null) widget.onTap!();
                  }),
                  onPanUpdate: ((details) {
                    if (widget.onPanUpdate != null) widget.onPanUpdate!(details);
                  }),
                ),
              ),
            ),
          ),
          // child,
        ],
      );

  @override
  void initState() {
    super.initState();
    Future.delayed(const Duration(milliseconds: 0), (() {
      _getSizes();
      _getPositions();
      toggleOverlay(widget.child);
    }));
  }

  @override
  void dispose() {
    removeOverlay();
    super.dispose();
  }

  @override
  void didChangeDependencies() {
    removeOverlay();
    super.didChangeDependencies();
  }

  @override
  Widget build(BuildContext context) {
    return CompositedTransformTarget(
      key: _globalKey,
      link: _layerLink,
      child: widget.child,
    );
  }
}

Flutter needs to support this io. Hit test will be failed after transformation when the Box goes outside. I am making a physic engine using flutter. Elements are transformed using translate and rotationto display correctly. I can't find a way to make it work with gesture detection.

Any news here?

@matthew-carroll

@matthew-carroll did you find a solution for this?
MouseRegions onHover and onEnter happen only when cursor is exactly on the 1px divider which is not ideal. Making the divider larger than 1px is not a solution, it takes unwanted real estate.

Secondly, when we use the GestureDetectors onPanUpdate to slide/move the panels, onExit fires and user loses the visual clue (I make the divider different color on onHover)

I'm working on a resizable panel system, like all the resizable areas in IDEs and numerous other app categories.

Resize boundaries show a resize cursor when you get close to the divider. Trying to use a MouseRegion for the divider is a problem, because the divider is only a couple pixels wide

@aytunch

@aytunch I never found any built-in solution. I think I paused my work on the resizable panel system. I'll probably come back to it.

I think my approach at this point would be to place an invisible hit area in the app Overlay, which is aligned with the divider widget that changes color when you hover over it.

This alignment might be accomplished with CompositedTransformTarget and CompositedTransformFollower. The Flutter Bounty Hunters also has our own version of those widgets that add more control over positioning, called Leader and Follower which are available in follow_the_leader: https://pub.dev/packages/follow_the_leader

Would you like the Flutter Bounty Hunters to build a package specifically for resizable panels for desktop apps? If so, would you like to collaborate on that?

@matthew-carroll

@matthew-carroll thanks for the detailed explanation. follow_the_leader has potential to mimic the behavior we are looking for. I will need to experiment to see if it covers all edge cases. Second thing I will try will be to create a render object and make the HitTestlarger than the RenderBox. I will write my findings if I have any luck.

I am just playing around with Flutter desktop to create a panel system. This is harder than I think, but Flutter community could greatly benefit from it. I can think of so many desktop apps being built on top of such a design system (code editors, image/video/audio editors, browsers, tools, etc)

Well, simply put: AIR allowed pixel perfect UI hit detection in at a reasonable cost, without caring about parent bounds or negative margins. Hit areas were as big as they looked visually (though this was easily overridden), if it was on-screen and visible you could touch it, and it didn't matter where you where in your parent's coordinates.

This is super flexible and robust, and really lets the author express themselves in whatever way they see fit, without battling the framework.

Here's a quick demo I put together, close to 1000 buttons in each. https://user-images.githubusercontent.com/736973/124080755-f7af0200-da07-11eb-92e2-40fa704a738b.mp4

Both AIR and Flutter use about 3% of my CPU at peak when doing mouse-overs for 10 seconds. The AS3 buttons are offset in negative space in their parent, just to show it does not matter.

If I drop the button count to a more reasonable 40 buttons, we see about 0.5% CPU usage, and again both mechanisms seem to have similar performance.

AS3: https://github.com/esDotDev/air-flutter-hit-tests/blob/master/air_hit_tests/src/Main.as https://github.com/esDotDev/air-flutter-hit-tests/blob/master/air_hit_tests/src/Button.as

Flutter: https://github.com/esDotDev/air-flutter-hit-tests/blob/master/flutter_hit_tests/lib/main.dart

Adobe did it 15 years ago and Flutter still can't do it

Navigation Menu

Repository navigation

Expanded hitTest area #31728

11 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Expanded hitTest area #31728

Description

Activity

jonahwilliams commented on Apr 28, 2019

jonahwilliams commented on Apr 28, 2019

spkersten commented on May 27, 2019

TonyDowney commented on Jul 25, 2020

malikwang commented on Sep 27, 2020

malikwang commented on Sep 29, 2020

esDotDev commented on Jun 21, 2021

Hixie commented on Jun 30, 2021

esDotDev commented on Jul 1, 2021

Hixie commented on Jul 1, 2021

11 remaining items

JasCodes commented on Apr 13, 2022

forumics commented on Dec 9, 2022

shtse8 commented on Nov 16, 2023

denysbohatyrov commented on Feb 14, 2024

aytunch commented on Mar 23, 2024

matthew-carroll commented on Mar 23, 2024

aytunch commented on Mar 23, 2024

CodesbyRobot commented on Nov 22, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions