Repository navigation
Support for Apple Pencil angle, pressure, and other sensor input on iPad #38135
Description
Activity
also interested if there are any plans to support apple pencil?
You can actually already use the Apple Pencil with e.g. a Listener. The PointerDown event has a kind property which corresponds to the kind of device used for the input. Just check if it's a stylus (which the Apple Pencil is) and then you can use the localPosition and pressure to do whatever you want with the pencil.
The only thing not supported is the apple pencil double tap feature. But you can certainly get around that by building a simple PlatformChannel.
Reacted by Robert Felker, mono — Masayuki Ono, Tom Schmidt, kiostudio, benzsuankularb, Nicolas Schlecker, Peter Hindes, Hiển Lê, AXBYU, Benjamin Gudehus and 9 moreReacted by Hiển LêReacted by Hiển LêInteresting, is there currently support for angle data as well?
I'm looking to target chrome and ios with pen support (chromebooks with pen support and ipads with apple pencil). Do you know if there is support for styluses when building for web?Reacted by quantumfrost, James-A-White, TJB-99, kiostudio, Mayur Dhurpate, Matt Chang and AXBYUI think this might be good to unify under a single issue of pen support on all platforms. For example what about Windows, macOS or chromeos. Or web on windows or macos? pen support should be a unified experience for developers so that they can publish their apps to all these platforms for users to use with pens and not need platform specific code for their app.
Reacted by William Palm, Damian Bast, zhengying, Bahaà Addin Balashoni, quantumfrost, Rustem Kakimov, Generalworks Inc., nitn3lav, Nico Teufel, Dan Wendon-Blixrud and 2 more- changed the title
[-]Support for Apple Pencil[/-][+]Support for Apple Pencil on iPad[/+]on Oct 22, 2020 Is this related to the Scribble feature? https://support.apple.com/guide/ipad/enter-text-with-scribble-ipad355ab2a7/ipados
The original intention was to support all the sensors like angle & force.
Should Scribble be a new bug entry ?10 remaining items
Self assigned. I am looking into this to see if it's a good project suitable for next year.
After looking into this issue, it seems to me that there is already support for angle, pressure, and altitude values on iPad. I'm using an Apple Pencil 2 with an IPad Air running iPadOS 16.3.1.
Below is the code I used to test -import 'package:flutter/material.dart'; void main() => runApp(const MyApp()); class MyApp extends StatelessWidget { const MyApp({Key? key}) : super(key: key); @override Widget build(BuildContext context) { return MaterialApp( home: Scaffold( appBar: AppBar(title: const Text("Pencil Sensor Example")), body: const Center( child: MyStatefulWidget(), ), ), ); } } class MyStatefulWidget extends StatefulWidget { const MyStatefulWidget({Key? key}) : super(key: key); @override State createState() => _MyStatefulWidgetState(); } class _MyStatefulWidgetState extends State { double pressure = 0.0; double orientation = 0.0; double tilt = 0.0; void _updateValues(PointerEvent event) { setState(() { pressure = event.pressure; orientation = event.orientation; tilt = event.tilt; }); } @override Widget build(BuildContext context) { return ConstrainedBox( constraints: BoxConstraints.tight(const Size(800.0, 800.0)), child: Listener( onPointerDown: _updateValues, onPointerMove: _updateValues, onPointerUp: _updateValues, child: Container( color: Colors.blue, child: Column( mainAxisAlignment: MainAxisAlignment.center, children: [ Text( 'PRESSURE: ${pressure.toStringAsFixed(2)}', ), Text( 'ORIENTATION: ${orientation.toStringAsFixed(2)}', ), Text( 'TILT: ${tilt.toStringAsFixed(2)}', ), ], ), ), ), ); } }There are a few caveats:
- We are reporting estimated rather than true values, but it seems to me the fidelity is already pretty good.
- Apple also reports a value that gives the orientation in form of unit vector, which we do not.
- The values that we use are offset from Apples - eg. when the pencil is perpendicular to the screen, Apple reports Pi/2, while we use 0.0.
Other than that, we might be able to close this issue as is.
@LouiseHsu Thanks for the research! And the example is very helpful.
We are reporting estimated rather than true values, but it seems to me the fidelity is already pretty good.
I think we can stick with using "estimated values", until someone explicitly ask for more accurate values in the future. (Note that true values come with a drawback of delay, according to apple's doc)
The values that we use are offset from Apples - eg. when the pencil is perpendicular to the screen, Apple reports Pi/2, while we use 0.0.
Pi/2vs0.0both seem reasonable to me. We don't have to strictly follow Apple, since other platforms could use0.0to represent perpendicular. I don't think it's worth it to have a breaking change of API for now.Apple also reports a value that gives the orientation in form of unit vector, which we do not.
This seems good to have, since you can compute
anglefrom unit vector orientation, but you can't get the unit vector from theangle. Though I'm not sure anyone actually need to use this value. Maybe we can work on after someone explicitly request this particular value?Since the existing API already provides the estimated
pressure,orientationandtiltvalues, I think we good for now. What do you think? @jmagman @leighajarettSeems like maybe this is just a documentation issue then?
Do you know if the event properties you used are also relevant for styluses used on an Android tablet?
- At the end it depends if a company would rely on Flutter to make and advanced procreate. What’s the most advanced hardware from Android as asked by Leigha to see if API can be aligned? ThxOn Fri 24 Mar 2023 at 01:04, Leigha Jarett ***@***.***> wrote: Seems like maybe this is just a documentation issue then? Do you know if the event properties you used are also relevant for styluses used on an Android tablet? — Reply to this email directly, view it on GitHub <#38135 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJ4MWL72AT6IGDIRJFZAYLW5TQJ5ANCNFSM4IK25F2A> . You are receiving this because you authored the thread.Message ID: ***@***.***>-- Robbie
Seems like maybe this is just a documentation issue then?
Do you know if the event properties you used are also relevant for styluses used on an Android tablet?
I took a quick look at the android version of the engine and it looks like those same properties are implemented as well. I don't have an Android device so I can't test it out, but I assume it works the same way for android styluses that support sensor input.
Looks like this was implemented with flutter-team-archive/engine#3955 then, #38135 (comment) did mention that it seems like this is already working.
Seems like maybe this is just a documentation issue then?
@leighajarett we could show this off in a fuller Apple Pencil demo app once double tap work is complete. If you have specific ideas how how to improve documentation in https://api.flutter.dev/flutter/gestures/PointerEvent-class.html we can definitely do that. Those docs already refer to iOS and the stylus type though.
Pencil double tap support tracked separately in #73172. If you see any bugs related to Apple Pencil angle/pressure/sensor input or Scribble, please file separately and we'll take a look.
Thanks for the investigation @LouiseHsu! Closing this feature request.
Reacted by mono — Masayuki OnoI like the idea of a demo app once double tap is working! And perhaps a doc that talks more high level about using a stylus with Flutter that links back to the API docs?
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.- locked as resolved and limited conversation to collaborators
on Apr 7, 2023
I'm working on creative apps and currently using forceTouch on iPhone. The flutter implementation is working well but I want to move my app on iPad when it's the pencil that is used to send the pressure information.
Also sadly you know that ForceTouch is leaving the Apple hardware in 2019 moving pressure to stylus only.