[MacOS] LogicalKeyboard involving shift key throws error: Physical key for LogicalKeyboardKey not found in macos physical key map

When I try to press the exclamation logical keyboard key using WidgetTester, I get the following error:

Error: 'package:flutter_test/src/event_simulation.dart': Failed assertion: line 243 pos 12: 'result != null': Physical key for LogicalKeyboardKey#00021(keyId: "0x00000021", keyLabel: "!", debugName: "Exclamation") not found in macos physical key map

Flutter version:

Flutter 2.8.1 • channel stable • [email protected]:flutter/flutter.git
Framework • revision 77d935af4d (2 weeks ago) • 2021-12-16 08:37:33 -0800
Engine • revision 890a5fca2e
Tools • Dart 2.15.1

@justinmc

CC @justinmc @gspencergoog

@matthew-carroll

Hi @matthew-carroll, Thanks for filing the issue. Can you please share a minimal and complete reproducible code to better address the issue.

Thanks

Here's repro code:

import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets("Can't type !", (tester) async {
    await tester.sendKeyDownEvent(LogicalKeyboardKey.exclamation, platform: "macos");
  });
}

@matthew-carroll

Thanks for the sample code @matthew-carroll, I am able to reproduce the same exception on both stable and the master channel. The tests are only failing when simulating key events which require a shift key e.g *, !, ( etc.
So simulating the below key events the test fails
LogicalKeyboardKey.asterisk
LogicalKeyboardKey.exclamation
LogicalKeyboardKey.braceLeft etc

This seems to be related to #90167

logs

mahesh@Maheshs-MacBook-Air-M1 counter % flutter test
00:15 +0: Can't type !                                                                           
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
Physical key for LogicalKeyboardKey#00021(keyId: "0x00000021", keyLabel: "!", debugName:
"Exclamation") not found in macos physical key map
'package:flutter_test/src/event_simulation.dart':
Failed assertion: line 243 pos 12: 'result != null'

When the exception was thrown, this was the stack:
#2      KeyEventSimulator._findPhysicalKeyByPlatform (package:flutter_test/src/event_simulation.dart:243:12)
#3      KeyEventSimulator.getKeyData (package:flutter_test/src/event_simulation.dart:260:21)
#4      KeyEventSimulator.simulateKeyDownEvent._simulateByRawEvent.<anonymous closure> (package:flutter_test/src/event_simulation.dart:763:16)
#5      KeyEventSimulator._simulateKeyEventByRawEvent.<anonymous closure> (package:flutter_test/src/event_simulation.dart:685:65)
#6      KeyEventSimulator._simulateKeyEventByRawEvent.<anonymous closure> (package:flutter_test/src/event_simulation.dart:681:39)
#9      TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#10     KeyEventSimulator._simulateKeyEventByRawEvent (package:flutter_test/src/event_simulation.dart:681:27)
#11     KeyEventSimulator.simulateKeyDownEvent._simulateByRawEvent (package:flutter_test/src/event_simulation.dart:761:14)
#12     KeyEventSimulator.simulateKeyDownEvent (package:flutter_test/src/event_simulation.dart:768:16)
#13     simulateKeyDownEvent (package:flutter_test/src/event_simulation.dart:904:28)
#14     WidgetController.sendKeyDownEvent (package:flutter_test/src/controller.dart:1034:12)
#15     main.<anonymous closure> (file:///Users/mahesh/Desktop/counter/test/widget_test.dart:6:18)
#16     main.<anonymous closure> (file:///Users/mahesh/Desktop/counter/test/widget_test.dart:5:31)
#17     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:170:29)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from class _AssertionError, dart:async, and package:stack_trace)

The test description was:
  Can't type !
════════════════════════════════════════════════════════════════════════════════════════════════════
00:15 +0 -1: Can't type ! [E]                                                                    
  Test failed. See exception logs above.
  The test description was: Can't type !
  
00:15 +0 -1: Some tests failed.

flutter doctor -v

[✓] Flutter (Channel stable, 2.8.1, on macOS 12.1 21C52 darwin-arm, locale en-GB)
    • Flutter version 2.8.1 at /Users/mahesh/Documents/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 77d935af4d (3 weeks ago), 2021-12-16 08:37:33 -0800
    • Engine revision 890a5fca2e
    • Dart version 2.15.1

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 60.1.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.63.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.32.0

[✓] Connected device (3 available)
    • Redmi K20 Pro (mobile) • 192.168.1.2:5555 • android-arm64  • Android 11
      (API 30)
    • macOS (desktop)        • macos            • darwin-arm64   • macOS 12.1
      21C52 darwin-arm
    • Chrome (web)           • chrome           • web-javascript • Google Chrome
      96.0.4664.110

• No issues found!

[✓] Flutter (Channel master, 2.9.0-1.0.pre.205, on macOS 12.1 21C52 darwin-arm,
    locale en-GB)
    • Flutter version 2.9.0-1.0.pre.205 at /Users/mahesh/Documents/flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6ec059c994 (6 days ago), 2021-12-28 00:29:08 -0500
    • Engine revision 31fcaf3ce4
    • Dart version 2.16.0 (build 2.16.0-134.0.dev)
    • DevTools version 2.9.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 60.1.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.63.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.32.0

[✓] Connected device (3 available)
    • Redmi K20 Pro (mobile) • 192.168.1.2:5555 • android-arm64  • Android 11
      (API 30)
    • macOS (desktop)        • macos            • darwin-arm64   • macOS 12.1
      21C52 darwin-arm
    • Chrome (web)           • chrome           • web-javascript • Google Chrome
      96.0.4664.110

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

I think you've mixed "having this feature in production" with "testing this feature".

Use keyboard events

For how to develop a keyboard feature that deals with characters: The native macOS API provides two kinds of information regarding the character: characters, which would be "@" in your example, and charactersIgnoringModifiers, which would be "2" in your example. Flutter converts the former to KeyEvent.character, and the latter (roughly) to KeyEvent.logicalKey. (I'm not that sure with the legacy RawKeyboard API but I would assume it's similar; let me know if not.) So you if you want the user to input arbitrary text, append the KeyEvent.character value to your state. Or maybe you should checkout the text input API, which also handles IME?

Test keybaord events

First off, instead of WidgetController.sendKeyEvent (and siblings), you might want to check out simulateKeyDownEvent (and siblings.) They allow customizing physicalKey and character. Yes, the WidgetController.sendKeyEvent methods are just wraps over them, and it's an oversight that these APIs are not exposed. We should fix it.

Now, if that solves your problem, great. If not, I'll continue explaining.

The mapping between physical keys, logical keys, and characters, is completely arbitrary. It's entirely possible to have a LogicalKeyboardKey("@"). Sure, create it on your own, simulate it in tests, and very likely it will be sent by some embedding.

The problem is, it will never be sent by a macOS embedding. Simulating a keyboard event (for the RawKeyboard API) requires converting your key event back to a macOS native key event, senting it to RawKeyboard, which then converts it again to RawKeyEvent API. But if macOS never sends a LogicalKeyboardKey("@") value, what native event can it even be converted to?

(This will be easier to understand once we migrate everything to the HardwareKeyboard API, whose framework part knows nothing about the native information.)

Furthermore, what exactly do you want to test? The user can input "@" via a variety of ways:

macOS, US keyboard, Shift-2.
macOS, ASERTY keyboard, 2.
Windows, US keyboard, Shift-2.
macOS, custom keyboard layout, CapsLock (for random reasons.)

Do you really want to test out all the infinite cases? Or do you only care about processing key events where KeyEvent.character equals to "@", no matter what their physicalKey and logicalKey are?

That's why the key event simulation API allows customizing all 3 fields: logicalKey, physicalKey, and character. The latter 2 fields are derived by default for convenience, but you shouldn't think that the default behavior is all that Flutter assumes.

@dkwingsmt

@dkwingsmt to make sure that I'm not misunderstanding you, can you write an example WidgetTester invocation that sends the "@" character in the manner that you think a Flutter developer should use in their tests?

Is this something that might suit you? (Testing for the text API is another issue, if that's what you want, and I'm not sure if it's possible just from event simulation.)

  testWidgets('Test @ on US keybaord', (WidgetTester tester) async {
    const String platform = 'macos';
    final FocusNode focusNode = FocusNode();
    final List<RawKeyEvent> events = <RawKeyEvent>[];
    await tester.pumpWidget(
      RawKeyboardListener(
        focusNode: focusNode,
        autofocus: true,
        onKey: (RawKeyEvent event) {
          events.add(event);
        },
        child: const SizedBox(width: 100, height: 100),
      )
    );
    await simulateKeyDownEvent(LogicalKeyboardKey.shiftLeft, platform: platform);
    await simulateKeyDownEvent(LogicalKeyboardKey.at, physicalKey: PhysicalKeyboardKey.digit2, platform: platform);
    await simulateKeyUpEvent(LogicalKeyboardKey.at, physicalKey: PhysicalKeyboardKey.digit2, platform: platform);
    await simulateKeyUpEvent(LogicalKeyboardKey.shiftLeft, platform: platform);
    expect(events[1].logicalKey, LogicalKeyboardKey.at);
    expect(events[1].character, '@');
  });

@dkwingsmt

@dkwingsmt I think that would technically be work-able.

I think it's worth considering whether Flutter should provide some common keyboard configurations so that app developers don't need to manually connect the "@" character to the "2" key. That level of control might be good at the ground level, but I think it's going to feel out of place in the common case.

In fact, I think it might make sense for Flutter to support the actual use-case that I'm dealing with, which is pressing keys that correspond to arbitrary characters. If that's not something that Flutter wants to offer, then it would be helpful to know where Flutter sees the boundary between the built-in test capabilities vs individual app developer test responsibilities.

One last point, the use of the word "simulate" in these calls makes them seem fundamentally different from existing WidgetTester calls, e.g., "sendKeyEvent". Are these calls fundamentally different? If so, please be sure to clearly describe those differences in the API docs for any new methods.

Heads up: I'm changing the behavior regarding Shift'ed keys and logical keys on macOS, Linux, and Web. The new behavior aligns with that's already on Windows, and should be more predictable and intuitive, and partly align with what you said in this issue. Some examples are:

When pressing physical keys...	on layout	Previous logical keys	New logical keys
1	US	`digit1`	`digit1` (unchanged)
Shift-1	US	`shiftLeft + exclamation`	`shiftLeft + digit1`
1	French	`ampersand`	`digit1`
Shift-1	French	`shiftLeft + digit1`	`shiftLeft + digit1` (unchanged)
a	Russian	`ф`	`keyA`

This means that you will be able to write test for shiftLeft + digit1 and expect it to work.

This will probably be a breaking change, but I'm still evaluating options.

For more information and tracking, see #100456

I also encountered the same problem on Android phones with buttons
When listening for keyboard events, I am unable to listen for F12 function buttons
But when I press any number key, I can listen to F12
So when I don't press the number keys, I can't listen to buttons like F12

The triaged-desktop label is irrelevant if there is no team-desktop label or fyi-desktop label.

Navigation Menu

Repository navigation

[MacOS] LogicalKeyboard involving shift key throws error: Physical key for LogicalKeyboardKey not found in macos physical key map #96022

8 remaining items

Use keyboard events

Test keybaord events

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[MacOS] LogicalKeyboard involving shift key throws error: Physical key for LogicalKeyboardKey not found in macos physical key map #96022

Description

Activity

matthew-carroll commented on Jan 1, 2022

maheshj01 commented on Jan 3, 2022

matthew-carroll commented on Jan 3, 2022

maheshj01 commented on Jan 3, 2022

8 remaining items

dkwingsmt commented on Jan 31, 2022

Use keyboard events

Test keybaord events

matthew-carroll commented on Feb 2, 2022

dkwingsmt commented on Feb 10, 2022

matthew-carroll commented on Feb 18, 2022

dkwingsmt commented on Mar 21, 2022

han1548772930 commented on Jul 1, 2023

flutter-triage-bot commented on Jun 7, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions