[macOS] Backspace collapses composing region on macOS

@dkwingsmt

Repro steps:

In a textfield such as in https://github.com/cbracken/jp_flutter switch input mode to Japanese, then enter the text 日本語 (type: 'nihonngo').
Press backspace.

Expected result: Textfield contains the text にほん
Actual result: Textfield contains the text にほん日本

Taking a look at this under lldb:

The character 語 is deleted from the end of the string
The composing region is incorrectly collapsed to (0,0)
TextInputModel::UpdateComposingText is called with the new composing text (にほん)

As a result, the new composing text is injected into the string at (0,0) rather than replacing the text 日本語.

/cc @dkwingsmt since I'm wondering if this may be related to the new direct keypress handling.

Part of #30660

Looking a bit further, it seems that the composing range is being reset to (0,0) in the framework.

Here’s what I'm seeing:

updateEditState: C: (0, 3)                          // send composing range 0,3 to framework
setMarkedText: C: (0, 3) T: '日本語' MT: '日本語'     // setMarkedText from macOS, composing range still 0,3 in engine
updateEditState: C: (0, 3)                          // send composing range  0,3 to framework
setEditingState: C: (0, 0)                          // receive composing range 0,0 from framework ******
updateEditState: C: (0, 0)                          // send composing range  0,0 to framework
setMarkedText: C: (0, 3) T: 'にほん日本' MT: 'にほん'  // setMarkedText from macOS, composing range is 0,3 in engine
updateEditState: C: (0, 3)                          // send composing range 0,3 to framework

Line marked with ****** is where things have gone wrong.

Verified that the last updateEditingState I see sent from the engine and received by the framework is:

{
  text: 日本語,
  selectionBase: 3,
  selectionExtent: 3,
  composingBase: 0,
  composingExtent: 3,
  selectionAffinity: TextAffinity.downstream,
  selectionIsDirectional: false
}

On backspace, the framework sends this message from TextInput._setEditingState:

{
  text: 日本,
  selectionBase: 2,
  selectionExtent: 2,
  composingBase: -1,
  composingExtent: -1,
  selectionAffinity: TextAffinity.downstream,
  selectionIsDirectional: false
}

Doing a bit more poking to see where we're resetting the composing range.

Quick update is that the keypress event forwarding in FlutterViewController dispatchKeyEvent:(NSEvent*)event ofType:(NSString*)type is forwarding keypresses to the framework that should be handled by the IME. Ideally, while we're in composing mode, we should disable forwarding keypress events to the framework.

I don't quite get the repro steps in the issue body (does step 1 end up in composing mode or out of?), but I guess I understood that the issue is that the embedding should recognize whether a key event is for IME, and IME key events should not be sent to the framework, just like in Windows.
Question: should these events be directly cut off, just like in Windows, since they should have been handled before the app by the OS?

I don't quite get the repro steps in the issue body (does step 1 end up in composing mode or out of?)

In step one, you switch to Japanese input (in Hiragana input mode -- in the menu, usually has an 'あ' icon or 'ひらがな'), then type the letters "nihonngo" one by one. As you type, the text will transform into kana characters, so after "nihonng" you'll get "にほんg" in the field (may be slightly different depending on your typing history). Then when you type the "o" you'll get 日本語. You're still in composing mode at this point, since you haven't pressed return. So at this point, still in composing mode, you press backspace.

We expect the last character to be deleted and for the textfield to contain "にほん” at this point. Instead, it contains にほん日本 due to the composing region being reset in the framework, and then the engine trying to replace the composing region with the new composing text (にほん) from macOS . I'll try to send a framework patch to fix that, since we shouldn't be resetting the composing region on backspace either way.

I did try inserting some code to block sending IME keystrokes to the framework at the spot I mentioned above, but I haven't found a nice macOS API that tells you when you switch the IME to composing mode rather than direct input mode. You could try "cheating" by checking to see if [FlutterTextInputPlugin hasMarkedText] returns YES but that won't be true until after the first keystroke (e.g. "n") has been input. Since we send the events to the framework first and only hand them to the macOS responder chain once they've gone to and returned back from the framework, that check is too late.

I'll do a bit of looking to see if there's an API I can use for this. NSTextInputContext has a bunch of information that is close to what we want, but not quite exactly what we want.

@dkwingsmt

Re-assigning to @dkwingsmt since I'm unlikely to be able to take a look at this any time soon. In the digging I did, this seems like this is most likely an issue with resetting the composing region on the framework side. There are only a couple locations in the code where we either reset the composing region or replace the TextEditingValue with a new one. It looked like this was an issue with replacing the TextEditingValue as opposed to resetting the composing region explicitly.

@cbracken

I made a quick fix for a similar problem just by controlling which keys are passed to the platform by the framework: #85335

Kind of like @cbracken mentioned at the bottom of #78061 (comment). Not sure if that PR is going to pass all the tests and end up being a solid solution, but these problems are definitely related.

@LongCatIsLooong

We believe @LongCatIsLooong understands the issue and is addressing currently.

Backspace (and maybe some other keys) needs to be processed by -[NSTextInputContext handleEvent:] (or -[NSResponder interpretKeyEvent:]) first when there's composing text. Need to determine how best to resolve #30725.

I believe this is also a problem on iOS when a physical keyboard is used. See #30725 (comment).

any update?

After research. I found that this issue occurs because flutter receive keyborad events before the input method on macOS.

For example. when the TextField is in the Composing state and then user presses the backspace key, the flutter framework will consume the delete event, so the composing text on the compsing floating window cannot be deleted.

Fortunately, all actions(delete, forward) in the TextFiled in flutter can be overridden, so I wrote a simple workaround for this issue.

Here is the code of the workaround. MixinNetwork/flutter-app#535

And it seems works nice on my app.

2022-02-25.5.18.40.mov

Hope ths workaround helps the guy who is suffering from this issue.

This should be fixed by flutter-team-archive/engine#32051 and should be available on the framework master branch soon.

This should be fixed by flutter/engine#32051 and should be available on the framework master branch soon.

nice work!!

But there seems to be a small problem here. I post the details to this new issue #100272

@boyan01

@boyan01 thanks for filing! We'll take a look!

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

[macOS] Backspace collapses composing region on macOS #78061

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[macOS] Backspace collapses composing region on macOS #78061

Description

Activity

cbracken commented on Mar 12, 2021

cbracken commented on Mar 12, 2021

cbracken commented on Mar 13, 2021

dkwingsmt commented on Mar 13, 2021

cbracken commented on Mar 13, 2021

cbracken commented on Jun 1, 2021

2 remaining items

justinmc commented on Jun 25, 2021

eseidelGoogle commented on Oct 19, 2021

LongCatIsLooong commented on Oct 20, 2021

justinmc commented on Oct 21, 2021

chenbaiyu0414 commented on Feb 12, 2022

boyan01 commented on Feb 25, 2022

cbracken commented on Mar 16, 2022

boyan01 commented on Mar 17, 2022

cbracken commented on Mar 17, 2022

github-actions commented on Mar 31, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions