Skip to content

Active subrange within composing range should be visually highlighted in Flutter #68547

Description

@cbracken

Description

When inputting text in languages that use multi-step input (composing mode), there are two ranges involved:

  1. The overall composing range
  2. The active subrange within the composing range

This is common when inputting text using ateji in Japanese, but occurs in general with longer composing ranges, person, and place names.

To support this, we'll need some visual highlighting of the text to make clear which range is the active subrange, as well as changes to the text input protocol so that the engine can specify this range to the framework. It seems unlikely to me that users will need to programmatically control the range via TextEditingController, so ideally the API surface should be as minimal as possible.

Repro steps and expected behaviour

For example, when inputting the name 真惟佳 (Maika), one would go through the following steps:

  1. Type the phonetic kana characters まいか.
  2. Press shift-left arrow to restrict the active subrange within the composing range to the first character, ま.
  3. Select the correct completion, then hit the right arrow to switch the active range to the remainder of the composing range. At this point the composing range reads 真いか and いか becomes the active subrange.
  4. Press shift-left arrow to restrict the active range to the second character only, then select the correct completion, then hit right arrow to switch the active range to the final character. The composing region now read 真惟か and か becomes the active subrange.
  5. Select the correct completion, then hit return to commit the composing region. The text 真惟佳 is now committed.

Visual representation

The composing region and active subrange are each visually highlighted to the user in some platform-specific way.

On macOS 11.0, Chrome, and Android, it is highlighted using a thicker underline with a small gap between it and the thinner composing region underline. In Linux GTK apps (and I believe older versions of macOS), the subrange is highlighted with a lighter colour than the standard selection colour.

macOS 11.0:
image

iPadOS 14.0:
image

Chrome on Linux:
image

Linux GTK apps:
image

Activity

  1. added
    a: text inputEntering text in a text field or keyboard related problems
    frameworkflutter/packages/flutter repository. See also f: labels.
    engineflutter/engine related. See also e: labels.
    on Oct 20, 2020
  2. changed the title [-]Active subrange within composing range is not visually represented in Flutter[/-] [+]Active subrange within composing range should be visually highlighted in Flutter[/+] on Oct 20, 2020
  3. cbracken commented on Oct 20, 2020

    @cbracken
    MemberAuthor
  4. cbracken commented on Oct 20, 2020

    @cbracken
    MemberAuthor

    I believe this is a requirement for Desktop launch in CJK locales.

  5. added
    P1High-priority issues at the top of the work list
    on Oct 20, 2020
  6. HansMuller commented on Oct 20, 2020

    @HansMuller
    Contributor
  7. justinmc commented on Oct 20, 2020

    @justinmc
    Contributor

    Is this something we're also missing on mobile?

  8. cbracken commented on Oct 21, 2020

    @cbracken
    MemberAuthor

    Is this something we're also missing on mobile?

    Yes. This is required on devices using a physical keyboard. I've added a screenshot of the UI for iPadOS 14. I haven't tested on an iPhone using a bluetooth keyboard but that is a supported configuration.

    I should have added a Windows screenshot today, but I can verify it looks effectively like Chrome.

  9. cbracken commented on Mar 11, 2021

    @cbracken
    MemberAuthor

    Example video on macOS of the way users can move the active region around the comping region, and how that affects candidate menu position to give a better idea:

    macOS_ime.mp4
  10. gnprice commented on Mar 12, 2023

    @gnprice
    Member

    This issue also affects Android, even without a hardware keyboard.

    With the Gboard Japanese keyboard, the expected behavior is just like in the original description above, except that instead of shift-left arrow, you use the software keyboard's left and right arrows. When a composing region is active, these arrows move the end of the active region, and are restricted to within the composing region.

    In a native Android app, the highlighting is:

    • The whole composing region is underlined.
    • The active subregion is highlighted in an aqua-green hue. (Distinct from the selection color, which is more blue.)
    • The rest of the composing region, if there is any, is highlighted with a lighter version of the same color.

    Here's a screenshot after step 4(a) of the repro steps above: the middle of three characters is the active region, and the last two characters together are the composing region. This is on a Pixel device running Android 13. A screenshot of a selection is at the right for comparison.
    image image

    (Oddly the caret is still painted at the end of the whole composing region. I checked multiple native apps and they all have this behavior. I don't know whether that's a design choice or a bug. Edit: Probably a bug, because it turns out that in a slightly different situation the caret gets painted in a place that surely can't be intended: #122490 (comment))

    The current actual behavior in Flutter main, on Android, is that the whole composing region is underlined and there is no highlighting or anything else to indicate the active subregion:
    image

    Current behavior in more detail

    In more detail, the current actual behavior is:

    • If you ignore the visual feedback in the text field itself, and just focus on the keyboard, it actually works perfectly.

      But the feedback in the text field is confusing enough that many users may never discover that.

    • The whole composing region is underlined, and the caret is painted at the end of the composing region, regardless of where the active subregion is.

      • Unless, that is, you shrink the active subregion all the way to be collapsed at the start of the composing region. In that case the caret is painted there at the start.

        Curiously the odd Android-native behavior mentioned above, where the caret is always painted at the end despite the active subregion, has this same exception: if you move all the way to the start of the composing region, then the caret is painted there instead of at the end.

    • There is no indication of the active subregion at all. (Unless you collapse it all the way to the start.) So when you press the left arrow to try to shrink the active range, it looks in the text field like it didn't do anything.

      • (On the other hand if you look at the keyboard's suggestions, they will have updated exactly as they should. Also if you hit delete, or type more kana, the deletion or insertion happens correctly at the end of the active subregion, even though that's not where the caret appears to be.)
  11. 13 remaining items

  12. flutter-triage-bot commented on Jun 7, 2024

    @flutter-triage-bot

    The triaged-desktop label is irrelevant if there is no team-desktop label or fyi-desktop label.

  13. flutter-triage-bot commented on Nov 1, 2024

    @flutter-triage-bot

    This issue is marked P1 but has had no recent status updates.

    The P1 label indicates high-priority issues that are at the top of the work list. This is the highest priority level a bug can have if it isn't affecting a top-tier customer or breaking the build. Bugs marked P1 are generally actively being worked on unless the assignee is dealing with a P0 bug (or another P1 bug). Issues at this level should be resolved in a matter of months and should have monthly updates on GitHub.

    Please consider where this bug really falls in our current priorities, and label it or assign it accordingly. This allows people to have a clearer picture of what work is actually planned. Thanks!

  14. added
    P2Important issues not at the top of the work list
    and removed
    P1High-priority issues at the top of the work list
    on Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work lista: desktopRunning on desktopa: internationalizationSupporting other languages or locales. (aka i18n)a: text inputEntering text in a text field or keyboard related problemsengineflutter/engine related. See also e: labels.frameworkflutter/packages/flutter repository. See also f: labels.team-text-inputOwned by Text Input teamtriaged-text-inputTriaged by Text Input team

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions