Console.ReadKey returns incomplete information for some key sequences on Linux

AB#1115513
Using PowerShell 6.0.0-alpha.10 on Ubuntu 14.04, I run

PS> [Console]::ReadKey()

and type Ctrl-@. I get back

KeyChar Key Modifiers
------- --- ---------
        0   0

I see somewhat similar results with other key presses like Ctrl-! or Ctrl-%. In some cases, KeyChar has the correct key, but Modifiers is still 0, resulting in incorrectly assuming Ctrl was not down and e.g. inserting a ! instead of doing whatever action Ctrl+! was bound to.

This issue affects some of the key bindings in PSReadline.

In general we don't have a good way to do better than this. They're is no API similar to what Windows has for getting detailed information on the keys that were typed; we simply do our best to infer it from the escape sequences sent to the terminal.

There are key bindings that work fine in bash but not with PSReadline, e.g. C-] is character search (a little like F3 on Windows). C-@ is also used in bash to set a mark, then C-x,C-x to exchange the point and mark.

GNU readline understands these escape sequences, so presumably .Net core can as well.

GNU readline understands these escape sequences, so presumably .Net core can as well.

If you can make it work, we'd welcome a PR. This is a known limitation we shipped with (mono has a similar limitation), and this is not something we plan to revisit soon.

ps I should clarify. I haven't checked the exact combinations you're talking about. If those combinations result in a unique sequence of escape codes we can reliably recognize, then fixing it should be relatively easy. My concern is they don't, and an entirely different mechanism would be needed.

Assuming I can attempt to see the sequences easily via Console.In.Read, the keys I'm concerned about don't actually generate anything, so some investigation is necessary. I'm not sure if it's something stty controls.

At any rate, I'll try and investigate further at some point, I expect some PowerShell users will want undo to work properly on the command line on Linux.

@lzybkr

@lzybkr were you able to make any progress?
cc: @ianhays

The affected key bindings aren't too critical, so I haven't found time to take a look yet.

@carlossanlop

A quick update - showkeys -a shows me that there are distinct sequences for some keys that we care about - below I typed a character followed by Alt and that character.

jason@jason:~$ showkey -a

Press any keys - Ctrl-D will terminate this program

,        44 0054 0x2c
^[,      27 0033 0x1b
         44 0054 0x2c
<        60 0074 0x3c
^[<      27 0033 0x1b
         60 0074 0x3c
.        46 0056 0x2e
^[.      27 0033 0x1b
         46 0056 0x2e
>        62 0076 0x3e
^[>      27 0033 0x1b
         62 0076 0x3e
/        47 0057 0x2f
^[/      27 0033 0x1b
         47 0057 0x2f
?        63 0077 0x3f
^[?      27 0033 0x1b
         63 0077 0x3f

After reviewing the code, I think I believe the bug is related to this method - it doesn't handle most of these characters, it returns default(ConsoleKey), which means the return from here is false, and then here we fall through to the !alt case and return just the Escape character.

It's not that useful to expect a valid ConsoleKey for these characters because it depends on the keyboard layout. On Windows, I'm forced to use the ConsoleKey because the KeyChar is sometimes '\0', but it doesn't have to be this way, so I'm open to returning a ConsoleKeyInfo with the ConsoleKey set to default(ConsoleKey).

Edit by @carlossanlop: since we moved to dotnet/runtime, the StdInReader.cs method mentioned above, returning default(ConsoleKey), is located here.

@lzybkr

Below is a summary of the existing shortcomings and how they relate to how things work in the Unix world.

Please note:

I haven't looked at the source code, so some of this is speculative. Do tell me where I'm wrong.

Context:

The fundamental restrictions regarding Control-based chords and Alt-based chords - which come from most Unix-like OSs running xterm-emulation terminal programs are:

Control-based chords are in effect limited to the 33 well-defined chords of caret notation - plus a few effective (but presumable accidental) aliases (see bottom); the ^ symbol is known as the caret and represents the Control key; for instance, ^A represents the chord Control+A (strictly speaking, it is the uppercase A, but at least for letters it doesn't matter whether Shift is held down or not):
- In short, these well-defined chords map onto the 32 C0 ASCII-range control characters starting with (0x0) NUL / ^A through (0x1f) US) / ^_, plus (0x7F) DEL / ^?) and programs running inside that terminal see only the resulting control character.
There are no Alt-based chords at all, but most modern terminal programs translate Alt+{char}-based chords (with some terminals on an opt-in basis, as on macOS) into Esc, {char} sequences, and programs running inside that terminal see only the resulting sequence.
That's why, in the bash world, for instance, readline definitions are expressed in terms of the latter.
(Additionally, a given terminal program itself may have key bindings for Control-based and/or Alt-based chords, which may preempt any attempts to use these chords by programs running inside these terminals.)

Shortcomings of the existing `[Console]::ReadKey()` implementation on Unix-like platforms:

^{Note: I'm using lowercase letters in the chord representations below to indicate chords not involving the Shift key.}

On Unix-like platforms, instead of letting programs see the translation of key chords the same way as described above - which may be a single ASCII control character or an Esc, {char} sequence - CoreFX selectively retranslates that into a chord, as you would see it on Windows_:

`Control`-based chords

Sensibly, Control-based chords that translate into control characters that have dedicated keys are reflected as such in the .Key property:
- Control+i a.k.a ^I, which is control char. HT, corresponds to the Tab key
- Control+h a.k.a ^H, which is control char. BS, corresponds to the Backspace key
- Control+[ a.k.a ^[, which is control char. ESC, corresponds to the Escape key.
- Unfortunately, Control+j a.k.a ^J, which is control char. LF (0xA) is mistakenly conflated with Control+m a.k.a ^M, which is control char. CR (0xD): both result in the .KeyChar property containing `n (i.e., LF), and .Key containing Enter, which is incorrect:
  - Only ^M corresponds to key Enter, and the .KeyChar property should contain `r (i.e., CR), not `n.
  - ^J has the correct .KeyChar value (`n), but its .Key property should be J, and its .Modifier property should be Control.
    * Somewhat ironically, this distinction is correctly observed on Windows.
Most letter-based Control-based chords (e.g., Control+a) are passed through the same way they are on Windows (rather than being translated into ASCII control characters).
- However, letter-based chords with uppercase letters, i.e. with Shift held down (e.g., Control+Shift+a) are currently quietly ignored.
- Additionally, curiously, Control+z, even in its "shift-less" (lowercase) variant is quietly ignored as well; note that Control+z has special meaning in traditional shells (not the terminal emulators themselves): in cmd.exe, it signals EOF during interactive input (copy con ...); in bash, it suspends the currently executing program and sends it to the background.
- For Control+c to be recognized as a keypress (as opposed to being treated as the signal to terminate a command), [console]::TreatControlCAsInput = $true must be executed first; without that, the terminating effect is curiously delayed: nothing appears to happen at first, but the very next next keypress then terminates the call, without returning anything (only the character typed is printed to the terminal).
Another quietly ignored chord is Control+\
The remaining punctuation-based chords (other than Control+[ a.k.a. Esc) pass the control character correctly in the .KeyChar property, but fail to populate the .Key and .Modifier properties:
- Control+]
- Control+^
- Control+_

`Alt`-based chords

CoreFX tries to translate Esc, {char} sequences , as reported by the terminal emulator, back into Alt+{char}-based chords, but not consistently:

Letter-based Alt-based chords, including uppercase variants, cords are correctly translated, and behave as on Windows.
By contrast, symbol-/punctuation-based Alt chords (e.g., Alt+') do appear to arrive as their raw Esc, {char} sequences, which [Console]::ReadKey() and therefore PSReadLine cannot handle: [Console]::ReadKey() only reports the Esc keypress

To predict if a given chord will currently work on Unix-like platforms, run the following from PowerShell:

# Press one key chord at a time to see its interpretation as a [ConsoleKeyInfo] instance,
# followed by the hex. representation of the resulting character (of the `.KeyChar` property).
# To exit, press Ctrl+c, then Enter.
while ($true) { $ck = [console]::readkey(); $ck | out-string; 'KeyChar code point: {0} (0x{0:x})' -f [int] [char] $ck.KeyChar }

To see what key chords [Console]::ReadKey() and therefore PSReadline should be able to handle, as @lzybkr has stated, run showkey -a in a terminal on a Linux platform (you may have to install the kbd package first).

Column 1 is the resulting character(s).
- For most Control- and Alt-based chords the character is represented in caret notation; and, as stated, since Alt+{char}-based chords are translated into Esc, then {char} sequences, you'll see the two resulting characters, the first of which is ^[, i.e., ESC (27 / 0x1b) in caret notation.
Columns 2-4 are the resulting character's code point in decimal, octal, and hexadecimal notation

The full list of caret-notation (Control-based) key chords and their - inconsistently supported - aliases:

CaretNotation TargetControlChar ShiftAlias DeFactoAliases Comment                             
------------- ----------------- ---------- -------------- -------                             
^@            0x0 (NUL)         ^`         {^<space>, ^2}                                     
^A            0x1 (SOH)         ^a                                                            
^B            0x2 (STX)         ^b                                                            
^C            0x3 (ETX)         ^c                                                            
^D            0x4 (EOT)         ^d                                                            
^E            0x5 (ENQ)         ^e                                                            
^F            0x6 (ACK)         ^f                                                            
^G            0x7 (BEL)         ^g                                                            
^H            0x8 (BS)          ^h                        NEEDED FOR NORMAL TERMINAL OPERATION
^I            0x9 (HT)          ^i                        NEEDED FOR NORMAL TERMINAL OPERATION
^J            0xa (LF)          ^j                                                            
^K            0xb (VT)          ^k                                                            
^L            0xc (FF)          ^l                                                            
^M            0xd (CR)          ^m                        NEEDED FOR NORMAL TERMINAL OPERATION
^N            0xe (SO)          ^n                                                            
^O            0xf (SI)          ^o                                                            
^P            0x10 (DLE)        ^p                                                            
^Q            0x11 (DC1)        ^q                                                            
^R            0x12 (DC2)        ^r                                                            
^S            0x13 (DC3)        ^s                                                            
^T            0x14 (DC4)        ^t                                                            
^U            0x15 (NAK)        ^u                                                            
^V            0x16 (SYN)        ^v                                                            
^W            0x17 (ETB)        ^w                                                            
^X            0x18 (CAN)        ^x                                                            
^Y            0x19 (EM)         ^y                                                            
^Z            0x1a (SUB)        ^z                                                            
^[            0x1b (ESC)        ^{                        NEEDED FOR NORMAL TERMINAL OPERATION
^\            0x1c (FS)         ^|         ^5                                                 
^]            0x1d (GS)         ^}                                                            
^^            0x1e (RS)         ^~                                                            
^_            0x1f (US)                    {^7, ^/}                                           
^?            0x7f (DEL)                   ^8

Caveats re aliases:

Alias chords ultimately map to the same, single control character, so they are effectively indistinguishable from the canonical caret-notation chord.
- This fact needs to be documented, so that users understand why different chords can trigger the same PSReadLine key binding.
To avoid confusion - and given that the aliases function inconsistently across terminal emulators - PSReadLine keybindings should only be defined in terms of the caret-notation chords.

Column ShiftAliases refers to, loosely speaking, aliases that are the "+ Shift" / uppercase equivalents, calculated by adding 0x20 to a caret-notation character's code point.
- With letter-based chords that means that whether you hold down Shift or not doesn't matter - this should apply to all Latin-alphabet-based keyboard layouts.
- With punctuation-based chords, whether you hold down Shift or not also doesn't matter with an US-English keyboard layout, except for _ and ? (the "shifted" ^_ and ^? actually produce each other, i.e. produce a different control char.)
  - On macOS, curiously, ^` for ^@ works neither in the default terminal (Terminal.app) nor in popular replacement iTerm.app (the former ignores ^` , the latter treats it as plain `); if you run an X11 xterm emulator, however, it does work.
Column DeFactoAliases lists additional aliases:
It is unclear to me why these aliases work and whether they do so accidentally. Their functioning may depend on the language-specific keyboard layout.
^space is a de-facto alias for ^@ that works in all terminals I've looked at.
By contrast, all the other ones - discovered experimentally with an US-English keyboard layout - seem to ONLY work in Linux xterm-256color terminals (and also in an X11 xterm emulator on macOS).

@lzybkr

Triage:
@lzybkr would you be interested in working on a PR for this?

This issue is likely to move to .NET 7.0, where we're anticipating a collection of Console-related work will be done. Leaving it in 6.0 for the time-being though.

@waf

@waf that's normal: In Windows the System.Console modifiers keys are detected but on Linux it only detects the Enter key and the Shift key is wrapped with Escape Sequences that needs to being decoded.

Please, correct me if I'm wrong: Linux (NET.Core) doesn't have convenient ConsoleKeyInfo modifier describer because there are problems with particular combinations detection (20-30% of all) in OS.

Is that correct?

The ConsoleKeyInfo modifiers in Linux (NET.Core) does not caught without decode the Ansi Escape Sequence. Only Windows OS caught the modifiers keys. Unfortunately, System.Console don't uses the Console Virtual Terminal Sequences on Windows, but only on Linux. Also, some keys aren't caught by Windows or Linux because these keys are used by the OS.

We are going to move this out to .NET 7 with the intention of grouping this with other Console-related efforts as being discussed in #52374.

https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1486859

On Linux the System.Console.ReadKey is sending Shift+PageDown on a Shift+CursorDown and Shift+PageUp on a Shift+CursorUp. Also it doesn't respond on pressing Shift+End, Shift+Home, Shift+PageDown and Shift+PageUp.

Update dependencies from https://github.com/dotnet/arcade build 20220…

Navigation Menu

Repository navigation

Console.ReadKey returns incomplete information for some key sequences on Linux #802

Context:

Shortcomings of the existing `[Console]::ReadKey()` implementation on Unix-like platforms:

`Control`-based chords

`Alt`-based chords

28 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Console.ReadKey returns incomplete information for some key sequences on Linux #802

Description

Activity

stephentoub commented on Oct 4, 2016

lzybkr commented on Oct 4, 2016

stephentoub commented on Oct 4, 2016

stephentoub commented on Oct 4, 2016

lzybkr commented on Oct 4, 2016

karelz commented on Nov 18, 2016

lzybkr commented on Nov 18, 2016

lzybkr commented on Oct 17, 2017

mklement0 commented on Apr 15, 2018

Context:

Shortcomings of the existing [Console]::ReadKey() implementation on Unix-like platforms:

Control-based chords

Alt-based chords

carlossanlop commented on Mar 4, 2020

28 remaining items

jeffhandley commented on Jun 24, 2021

BDisp commented on Jun 25, 2021

Alex-K-O-R commented on Jun 30, 2021

BDisp commented on Jun 30, 2021

jeffhandley commented on Jul 7, 2021

danmoseley commented on Mar 11, 2022

BDisp commented on Jun 27, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Shortcomings of the existing `[Console]::ReadKey()` implementation on Unix-like platforms:

`Control`-based chords

`Alt`-based chords