Highlight ASCII control characters unambiguously

[alois31]

Previously, all ASCII control characters would be shown as the replacement character. This leads to a visual loss of information and ambiguities, particularly when opening files that contain lots of control characters (or binary files). Use the common circumflex notation, known from software such as cat -v or vim, instead.

Fixes #2936

[mawww]

I am afraid this will make vertical movements a bit broken as we use the buffer line column width so we are not relying on highlighting at all for this. If we really want to change this I'd prefer to use the unicode control pictures which should not shift anything vertically.

[alois31]

Thank you for catching the breakage. Not sure if the Unicode pictures are much better than the current state, they are almost completely unreadable with many fonts in normal sizes.

[arachsys]

Random thought: would a more readable alternative to a replacement character or a tiny unicode control picture be a specially highlighted letter, e.g. ^a (optionally?) becomes an 'a' in a special face instead of �?

[Delapouite]

Talking about cat -v, bat which aims to be a "modern" alternative with bells and whistles adds a way to decide between Caret Notation and Control Pictures (␜, ␝, ␞, ␟…).

Here's the relevant extract of its man:

-A, --show-all

    Show non-printable characters like space, tab or newline. Use '--tabs' to control the width of the tab-placeholders.

--nonprintable-notation <notation>

    Specify how to display non-printable characters when using --show-all.

    Possible values:

caret
    Use character sequences like ^G, ^J, ^@, .. to identify non-printable characters
unicode
    Use special Unicode code points to identify non-printable characters

I'm not sure how such an option would translate in Kakoune world.

Not sure if the Unicode pictures are much better than the current state, they are almost completely unreadable with many fonts in normal sizes.

I agree that these chars are usually quite small, but it would nonetheless carry more information that a soulless � and when such (rare) situation happens, well you can always temporarily zoom the font size in your terminal.