How to do: underline, bold, italic, strikethrough, color, background, and size in Gnome Terminal?

Question

How do you merge two data tables (or data frames) in R keeping the non-NA values from each matching column? The question Merge data frames and overwrite values provides a solution if each individual column is specified explicitly (as far as I can tell, at least). But, I have over 40 common columns between the two data tables, and it is somewhat random which of the two has an NA versus a valid value. So, writing ifelse statements for 40 columns seems inefficient.

Below is a simple example, where I'd like to join (merge) the two data.tables by the id and date columns:

dt_1 <- data.table::data.table(id = "abc",
                               date = "2018-01-01",
                               a = 3, 
                               b = NA_real_,
                               c = 4, 
                               d = 6,
                               e = NA_real_)
setkey(dt_1, id, date)

> dt_1
    id       date a  b c d  e
1: abc 2018-01-01 3 NA 4 6 NA

dt_2 <- data.table::data.table(id = "abc", 
                               date = "2018-01-01",
                               a = 3, 
                               b = 5,
                               c = NA_real_, 
                               d = 6,
                               e = NA_real_)
setkey(dt_2, id, date)
> dt_2
    id       date a b  c d  e
1: abc 2018-01-01 3 5 NA 6 NA

Here is my desired output:

> dt_out
    id       date a b c d  e
1: abc 2018-01-01 3 5 4 6 NA

I've also tried the dplyr::anti_join solution from left_join two data frames and overwrite without success.

Solution 1

The ANSI/VT100 terminals and terminal emulators are not just able to display black and white text; they can display colors and formatted texts thanks to escape sequences. Those sequences are composed of the Escape character (often represented by "^[" or "Esc") followed by some other characters: "Esc[FormatCodem".

In Bash, the character can be obtained with the following syntaxes:

\e
\033
\x1B

The commands (for easy copy-paste):

echo -e "\e[1mbold\e[0m"
echo -e "\e[3mitalic\e[0m"
echo -e "\e[3m\e[1mbold italic\e[0m"
echo -e "\e[4munderline\e[0m"
echo -e "\e[9mstrikethrough\e[0m"
echo -e "\e[31mHello World\e[0m"
echo -e "\x1B[31mHello World\e[0m"

Source (including all types of foreground/background color codes): http://misc.flogisoft.com/bash/tip_colors_and_formatting

Solution 2

To extend Sylvain's answer, some helper functions:

ansi()          { echo -e "\e[${1}m${*:2}\e[0m"; }
bold()          { ansi 1 "$@"; }
italic()        { ansi 3 "$@"; }
underline()     { ansi 4 "$@"; }
strikethrough() { ansi 9 "$@"; }
red()           { ansi 31 "$@"; }

Then

Solution 3

GNOME Terminal 3.28 (VTE 0.52), debuting in Ubuntu 18.04 LTS, adds support for a few more styles including curly and colored underlines as seen in Kitty, overline as seen in Konsole, and finally everyone's much loved or much hated blink attribute as well.

These also automatically work in any other VTE-based terminal emulator (e.g. Tilix, Terminator, Xfce4-terminal, Guake etc.), given that VTE is at least at version 0.52.

Here's a list demonstrating the standard escape sequences, as well as GNOME Terminal's (VTE's) additions. Note that for every opening sequence I'm also showing the closing sequence of that property only, rather than the generic \e[m or \e[0m that disables all special modes.

echo -e '\e[1mbold\e[22m'
echo -e '\e[2mdim\e[22m'
echo -e '\e[3mitalic\e[23m'
echo -e '\e[4munderline\e[24m'
echo -e '\e[4:1mthis is also underline (new in 0.52)\e[4:0m'
echo -e '\e[21mdouble underline (new in 0.52)\e[24m'
echo -e '\e[4:2mthis is also double underline (new in 0.52)\e[4:0m'
echo -e '\e[4:3mcurly underline (new in 0.52)\e[4:0m'
echo -e '\e[5mblink (new in 0.52)\e[25m'
echo -e '\e[7mreverse\e[27m'
echo -e '\e[8minvisible\e[28m <- invisible (but copy-pasteable)'
echo -e '\e[9mstrikethrough\e[29m'
echo -e '\e[53moverline (new in 0.52)\e[55m'

echo -e '\e[31mred\e[39m'
echo -e '\e[91mbright red\e[39m'
echo -e '\e[38:5:42m256-color, de jure standard (ITU-T T.416)\e[39m'
echo -e '\e[38;5;42m256-color, de facto standard (commonly used)\e[39m'
echo -e '\e[38:2::240:143:104mtruecolor, de jure standard (ITU-T T.416) (new in 0.52)\e[39m'
echo -e '\e[38:2:240:143:104mtruecolor, rarely used incorrect format (might be removed at some point)\e[39m'
echo -e '\e[38;2;240;143;104mtruecolor, de facto standard (commonly used)\e[39m'

echo -e '\e[46mcyan background\e[49m'
echo -e '\e[106mbright cyan background\e[49m'
echo -e '\e[48:5:42m256-color background, de jure standard (ITU-T T.416)\e[49m'
echo -e '\e[48;5;42m256-color background, de facto standard (commonly used)\e[49m'
echo -e '\e[48:2::240:143:104mtruecolor background, de jure standard (ITU-T T.416) (new in 0.52)\e[49m'
echo -e '\e[48:2:240:143:104mtruecolor background, rarely used incorrect format (might be removed at some point)\e[49m'
echo -e '\e[48;2;240;143;104mtruecolor background, de facto standard (commonly used)\e[49m'

echo -e '\e[21m\e[58:5:42m256-color underline (new in 0.52)\e[59m\e[24m'
echo -e '\e[21m\e[58;5;42m256-color underline (new in 0.52)\e[59m\e[24m'
echo -e '\e[4:3m\e[58:2::240:143:104mtruecolor underline (new in 0.52) (*)\e[59m\e[4:0m'
echo -e '\e[4:3m\e[58:2:240:143:104mtruecolor underline (new in 0.52) (might be removed at some point) (*)\e[59m\e[4:0m'
echo -e '\e[4:3m\e[58;2;240;143;104mtruecolor underline (new in 0.52) (*)\e[59m\e[4:0m'

(*) Truecolor values for underlines are slightly approximated.

And a bit odd one that doesn't quite fit in this picture, as it's more of a functionality than a style, yet is probably worth mentioning here, is hyperlink support co-designed with iTerm2, available since GNOME Terminal 3.26 (VTE 0.50):

echo -e '\e]8;;http://askubuntu.com\e\\hyperlink\e]8;;\e\\'

Here's a screenshot demonstrating the result:

Solution 4

Something that has not been covered yet is the combination of two or three parameters, e. g. bold and underline, in a predefined color. This is achieved by a 3-way syntax, for instance:

~$ printf "\e[3;4;33mthis is a test\n\e[0m"

will cause "this is a test" to be printed in yellow color (33m), italic (3m) AND underlined (4m).
Note that it is not necessary to repeat the \e[ every time.
Note too that (alike to Sylvain) I also added a \e[0m to reset settings every time, because otherwise the yellow color and the font style will remain active in terminal! Needless to say that you absolutely have to watch out for these to get reset in scripts, because users who use your scripts may dislike it if your script permanently modifies their color + style settings in terminal!

Solution 5

Replace these hard-coded sequences by:

tput smul # set underline
tput rmul # remove underline

tput smso # set bold on
tput rmso # remove bold

tput setaf 1 #red
tput setaf 2 #green
...
tput cup 0 0 # move to pos 0,0

Refer to "man terminfo" and "man tput" for complete descriptions of these commands.

Example :

function f_help
{
  c_green=$(tput  setaf 2      2>/dev/null)
  c_reset=$(tput  sgr0         2>/dev/null)
  c_bold=$(tput smso           2>/dev/null)
  echo "${c_bold}DESCRIPTION${c_reset} : .... ${c_green}My green text${c_reset}My plain text"
}

View more solutions

Comments

[Radu Rădeanu]

And what exactly want to change? The prompt, the text you enter, the output of your commands?

[Anon]

export PS1 actually. Although I'd like it for basic echo as well.

[Jürgen A. Erhard]

If it supported "Faint", I'd switch. :(

[Hamed]

What happens if you have non-NA values with the same id for the same variable? Are you happy to keep the one from your 'left' table?

[Ursus Frost]

@Hamed Great question. I was thinking about that but didn't want to over-complicate things. Yes, let's assume we keep the left table's values. Thanks!

[Natim]

You can use echo -e

[Anon]

I added three more for size, font, and background. Are these possible?

[Sylvain Pineau]

@Akiva You can easily change the background color (See the dedicated section). Regarding size, I don't think it's possible. For font, the only setting is a global gconf value (/apps/gnome-terminal/profiles/Default/font)

[Sylvain Pineau]

@Akiva: I'd like to know what's missing^

[Anon]

Oh I didnt realize that I did this. I think I must have accepted it, then had an old page where it showed it was not accepted, and accepted it again :P

[dashesy]

What about overline, is it possible?

[Sylvain Pineau]

@dashesy: According to en.wikipedia.org/wiki/ANSI_escape_code, 53 should do the overline. But in my terminal is does nothing.

[sherrellbc]

Is the trailing m just used as a delimiter? I could not find any documentation discussing it.

[Sylvain Pineau]

@sherrellbc a delimiter indeed (The "m" terminates each term of the escape sequence, according to tldp.org/LDP/abs/html/colorizing.html)

[Benubird]

None of the \e codes work for me - only the colors (\x). I'm using GNU bash, version 3.2.57

[egmont]

@dashesy konsole is the only terminal emulator I'm aware of that supports overline (SGR 53). What would you use this for? I'm considering implementing it in gnome-terminal, technically it's a piece of cake, but I wouldn't want to add features that no one would use.

[dashesy]

@egmont I remember I wanted to use it in PS1 (in gnome-terminal actually) because underline kind of blended in the text below it making it harder to read, and of course strike-through looked just wrong. Having a line there would help spotting the previous commands when scrolling up (so does color).

[egmont]

@dashesy Thanks, it's actually a pretty cool use case :) Gnome-terminal progress is tracked here.

[Ken Sharp]

All work but italic for some reason. GNU bash, version 4.3.48(1)-release

[Ken Sharp]

Never mind, I think my problems stem from using the Gnome/MATE terminal.

[egmont]

@KenSharp GNOME Terminal has supported italic text for like 5 years now; recent (GTK3-based) versions of MATE Terminal also do.

[Ken Sharp]

Hmmmm. I don't know then. I'll have to look into it more.

[Marius Gedminas]

A screenshot of the result would be lovely, for those of us who don't have these gnome-terminal/vte versions.

[egmont]

@MariusGedminas Done.

[Tony Barganski]

On my macOS High Sierra, echo -e does Not work but using printf Does work.

[Frank]

Or dplyr::coalesce(dt_1, dt_2)?

[Ursus Frost]

As @Frank alluded to, I'm not seeing a practical difference in the results from just using dplyr::coalesce. Am I missing something? Nonetheless, coalesce is a helpful function I wasn't previously aware of, so thank you!

[filbranden]

Awesome answer! Thanks for taking the time to collect all these! I wonder if they also work on iTerm2 (will try to check that for myself shortly...)

[user]

Doing it in Python: stackoverflow.com/questions/6199285/…

[Old Pro]

Note that smso sets standout not bold. standout is usually the combination of reverse and bold. There is no standard tput/terminfo/termcap capability for turning off bold, however ECMA-48:1991 defines SGR 22 as "normal colour or normal intensity (neither bold nor faint)" so you can reasonably use echo -ne "\\033[22m" in bash.

[Pierre-Antoine Guillaume]

echo "$(bold "this doesn't $(italic really) work for all purposes though")"

[Melab]

\e[0m clears everything, including colors that have been set and other font types that may want to be retained, so are there escape codes for undoing specific font types?