How can I get the raw formatting output of commands?

osx colors special-characters termcap terminfo

6,095

Solution 1

Use script /tmp/output to start recording in a new shell, then type your commands and look in the /tmp/output file, e.g. with an editor or cat -vet. Type exit to the shell to exit the recording.

Solution 2

For ls you can do

export CLICOLOR_FORCE=X
ls -G | cat -vet

Solution 3

Probably best to run the output through a hex viewer (e.g. od, hexdump, xxd):

% man less | hexdump -C | head -5
00000000  4c 45 53 53 28 31 29 20  20 20 20 20 20 20 20 20  |LESS(1)         |
00000010  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20  |                |
*
00000040  20 20 20 20 20 20 20 4c  45 53 53 28 31 29 0a 0a  |       LESS(1)..|
00000050  0a 0a 4e 08 4e 41 08 41  4d 08 4d 45 08 45 0a 20  |..N.NA.AM.ME.E. |
%

As then you can lookup any non-printable characters in ascii(7).

As for the "doesn't work" nonsense, a quick look through the ls(1) manual results in:

% env TERM=xterm-color CLICOLOR_FORCE=1 ls -G | hexdump -C | head -3
00000000  1b 5b 33 34 6d 41 70 70  6c 69 63 61 74 69 6f 6e  |.[34mApplication|
00000010  73 1b 5b 6d 1b 5b 6d 0a  1b 5b 33 34 6d 44 65 73  |s.[m.[m..[34mDes|
00000020  6b 74 6f 70 1b 5b 6d 1b  5b 6d 0a 1b 5b 33 34 6d  |ktop.[m.[m..[34m|
%

Solution 4

Most core utilities behave differently when they output to an interactive terminal. If they detect you are outputting into a file or a pipe, they don't format (reasonably so: we don't want color escape sequences when we are processing file lists with a script). For ls, you can force it:

CLICOLOR_FORCE=1 ls -G

(That's for OSX; on Linux, that would be ls --color=always.)

Other tools have similar switches (grep, for instance).

View more solutions

6,095

1.61803

Updated on September 18, 2022

Comments

1.61803 over 1 year
I'd like to know which control sequences are sent to bash by programs to format their outputs.

For example, if I dump man less I can discern which parts are bold or underlined.
```
DDEESSCCRRIIPPTTIIOONN
       _L_e_s_s is a program similar to _m_o_r_e (1), but which allows backward  move-
       ment in the file as well as forward movement.
```
But if I send ls -G's output to a file there's no data regarding its colors format.

I'm using Mac OS X.
New Atech over 8 years

The question was how can you get the raw formatting output.
jimmij over 8 years

@1.61803 Doesn't ls --color=always >file do what you want?
1.61803 over 8 years

@jimmij, no, in bash 3.2.53(1)-release (x86_64-apple-darwin12) I get only text without any control sequence
jimmij over 8 years

@1.61803 bash action is reduced here to redirect stdout of ls to a file with > so doesn't matter that much. When you open the output file in the text editor like emacs don't you see control characters?
1.61803 over 8 years

@jimmij, opened it in vi, no control characters — does it in your system?
orion over 8 years

With always switch, it should output the control characters to a file; tell us how are you viewing the file. It is possible that whatever editor you are using to view the file is ignoring the control characters. If unsure, see a hexdump. Or at least a graphical viewer to remove the possibility that the terminal is parsing the control characters again. It is also possible that your implementation of ls does not behave this way - I'm familiar with the gnu version, you are on a pretty different system, there are always some pitfalls.
jimmij over 8 years

@1.61803 oh, wait, you have apple, so ls -G instead of ls --color=always is the way to go. And concerning control characters - try ls -G | cat -vte.
1.61803 over 8 years

ls -G > file and then open with vi, no dice
1.61803 over 8 years

Great answer! One detail though is, my aliases and functions stopped working after running script. My /etc/profile sources env vars, aliases and functions from a file in a custom location.
meuh over 8 years

By default, script uses $SHELL as the command to run. You might have more success starting a login shell, eg script -c 'bash -l' /tmp/output.
1.61803 over 8 years

My version of script doesn't have -c option, but allows the syntax script [-akq] [-t time] [file [command ...]] so script typescript bash -l works fine.
Gilles 'SO- stop being evil' over 8 years

@1.61803 On OSX, you need to set the environment variable CLICOLOR=FORCE. See the man page.
Gilles 'SO- stop being evil' over 8 years

@1.61803 The version of bash is irrelevant, it is not involved here. What's relevant is that you're on OSX.
1.61803 over 8 years

@Gilles, checked the man page, doesn't work. Btw, it's not only ls I'm interested in.
1.61803 over 8 years

That's a great answer, too! What does the value X of CLICOLOR_FORCE actually do? I searched the man pages but only ls refers to it, but doesn't tell anything about that or any value. If I set to any other value it has the same effect.
fd0 over 8 years

@1.61803 or would you prefer golden mean- The X set the value of CLICOLOR_FORCE to a value other than null, as you found out.
1.61803 over 8 years

phi is always better. I just couldn't find a reference for setting the var to not null. I just made this an aliased function in my profile. Thanks.
xeruf almost 4 years

btw, for cat, -e is equivalent to -vE and -t is equivalent to -vT - so -et is the same as -vet or -vET. For understanding the coloring, -v should suffice, the others only add control characters for Tab and EOL, which can be more annoying than helpful.
xeruf almost 4 years

cat -v seems much more handy than hexdump tbh