posix shell: print list of environment variable names (without values)

Question

I have hive table like the following and I need to count occurrence of a team

team_name  opponent_team    match_won
csk        mm               mm
csk        dc               csk
mm         csk              csk
dc         csk              dc  

Now I need data like:

team_name  total_matches    matches_won
csk        4                2
mm         2                1
dc         2                1

Can anybody help me with a query?

Solution 1

It's pretty easy in awk.

awk 'BEGIN{for(v in ENVIRON) print v}'

However beware some awk implementations add environment variables of their own (e.g. GNU awk adds AWKPATH and AWKLIBPATH to ENVIRON).

The output is ambiguous if the name of an environment variable contains a newline, which is extremely unusual but technically possible. A pure sh solution would be difficult. Your best bet is to start with export -p but massaging it in pure sh is difficult. You can use sed to massage the output of export -p, then use eval to get the shell to remove what it quoted. Bash and zsh print non-standard prefixes.

report () { echo "${1%%=*}"; };
eval "$(export -p | sed "s/^export /report /;
                         s/^declare -x /report /;
                         s/typeset -x /report /")"

Note that depending on the shell, export -p may or may not show variables whose name is not valid in the shell, and if it doesn't, then it may or may not quote the names properly. For example, dash, mksh and zsh omit variables whose name includes a newline, BusyBox dash and ksh93 print them raw, and bash prints them raw without their value. If you need to defend against untrusted input, don't rely on a pure POSIX solution, and definitely don't call eval on anything derived from the output of export -p.

Solution 2

Here is my solution, which nicely handles environment variables having multi-line values:

env -0 | cut -z -f1 -d= | tr '\0' '\n' | sort | column

This will print the environment using NUL to delimit variables instead of newlines (env -0).

Then we eliminate the variable values using cut -z -f1 -d=, where the -z option specifies NUL-delimited records, -f1 is the first field, and -d= says that fields are delimited by =.

tr '\0' '\n' will convert the NUL delimiters to newlines.

Then I like to sort the variable names and display them in columns.

Solution 3

I like simple things; this will work for POSIX systems:

printenv | sed 's;=.*;;' | sort
HOME
HOSTNAME
PATH
PWD
SHLVL
TERM

Comments

[Gordon Linoff]

I removed the incompatible database tags. Tag only with the database you are really using.

[Pavith]

hi Jens, since i'm new to sql i was not aware of case statements and was trying different combination with union all

[Juan]

I don't need anything to input to a shell. I just want the names of the environment variables. So the extra cruft (e.g., the 'export' word at the front of each line) would just have to be removed anyway. Why would I want to use export -p for this?

[Kusalananda]

You'd want to use export -p because that would give you a consistent output across all POSIX shells, which you said you wanted.

[Juan]

I want a solution that will print just the variable names that is a consistent solution across shells. export -p does not fit the first requirement - printing just variable names without values.

[Juan]

Fair enough, I agree export -p (make sure to set +o posix in bash, otherwise the output uses the declare -x bashism) would be a consistent starting point for name+value output for posix shell compatibility.

[Kapocsi]

@Juan I think you mean set -o posix

[Juan]

@Kapocsi Yep, I meant set -o posix

[Pavith]

Hi Jarlh, query worked perfectly fine after adding alias to the sub query. Many thanks...

[Juan]

I'm not sure how "easy" it really is with sed(1) with multiline values - I'd like to see that. But thanks for the awk(1) method. I think this is the most portable way so far (although I don't think the exit is necessary).

[Stéphane Chazelas]

Note that if you get FOO<newline>BAR, you don't know whether it's a FOO<newline>BAR environment variable (which export -p wouldn't show with most shells, see env $'FOO\nBAR=test' awk 'BEGIN{for (v in ENVIRON) print v}') or both a FOO and BAR environment variable.

[Stéphane Chazelas]

Note that GNU awks sets environment variables of its own (AWKPATH and AWKLIBPATH on my system)

[Juan]

@StéphaneChazelas - re: embedded newline in var name - agreed - hard to defend against that with shell code. Re: AWKPATH & AWKLIBPATH pollution, I noticed that insidious insertion, too. That's gnu awk, not bsd awk.

[roaima]

export AAA=$'multi\nBBB=line'

[Juan]

@todd_dsm - This fails for environment variables that have values that span more than one line (e.g., sometimes TERMCAP - I see this when in a screen(1) session, IIRC). So this answer does not meet the requirements described in the original question. I also like simple, but this does not work generally.

[Juan]

The tidbit you edited out in your original post was interesting for bash: compgen -e. It doesn't help for my portable scripting (e.g., when bash is not available), but it's interesting.

[todd_dsm]

Using dash in debian, I get the same results with the command above or, the modified printenv | sed 's;*=.;;' | sort to get the values. I exported the variable yo and assigned it your first comment above; it printed as expected, with multiple lines. not sure what you're experiencing but there shouldn't be truncated output. run the command in the shell; whatever it outputs there is how it should work; expect no truncating. Then, within the context of TERMCAP/screen/iirc; should be the same. If the output does not match then it's likely an issue with one of those programs.

[Juan]

@todd_dsm: re: why posix? For portability generally. I want my scripts to work with any shell, notably any /bin/sh whether that underlying shell is bash, dash, ksh, "standard" bourne shell. More simply put, bash is not available on many systems that my scripts must run on.

[Juan]

@todd_dsm: here's an example of the failure with multiline env vars: env ML=$'abc=123:\n def=456' sh -c 'printf "${ML}\n"; echo ; printf "$ML" | sed "s;=.*;;"; echo'

[todd_dsm]

@juan okay, you're looking for the functionality of an associative array. That's not available in ash/dash/sh. You should definitely be using something like bash. If not, those work-arounds will start getting super frustrating; good luck.

[Kusalananda]

Note that env -0 is not POSIX. Neither is cut -z or the column utility. The tr and sort commands used in your answer are however compliant.

[Stéphane Chazelas]

@Kusalananda, but here sort should go before tr and with -z. GNU sort has had -z longer before GNU cut and GNU env, that would be the least unportable of the lot.

[they]

This breaks with variables whose values contain =. Also note that -o is not standard with grep.

[TMG]

@they thanks for the comment. Good catch, I've improved the answer to make it cut on the first = in a line (for the record, before edit the answer was printenv|grep -o "^.*="|tr -d '='). Also thanks for spotting non-standard -o - I must admit it just works for me, but I have no idea how common it is.

[they]

I just mentioned the use of -o since the question asks for "a posix-compatible way [...]". Your updated answer solves the issue with = mentioned in my previous comment, but also introduces \+, which only works with GNU grep AFAIK.