Piping bash string manipulation

Question

I've read some other piping bash string manipulation questions but they seem to be specialized applications.

Essentially, is there a way to do the below simpler?

instead of

$ string='hello world'; string2="${string// /_}"; echo "${string2^^}"
HELLO_WORLD

something like

$ echo 'hello world' | $"{-// /_}" | "${ -^^}"
HELLO_WORLD

---

Edit I'm interested in staying within bash manipulations if possible to maintain speed (as opposed to sed/awk which have a tendency to greatly slow down my scripts)

Edit2: @jimmij

I like the second example and led me to making a function.

bash_m() { { read x; echo "${x// /_}"; } | { read x; echo "${x^^}"; }; }
echo hello world | bash_m
HELLO_WORLD

Solution 1

What jimmij said. His last example is the closest you can get to what you're attempting in your piped expression.

Here's a variant on that theme:

echo 'hello world'|echo $(read s;s=${s^^};echo ${s// /_})

I'd be inclined to use tr, as it's quite fast.

echo 'hello world'|tr ' [:lower:]' '_[:upper:]'

I suppose it's a shame that bash doesn't permit nested parameter expansion; OTOH, use of such nested expressions could easily lead to code that's painful to read. Unless you really need things to run as fast as possible it's better to write code that's easy to read, understand and maintain, rather than clever-looking code that's a PITA to debug. And if you really do need things to be done at top speed you should be using compiled code, not a script.

Solution 2

You cannot pass parameter expansions in such a way. When you refer to x using $ symbol as in "${x}" form, then it has to be real variable name, not a standard input, at least not in bash. In zsh you can perform nested parameter substitutions in the following way:

$ x=''hello world'
$ echo ${${x// /_}:u}
HELLO_WORLD

_{(note: :u is for zsh the same as ^^ for bash)}

Nesting in bash is not possible and I think what you wrote in question is the best one can get, but if for any strange reason you need to involve pipes into equation then you may want to try this:

$ echo 'hello world' | { read x; echo "${x// /_}"; } | { read y; echo "${y^^}"; }
HELLO_WORLD

Comments

[Ketan Maheshwari]

why do you think sed/awk would be slow for this purpose? They are as fast as they come.

[jw013]

@Ketan Sed and awk are separate processes, so they can never be fast as something that bash can do natively without launching a separate process. Normally this difference is hardly noticeable, but where performance matters in shell scripts is usually a certain loop or computation is being repeated a very large number of times, and spawning thousands of processes will be noticeably slower than doing a simple string manipulation in bash natively.

[jimmij]

@jw013 This is true for short strings as "hello world" from the question, but if string is very long, say the tr manual, then the opposite is true because time of spawning the processes is negligible in comparison to time of string manipulation for which sed and awk are dedicated. If string is extremely long, say the whole bash manual, then bash can just refuse to proceed altogether, because of some internal limitations.

[jw013]

@jimmij Are you claiming that bash's string manipulation code is so inefficient compared to sed or awk that it is more efficient to fork off an extra process to run the better code in sed / awk than it is to use bash's code? I find that to be unlikely - if bash can do something it should be faster than forking a process to do the same thing. As to the other point, if you have a string so big it won't even fit in a bash variable then the performance question is moot.

[BigE]

Many uses I have for string manip involving while/for loops, as well as being run on the rpi. using sed for a short string x100 took 4 seconds, using bash took .2 seconds. This might not make a great difference small scale, but I'll occasionally do string changes of thousands of files where it does make a difference.

[jimmij]

@jw013 I'm claiming that bash's string manipulation code is less efficient then dedicated tools as sed, awk, tr or similar. Look at the gena2x answer, which I edited some time ago adding exactly this information: unix.stackexchange.com/questions/162221/… you may want to compare it with terdon answer to the same question where he gives time for short strings in which case process spawning takes most time. You can test it yourself and post result.

[jw013]

@jimmij Fair enough. For 90% of use cases where bash performance matters, I suspect the strings will be small enough where the penalty of process creation outweighs the benefits of faster regex handling.

[jw013]

@Miati Why would you think this extra read x; echo $x is any better for performance? The syntax does not look any shorter or cleaner. x=${x// /_}; x=${x^^} is a much more concise way to do the same thing as {read x; echo ${x.... As far as performance goes, @jimmij has pointed out that tr / sed would be faster than bash, fork count being equal. Using a pipe always results in an extra process so the argument of saving a fork no longer applies. Thus, if are using pipes, just use sed / tr etc. If you can do it in bash, do so and skip this read x; echo $x nonsense.

[jw013]

Considering our recent conversation about how tr / sed are faster than bash at string processing, and considering how you are using pipes to pass strings via standard I/O, I see literally zero point to doing those operations in bash as opposed to tr / sed. Why would one ever | { read x; echo $x... } as opposed to a | sed that does the same thing?

[jimmij]

@jw013 frankly speaking I see next to no point. It is just an example to forcefully engage pipes to the problem, because OP explicitly asked for them and didn't want to use external programs (both echo and read are bash built-ins, so in principle a little bit faster). As I already wrote in the answer progressive parameter manipulations which OP has in the question is the best one can get in my opinion for this task in bash. Anyway the problem is rather academic.