When to use Bash and when to use Perl/Python/Ruby?

Question

We are doing all our scripting with Bash so far, but I'm starting to feel a bit silly about it. While we can of course do everything we want with Bash (it's quite powerful), I'm starting to wonder if we shouldn't use a proper scripting language (in our case most likely Ruby) instead.

How do you decide when to use Perl/Python/Ruby over Bash for a script? I don't think an init script with Ruby makes sense, but how about a slightly longer script that adds email accounts?

Solution 1

Given a problem that both can handle, you'll want to use the one you're most comfortable with. Ultimately, there are a lot of small details, and only experience can teach you to see them.

Bash is a general purpose scripting language just like Python, Ruby, Perl, but each has different strengths over the rest. Perl excells at text analysis, Python claims to be the most elegant of the bunch, Bash scripts are excellent at "piping stuff around", if you know what I mean, and Ruby... well, Ruby is a little special in a lot of ways.

However, the differences between them only really matter once you have a healthy amount of scripting experience under your belt. I suggest you pick one language and push it to it's limits before moving to the next. You can do a lot in a shell script, more than most people would admit. Any language is just as hard as you want to make it. After you've written a couple things in it, every language is "easy" to you.

Being familiar with the shell pays off quickly if you live in Linux, so maybe you want to start with that. If you find a task you that is impossible or impractical to solve in a shell script, use something else.

Also, bear in mind that learning shell scripting is very simple. The real power of it lies in other programs, like awk, sed, tr, et al.

Solution 2

TL;DR - use bash only for installing a better language (if it isn't already available), otherwise you're wasting unrecoverable, precious human time. If you can't do it on the command line by hand without mistakes, don't script with bash/shell.

It's 2015, so I'd consider the following:

1. memory overhead

   • Ruby/Python runtime memory overhead compared to bash is tiny (because of shared libraries), while one probably can't maintain a non-trivial bash script anyway (i.e. one with > 100 lines) - so memory usage is not a factor

2. startup time

   • Ruby/Python startup may be a tiny bit slower, but chances are you're not going to be running lots of full Ruby/Python processes in a tight loop 100 times per second (if you have those kinds of needs, bash/shell is too much overhead anyway and you'll probably need to drop to C/C++)

3. performance

   • almost all typical data crunching will be faster in Ruby/Python - or at least comparable (or, you need C/C++/Haskel/OCaml/whatever anyway)
   • the real performance/bottleneck in execution (or even productivity) will almost never ever be "a lack of using bash/shell" (even Ubuntu switching dash for startup shows how bash is actually the problem - and a busybox is probably the only use case, because there is nothing more than 'bash' and 'vi' to write and run code, and there's often no way to add/download or store anything else)
   • running other processes to do the job (like sed/awk/grep) is actually a magnitude slower than than calling a method on a live object in memory

4. productivity

   • it's too easy to make mistakes in Bash/shell compared to using "real" methods, parameters, variables and exceptions in Ruby/Python
   • Agile is mainstream, while Bash has no support for it (lacking in unit testing capabilities, libraries, OO, modularity, linting, introspection, logging, metaprograming; almost impossible to refactor without breaking something)
   • way too many incompatibilities with other shells, minor environment variables can completely break a script (and some important dev-ops tools like Puppet ignore shebang lines and pass on or rewrite important shell variables), while Ruby/Python have well-defined relatively smooth migration paths even for major version changes
   • learning a new language takes a fraction of the time alternatively spent debugging shell scripts due to shell-specific issues (notably - variable names, no booleans, no exceptions, etc.)
   • even startup scripts are a landmine (especially because they can fail during system startup), and given the recent security flaws with bash, you may be better off using plain C (with good libraries) - yes, C needs compiling, configuration, etc., but even a simple shell script may need a repository, then versioning, then packaging anyway.
   • whatever is available with sed/awk/grep is likely already built into Ruby/Python - without it being a dependency, or "differences" between versions of those tools across platforms (so what if it works on your setup)
5. job security
   • what's the point of securing a job you don't like? (unless you love spending all those hours hard-to-debug but trivial-to-make shell script bugs)

I find there's no reason to use Bash/Shell if you have Ruby/Python installed.

And probably getting Ruby/Python installed doesn't even need a bash script in the first place (with the exception of busybox, some system tools depend on Python/Perl being present anyway).

And every time you write a shell script, you're "practicing" doing exactly that - instead of learning something more powerful/productive.

Why do people use Bash nowadays? Because it's a terrible, hard-to-break habit. A script is rarely "finished forever" after the first few minutes - no matter how strongly people tend to think that way. Along with the "it's the last bug in this script" fallacy.

Conclusion: use bash/shell only when you're absolutely forced to (like ~/.bashrc, busybox), because it's almost never "the right tool for the job" nowadays.

Solution 3

I use bash when my primary focus is on file handling. This could include moving, copying, and renaming files, as well as using files as input for other programs or storing other program's output in files. I rarely write bash code that actually examines the contents of a file or generates the output to write to a file; I leave that to the other programs (which I may write in Perl or python) that I launch via bash.

I use Perl and python when my primary focus is on reading data from files, processing that data in some way, and writing output to files. If I find myself using (in Perl) the system command, back ticks or (in python) the subprocess module too extensively, I consider writing the script in bash. On the other hand, I sometimes start adding so much functionality to a bash script that eventually it makes more sense to rewrite it in Perl/python rather than deal with bash's limited (by comparison) support for variable scoping, functions, data structures, etc.

Solution 4

I like the criteria set out in this blog post.

• If there aren’t any arguments to pass in, it’s probably a shell script.
• If there isn’t much for control logic (besides a single loop or if/else) it’s probably a shell script.
• If the task is automation of command-line instructions it’s almost definitely a shell script.

Solution 5

I found this Perl versus Bash analysis useful...

http://blogs.perl.org/users/buddy_burden/2012/04/perl-vs-shell-scripts.html

For convenience, I'm copying a summary of that author's 1) when bash is better findings and 2) when perl is better conclusion...

When Bash is Better...

• Job Failure
• Commands on Exit
• Processing Job Output Lines
• Here Documents
• File Equivalencies
• File Timestamp Comparisons
• Tilde Expansion

When Perl is Better...

Perl still beats the crap out of bash for most applications. Reasons I might prefer Perl include (but are not limited to):

• It’s going to be faster. Mainly because I don’t actually have to start new processes for many of the things I want to do (basename and dirname being the most obvious examples, but generally cut, grep, sort, and wc can all be eliminated as well).
• String handling in bash is rudimentary at best, and the whole $IFS thing is super-clunky.
• Conditionals in shell scripts can be wonky.
• Quoting in shell scripts can be a nightmare.
• bash’s case statement leaves a lot to be desired beyond simple cases (NPI).
• Arrays in bash suck. Hashes in bash (assuming your bash is new enough to have them at all) suck even harder.
• Once processing files or command output goes beyond the simple case I listed above, Perl starts really smoking bash.
• CPAN.

So it’s not like bash is going to take over for Perl any time soon. But I still find, after all these years, that many times a simple shell script can sometimes be simpler than a simple Perl script. As I say, I welcome all attempts to convince me otherwise. But, then again, there’s nothing wrong with having a few different tools in your toolbox.

View more solutions

Comments

[Dennis]

If you can do it using either, does it really matter? The choice is only interesting if the resulting script presents any differences. For example, execution time could differ drastically (not the case an issue when email accounts).

[mkaito]

Bash's scripting subset is just as general purpose as any general purpose scripting language. Add general programs like sed into the mix, and you got yourself covered for 90% of all scripting needs you'll ever have.

[mkaito]

The shell is a glorified program launcher indeed, but it's scripting capabilities are far beyond that. I suggest you learn some real shell scripting.

[bakytn]

I can sleep on the floor but i would rather do it on a bed. The languages cant be compared they have dofferent purposes.

[SangWon Im]

I've got quite some experience with bash scripting, as with Ruby, but I sometimes still don't know what to use. Choosing based on personal preference at that moment in time seems silly. But you're right, I'll ask myself what I want to do and pick the best tool for the job.

[mkaito]

Well, if Bash is the floor, something like Haskell must be a nailboard :-)

[mkaito]

Personally, I'll use a scale of the languages I know well, in ascending order of complexity and power, and use the simplest that will fit the bill. But really, the only way to know for sure is writing your script in all languages and comparing the outcome. It's all more of a gut feeling than a certainity. After some years writing scripts, you will have come across most "types of problems", and know which language did well solving them.

[Freedom_Ben]

I used t0 also bail on Bash whenever file reading/text generation was required, but after learning the =~ operator supported by [[ ]] I have been loving me some Bash for simple file reading

[Cezary Baginski]

Interestingly, in Ruby, all (?) of the points do not apply, because Ruby has at_exit(), realpath(), expand_path(), stat(), heredocs, and exceptions (fail "error" unless system("foobar")).

[GMaster]

Just wanted to add that using small programs in the GNU coreutils package (like tr, sort, uniq) and piping them it's possible to accomplish a lot of tasks.

[ZenVentzi]

what about Perl? Is it purposefully omitted?

[JoeS]

I would argue that this is bit too much of a broad sweeping statement. If you are actually developing an application and you care about performance then use an actual programming language like perl/python/ruby (or Rust/Go/C). But if you just want something simple like running a command n times on startup then use whatever you are comfortable with. The benefit of bash is that if you don't know what environment your script is going to be ran in then pretty much every *nix has some kind of POSIX shell that will run most bash scripts.