[ wasn't an operator or language construct but an external program similar or the same as test. ] was a final argument.
In today's world where bash is and does supplant POSIX for all intents and purposes, use [[ and (( because they're part of the interpreter and more flexible.
When I need to write shell scripts, it's because I need portability, so I write POSIX shell scripts. If I can compromise on portability anyway, I jump right past Bash to a more sane general purpose language, like Python.
Until you realize that some of the expressions in [[ are evaluated as arithmetic expression which has some surprises around array subscript and command substitution.
In short, because the following works despite quoting and will execute the echos
a=(); x='a[$(echo >&2 what; echo 0)]'; [[ 'x' -eq 0 ]]
The relevant parts from the bash manpage:
> arg1 OP arg2
> OP is one of -eq, -ne, -lt, -le, -gt, or -ge. [...] When used with the [[ command, Arg1 and Arg2 are evaluated as arithmetic expressions (see ARITHMETIC EVALUATION above).
> ARITHMETIC EVALUATION
> Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax.
> The value of a variable is evaluated as an arithmetic expression when it is referenced
> Arrays
> The subscript is treated as an arithmetic expression that must evaluate to a number.
> [[ expression ]]
> The shell performs [...] arithmetic expansion, command substitution [...] on those words.
Given that reading, that both subscripts and values of variables in arithmetic expression are treated as arithmetic expressions, and that command substitution is peformed after arithmetic expansion in `[[`, I kinda expected the following to work:
$ x='$(echo >&2 what; echo 0)'; [[ 0 -eq x ]]
bash: [[: $(echo >&2 what; echo 0): syntax error: operand expected (error token is "$(echo >&2 what; echo 0)")
I don't fully understand what's going on here, but note that arithmetic expansion is for stuff like $((6 * 7)). There's no arithmetic expansion involved in your or Calzifer's examples.
You mean this part of my comment:
>> The shell performs [...] arithmetic expansion, command substitution [...] on those words.
> [...] and that command substitution is peformed after arithmetic expansion in `[[` [...]
You're right. That doesn't apply. I was equating "arithmetic expansion" with "evaluation of arithmetic expressions", but I see now the former is specifically about `$(())`.
Thanks! Added this example to my training slides.
No, don't do that.
You will understand why when you try this with /bin/sh on Debian/Ubuntu.
This incompatibility was introduced for reasons of speed and standards compliance.
There is a time and a place to rely on advanced shell features, but it should not be the default preference.
Obsessing over POSIX purity is a pointless endeavor. It should stay in its corner, but is largely a waste of time outside of corner cases. Do not mix the two and understand the differences, but remaining locked inside a cage you make for yourself is a pointless show. The real world has bash everywhere that matters.
PS: Be good, or I'll replace you with a very small zsh script. ;) Interestingly, it possible to dynamically add additional native commands to both zsh and bash in other languages. LuaBash and bash-loadables are examples.
Advanced shell features will not run with dash, which is (almost) strict POSIX.
They also fail with busybox.
There are many subtle differences between BASH and Korn, so much will not run without rewrite on Android's mksh.
Because you have not needed portable scripts yourself does not imply that the need is not vast.
In many cases, the fact that it's far from trivial to figure out whether your script is portable or not, makes them not portable for practical intents and purposes. I would say that for most people and most use cases, shell scripts are write once and hope you never have to edit them again, and even knowing one way of comparing values may be problematic while another might not be is beyond the given pay grade.
You can of course test portability by testing, or simply writing, your script against a POSIX-only shell interpretation. It's a lot easier to change a shebang (#!/bin/ash -> #!/bin/sh) than to rewrite the full script during a firefight.
One might even suggest "/bin/bash considered harmful".
One also may suggest "/bin/sh considered harmful". Does the rest of the world have to suffer just because Solaris can't be arsed to upgrade its tooling (much of it is very tentatively POSIX-compliant anyhow) so they block any new additions to the /bin/sh features in the POSIX?
The dash shell compiles to 80k on an i386, which is viable for embedded applications.
The POSIX shell was standardized in view of the original Korn shell, which could compile to 64k on Xenix 286.
The functionality was reduced in an effort to increase code maintainability.
Edit:
"A lot of effort was made to keep ksh88 small. In fact the size you report on Solaris is without stripping the symbol table. The size that I am getting for ksh88i on Solaris is 160K and the size on NetBSD on intel is 135K.
"ksh88 was able to compile on machines that only allowed 64K text. There were many compromises to this approach. I gave up on size minimization with ksh93."
<https://news.ycombinator.com/item?id=41682985>
It's hardly just Solaris.
This can be said of most programming, in Java, C++, golang, Pascal and Ruby I have been on projects that could not compile. The dependencies are so out dated a complete rewrite is easier. I do not find bash worse or better. You just do alot more with less lines in it.
If you use arrays, you cannot use dash.
The correct solution here is to pick one of the following options:
1) Write a bash script and put /bin/bash in your shebang
2) Write a POSIX shell script and integrate checkbashisms into your editor.
Neither is more correct than the other.
I believe many Linuxes still symlink /bin/sh -> /bin/bash, so the shebang itself won't save you. Debian's assignment of /bin/dash as the default /bin/sh is relatively recent (at least for those who've been running Debian since the 1990s).
I virtually exclusively write Bash scripts for my own use. I've been bit by bashisms however, and at the very least don't casually dismiss the concerns which are voiced against doing this. I'm also daily using systems which don't have a full Bash, and which really don't have the resources to run it (mostly storage, though other factors may be involved).
> Write a POSIX shell script and integrate checkbashisms into your editor.
Suggestion: Use shellcheck. It will complain about bashisms if your script says #!/bin/sh and will also catch loads of other problems.
3) #!/bin/dash
#!/usr/bin/env dash
I mean, if we're targeting portability...
Classic IAGNI/YAGNI conflation.
I've got sympathies with both sides of this argument.
Bashisms are powerful, flexible, and occasionally performant.
They're also non-portable, and you're apt to discover this whilst in an emergency situation with limited resources, most of all time. Having to edit shell scripts into POSIX conformance on a smartphone SSH session is No Bueno.
There are an awful lot of systems which have either no, or old, Bash. Most prevalent of the latter is MacOS / OSX, which is now nagging me on each shell initialisation that the current supported standard is not bash but zsh. Scripts linked to /bin/bash will of course run, but the last update was version 3.2.57(1) if my system reports correctly. The most recent GNU Bash release is 5.3 according to <https://ftp.gnu.org/gnu/bash/>. And that's hardly the only old Bash floating around.
On many systems, including many routers, DSL/cable modems, and other embedded devices, shell is some alternate variant (dash is /bin/sh on Debian, ash on RHEL's initrd IIRC, which last I fought with it was a script-only interpreter, not capable of running interactive commands, fucking annoying as hell). And there are still legacy Unix systems running Bourne, ksh, or other shells. Not to mention the phenomenal number of systems for which busybox provides much if not all of the userland environment, including /bin/sh. And is not bashism-capable.
Ignoring POSIX works fantastically until it doesn't. Caveat scriptor.
If you want the most powerful shell, ksh93 has the most features without doubt.
Since David Korn retired, it has been adrift with few updates.
The dash shell can be compiled with libedit to implement "set -o vi" (which is also POSIX), and is pleasant to use this way.
I don't know why someone would subject themselves to bash if the common argument in favour of it is a lie.
People say that bash is already installed on the system. Then people like you point out that it isn't.
Bash is installed on my own primary system(s). And generally a quite recent version. When I'm coding my own utility scripts (or more likely: writing one-liners which may get reused through shell history, possibly eventually being memorialised as scripts), I'll make heavy use of bashisms for the reasons I've noted elsewhere: powerful, convenient, often performant.
But if I'm inflicting those or other scripts on other systems I'm highly cognisant that not all systems have either a modern bash, or bash at all, and will either write POSIX-conformant scripts there, or remove bashisms from my own scripts.
That is, availing myself to benefits whilst recognising limits to my approach.
A key benefit of scripting in the shell language used most frequently is that one is highly familiar with it. Scripting-only languages tend to be less familiar. They're also less available (sed and awk being the principle exceptions, which are also mandated under POSIX, and for which GNU variants also have idiosyncrasies (I run into this on MacOS all the time, and tend to make use of GNU sed/awk there).
Perl, Ruby, Python, Lua, node.js, Tcl, etc., have more capabilities, but are also unavailable on a large number of systems. (Often old or small systems. to be sure, but numerous all the same.)
It's fine to write a bash script, if you know you'll have bash. It's not great to write a bash script with #!/bin/sh It's also not great to write a bash script when you have to run without bash.
>You will understand why when you try this with /bin/sh on Debian/Ubuntu.
Okay so just ... don't do that? You're meant to put /bin/bash in the shebang ... because it's a bash script. I don't understand this self-inflicted problem; obviously if you run a bash script with sh, it won't do what you want! They're different things! You may as well complain it won't run under /bin/python.
If you use #!/bin/sh then you must understand the POSIX.2 shell standard.
If you use arrays, coprocesses, or advanced conditional [[ then your script will fail.
The rules for #!/bin/sh as POSIX.2 are here:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
Will this fail on rhel? No, as bash implements #!/bin/sh. Will it fail on Ubuntu? Absolutely.
>If you use #!/bin/sh ...
And I'm saying just don't do that! If you use #!/bin/sh, it won't work, so don't use #!/bin/sh! You're writing Bash, so use #!/bin/bash! Because it's Bash. Not sh.
Bash and POSIX sh are different languages. If you write a script in language A, but try to run it as language B, it obviously won't work! And it's your own stupid fault for trying. Why is this such a difficult concept?
Literally I do not understand, why people write something in Bash then put the wrong shebang line at the top, on purpose. And then it doesn't work right, and then they act like this is a problem with Bash. Instead of their own fault, for trying to execute it with the wrong interpreter. Like .. just stop punching yourself in the face on purpose! Stop being stupid!
Bash developed before POSIX.2, so the old behavior of bash that is incompatible is suppressed when it runs as #!/bin/sh or when POSIX mode is otherwise set.
My particular bugbear is "alias p=printf" which works in scripts when bash is /bin/sh but fails when it is /bin/bash.
This is because bash historically did not honor alias in scripts. The solution is to set POSIX mode.
So, you see, there are two bash shells, and you likely do not know know the pre-POSIX shell as well as you might think.
Just writing for POSIX.2 is more straightforward.
Just never do #!/bin/sh then
When I benchmarked it, I saw no difference between [ and [[ when using bash, so I assume [ is handled (now) by the shell as well.
The syntax inside [[ is different though.
The dash shell is supposedly four times faster than bash, and dramatically reduced Ubuntu's boot time.
The [[ test syntax is not implemented in dash. It is listed as a future reserved word in the POSIX shell standard, but is not (yet?) specified.
(Google: POSIX shell)
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
If you pull the HTML file off the above link, you will get a directory listing of all of the POSIX.2 utilities (tar, sed, awk, etc.). The options that you find in the specifications are maximally portable.
>However, the value was mostly gone by the mid-to-late 1990s, and the few remaining issues were cleaned up before 2010 — shockingly late, but still over a decade ago.
This is an argument for it: You never know when you'll be on a legacy system. "A decade" isn't that long.
Previously:
2023: https://news.ycombinator.com/item?id=35019232 (161 comments)
2021: https://news.ycombinator.com/item?id=26776956 (227 comments)
I feel like something's missing here. I recall having to resort to the x-hack not that many years ago to acommodate for compatibility with a system shell somewhere. Beats me if I recall which and where but I'm pretty sure it had to do with empty strings.
Of course, I am now wondering why people are using /bin/[ in a shell script in 2024?
Because it makes a lot of sense in environments where sysadmins like to know what is going on.
Using /bin keeps a system in sync with the platform distribution. Keeping things tied to /bin should work between upgrade cycles.
But are there even "sysadmins" anymore?
Somewhere I would like to believe there are. I certainly identify as sysadmin.
Several long threads in the past, too.
Nobody used it with the quotes. The point was to get rid of the quotes.
[ x$var = xval ]
It does nothing to help with quoting, if $var is broken then so is x$var. About half the usages I've found (and this is some pretty old code) quote the variables correctly.
The assumption for this use was var was a valid single word but could conflict with switches to [ / test or it could be the empty string. If var was already guaranteed to be [^[:space:][:punct:][:cntrl:]]+, then x would also have been unnecessary too.
If we have set var='hello world' then the syntax breaks.
This is discussed in the article.
Not at all.
And even our autotools are still full of x-hacks without double quotes. He completely missed the point of the x-hack. Which obviously fails with spaces.