I usually do this by hand. Good to see a tool for it :-)
Feature request, which I would love to have in all my automation scripts:
Replace short flags with the long switches. Short flags are great when typing in a terminal but I don't want to figure out 2 years from now what the
obscurecommand -v -f -n
I try to use long options in my script, therefore (especially in a team, where not everyone is familiar with every single command)
It would be a great rule for shellcheck, by the way.
Line 6:
curl -fsSL "${url}"
^-- SC8537 (warning): use long options instead (`--fail`, `--silent`, `--show-error`, `--location`).I would want it opt-in, because I use shellcheck on scripts that will be run on busybox or *BSD where there aren't long options
Of course.
Oh, I didn't realize shellcheck already had optional checks (see `shellcheck --list-optional` for a list), so that was not obvious to me initially. Then yes, that'd be a good thing to have available.
When I saw "deminifier for shell commands" in title I had exactly the same in mind.
Cool! My personal preference is Knuth-style line-breaks on binary operators and pipes, which means breaking before the operator/pipe symbol.
foo -a -b \
| bar -c -d -e \
| baz -e -f
foo -a -b | \
bar -c -d -e | \
baz -e -f
If you end with a pipe, you don't need the backslash before the newline. It's implicit.
https://unix.stackexchange.com/questions/253518/where-are-ba...
When writing bash, if I have a command with many switches, I use an array to avoid backslashes and make the intent clearer:
curl_cmd=(
curl
--silent
--fail
--output "$output_file"
"$url"
)
"${curl_cmd[@]}"
`shfmt` formats multi-line pipes into:
foo -a -b |
bar -c -d -e |
baz -e -f
\ linebreaks are not something I love,and a while ago I started using chained blocks..
These are usually a step between "overely complicated one liner" and structured script, and often get refactored to functions etc if the script evolves that far. But lots don't, and if I just want something readable, that also lends itself to comments etc, this works for me.
{ foo -a -b }|{ bar -c -d -e }|{ baz -e -f }
But I suspect it's not everyone's cup of tea!
Ha.. my linebreaks got removed!
Next level:
foo -a -b \
| bar -c -d -e \
| baz -e -f \
&& echo "DONE." # && /bin/true
A pattern I typically do
foo && \
bar && \
baz && \
:
You don't need the backslashes in that case. As with lines ending in pipes and a few other places, the line continuation is implicit after the &&:
https://unix.stackexchange.com/questions/253518/where-are-ba...
Huh, neat. So I picked that habit up from writing Dockerfiles, which does let you do
RUN foo && \
bar && \
:
RUN foo &&
bar &&
:
RUN <<EOF
foo
bar
EOF
Clever! ...almost TOO clever... ;-)
That's a great technique, but the `:` as no-op is tough to expect bash-normies to understand (and a tough "operator" to search for). Thanks for sharing, it'll definitely stay in my back pocket!
I do it this way but I indent the rest of the pipeline (like one typically would in a functional language with a pipeline operator or thread macro, or in an OOP language with method chaining via `.`):
foo --long-something \
| bar --long-other
foo --long-flag \
| bar \
--long-other-flag \
--long-option a \
--long-option b \
| baz \
--long-another-flag \
--long-flag-again \
> /path/to/some/file
foo \
| bar \
| baz
and echo done
test -n "$SOMETHING"
or set -x SOMETHING some-default
command $SOMETHING
One of my favorite patterns for readability is to use indented, long-form pipelines like this to set variables. They work fine inside subshells, but for uniformity and clarity I prefer to do
shopt -s lastpipe
foo \
| bar \
| baz \
| read SOMEVAR
If you do this, you end up with a script is mostly 'flat' (having very little nested control flow, largely avoiding loops), has very few variable assignments, and predictably locates the variable assignments it does have at the ends of pipelines. Each pipeline is a singular train of thought requiring you to consider context and state only at the very beginning and very end, and you can typically likewise think of all the intermediate steps/commands in functional terms.
I tend to write all of my shell scripts this way, including the ones I write interactively at my prompt. One really cool thing about shell languages is that unlike in 'real' programming languages, loops are actually composable! So you can freely mix ad-hoc/imperative and pipeline-centric styles like this (example is Fish):
find -name whatever -exec basename '{}' \;
| while read -l data
set -l temp (some-series $data)
set -l another (some-command $temp)
blahdiblah --something $temp --other $another
end \
| bar \
| baz \
> some-list.txt
I tend to follow my guideline suggested earlier and do this only close to the very beginning or very end of a pipeline if that loop actually modifies the filesystem or non-local variables, but it's really nice to have that flexibility imo. (It's especially handy if you want to embed some testing or conditional logic into the pipeline to filter results in a complex way.)
Shell script authors like yourself make me very happy. The pipe-to-read is a fun idea, I’ll use it.
One stanza I have at the beginning of every script:
[[ -n “${TRACE:-}” ]] && set -o xtrace
This was typed from memory on mobile so if the above is bugged, my bad :)
> Shell script authors like yourself make me very happy.
Shell script is really good for some things, so we're gonna end up writing it. And if we write it, it might as well be legible, right? Shell scripts deserve the same care that other programs do.
> This lets you trace any script just by setting the environment variable. And it’s nounset-safe.
Nice! I think I'll start adding the same.
> This was typed from memory on mobile so if the above is bugged, my bad :)
I think it's fine aside from the smart quote characters your phone inserted instead of plain vertical double quotes around the parameter expansion you use to make the line nounset-friendly!
Same w/ the "pipe to read" (although it doesn't seem to work right with OSX's bash-3.2). I found this gizmo somewhere and it's worth sharing as well...
# MAGIC DEBUGGING LINE
#trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND? ")' DEBUG
I need to write up some thoughts on bash being effectively a lisp if you stare at it the right way.
> Same w/ the "pipe to read" (although it doesn't seem to work right with OSX's bash-3.2). I found this gizmo somewhere and it's worth sharing as well...
Yes, you need a recent Bash for this (and lots of other nice things like 'associative arrays' (maps)). To free myself up to write more concise and legible scripts, I lean into bashisms and make use of external programs without restriction, regardless of what's shipped by default on any system. To recover portability where I need it, I use Nix to fix my Bash interpreter versions as well as the versions of external tools.
When my scripts are 'packaged' by Nix, their bodies are written as multiline strings in Nix. In that case, Nix takes care of setting the shebang to a fixed version of Bash, and I interpolate external commands in like this:
"${pkgs.coreutils}/bin/realpath" . # these do
"${pkgs.coreutils}/bin/pwd" --physical # the same thing
In other contexts (e.g., cloud-init), I use bash a bit more restrictively depending on what versions I'm targeting. But I still use Nix to provide dependencies so that my scripts use the same versions of external tools regardless of distro or platform:
nix shell nixpkgs#toybox --command which which # these do
nix run nixpkgs#which -- which # the same thing
There are is a really cool tool[1] by the Nixer abathur for automagically converting naively-written scripts to ones that pin their deps via Nix, as in the first example. I'm not using it yet but I likely will if the scripts I use for our development environments at work get much bigger-- that way I can store them as normal scripts without any special escaping/templating and linters will know how to read them and all that.
Anyhow, it's totally safe to install a newer Bash on macOS, and I recommend doing it for personal interactive use and private scripts. Pkgsrc and MacPorts both have Bash 5.x, if you don't have or want Nix.
> I need to write up some thoughts on bash being effectively a lisp if you stare at it the right way.
You should! It's totally true, since external commands and shell builtins are in a prefix syntax just like Lisp function calls. Some shells accentuate this a bit, like Fish, where subshells are just parens with no dollar signs so nested subcommands look very Lisp-y, and Elvish, whose author (xiaq) has tried to lean into that syntactic convergence designing a new shell (drawing inspiration from Scheme in various places).
> I found this gizmo somewhere and it's worth sharing as well...
# MAGIC DEBUGGING LINE
#trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND? ")' DEBUG
--
I love that it forms a literal pipe line
This is really cool; for a second I thought I could use it to stop manually maintaining both the minified and full-text versions of my “shell prefix” that makes it possible to run rust source code directly as if it were a shell script [0] where I’ve accidentally diverged between the two in the past, but then I revisited it and saw that the real value was in the comments and explanations more than just placing output in variables and breaking up command pipelines across shell lines.
But the opposite might work, does anyone have a good minifier they could recommend (preferably one that does more than just whitespace mangling, eg also renames variables, chains executions, etc) that doesn’t introduce bash-isms into the resulting script?
This looks really handy! I should add this to the environment for some of my shell-centric projects at work.