{"article":{"slug":"moreutils-the-utilities-package-every-unix-linux-mac-os-developer-should-know","title":"moreutils: the utilities package every UNIX/Linux/Mac OS developer should know","subtitle":null,"summary":"Miguel Rentes walks through the moreutils package of Unix command-line tools, explaining with examples what chronic, combine, ifne, pee, sponge, ts, vidir, vipe, zrun and the other utilities do and how to install them.","content_type":"tutorial","language":"en","canonical_url":"https://rentes.github.io/unix/utilities/2015/07/27/moreutils-package/","author":{"name":"Miguel Rentes","url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Clean Coding","url":"https://rentes.github.io/","listing_slug":null,"listing":null},"topics":[{"name":"Linux","slug":"linux","url":"https://listedarticles.com/topics/linux"},{"name":"Developer Tools","slug":"developer-tools","url":"https://listedarticles.com/topics/developer-tools"},{"name":"Tutorials","slug":"tutorials","url":"https://listedarticles.com/topics/tutorials"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":2541,"reading_minutes":11,"published_at":"2015-07-27T00:00:00.000Z","added_at":"2026-10-24T02:07:26.082Z","updated_at":"2026-10-24T02:07:26.082Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/moreutils-the-utilities-package-every-unix-linux-mac-os-developer-should-know","markdown_url":"https://listedarticles.com/articles/moreutils-the-utilities-package-every-unix-linux-mac-os-developer-should-know.md","example":false,"citation":"Miguel Rentes, Clean Coding. \"moreutils: the utilities package every UNIX/Linux/Mac OS developer should know.\" 27 Jul 2015. https://rentes.github.io/unix/utilities/2015/07/27/moreutils-package/ (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://rentes.github.io/unix/utilities/2015/07/27/moreutils-package/"},"body_markdown":"# moreutils: the utilities package every UNIX/Linux/Mac OS developer should know\n\n*July 27, 2015 · Miguel Rentes*\n\nMost UNIX professionals know about the [GNU core utilities](https://en.wikipedia.org/wiki/GNU_Core_Utilities), from the “used-everyday” ls, cp, ln, rm, touch, tail, wc, etc., to the “not-so-used” shell commands such as tsort, tac, factor, or seq. Among the UNIX utilities available, there is one package that includes some awesome utilities that makes our life’s a lot easier: the [moreutils](https://packages.debian.org/unstable/utils/moreutils) package.\n\n- [Installing](#installing)\n- [The moreutils tools](#the-moreutils-tools)\n - [chronic: *runs a command quietly unless it fails*](#chronic)\n - [combine: *combine the lines in two files using boolean operations*](#combine)\n - [errno: *look up errno names and descriptions*](#errno)\n - [ifdata: *get network interface info without parsing ifconfig output*](#ifdata)\n - [ifne: *run a program if the standard input is not empty*](#ifne)\n - [isutf8: *check if a file or standard input is utf-8*](#isutf8)\n - [lckdo: *execute a program with a lock held*](#lckdo)\n - [mispipe: *pipe two commands, returning the exit status of the first*](#mispipe)\n - [parallel: *run multiple jobs at once*](#parallel)\n - [pee: *tee standard input to pipes*](#pee)\n - [sponge: *soak up standard input and write to a file*](#sponge)\n - [ts: *timestamp standard input*](#ts)\n - [vidir: *edit a directory in your text editor*](#vidir)\n - [vipe: *insert a text editor into a pipe*](#vipe)\n - [zrun: *automatically uncompress arguments to command*](#zrun)\n\n## Installing\n\nBefore describing each of the utilities included, let me tell you that this package is not exclusive for Linux systems. You can install it also under Mac OS and different UNIX flavours.\n\nUnder Mac OS, just use the following command:\n\n```\n$ brew install moreutils\n```\n\n**Note**: There is a conflict between the GNU parallel utility and the shipped parallel on the moreutils package. If you want to install GNU parallel instead of the moreutils’ parallel, use the following commands:\n\n```\n$ brew install moreutils --without-parallel\n$ brew install parallel\n```\n\nYou can read more about it [here](http://superuser.com/questions/545889/how-can-i-install-gnu-parallel-alongside-moreutils).\n\nUnder Linux, use your favourite package manager or the command line to retrieve this package.\nIf you are using Arch Linux all you have to do is:\n\n```\n$ sudo pacman -S moreutils\n```\n\nUnder Debian/Ubuntu/Mint a similar command is:\n\n```\n$ sudo apt-get install moreutils\n```\n\nAnd on RHEL, Fedora, CentOS:\n\n```\n$ sudo yum install moreutils\n```\n\nDon’t forget to check your Linux distribution to know how to install this package if the above commands are not for your particular Linux distribution.\n\n##The moreutils tools\n\n###chronic\nThis tool runs a command , and arranges for its standard out and standard error to only be displayed if the command fails (exits nonzero or crashes). If the command succeeds, any extraneous output will be hidden.\n\nBefore discovering this tool I would do something like this to hide a command output:\n\n```\n$ command >& /dev/null\n```\n\nbut using chronic is so much simpler, and has the benefit that I know if the command succeeded or not, which I would not know from the line above (all output is redirected to `/dev/null`).\n\nA common use for chronic is for running a cron job. Rather than trying to keep the command quiet, and having to deal with mails containing accidental output when it succeeds, and not verbose enough output when it fails, you can just run it verbosely always, and use chronic to hide the successful output.\n\n```\n0 1 * * * chronic backup # instead of backup >/dev/null 2>&1\n```\n\n### combine\n\nThis tool combines the lines in two files using boolean operations.\n\n```\n$ combine file1 and file2\n\n$ combine file1 not file2\n\n$ combine file1 or file2\n\n$ combine file1 xor file2\n```\n\nDepending on the boolean operation specified, the contents will be combined in different ways:\n\n- **and** Outputs lines that are in file1 if they are also present in file2.\n- **not** Outputs lines that are in file1 but not in file2.\n- **or** Outputs lines that are in file1 or file2.\n- **xor** Outputs lines that are in either file1 or file2, but not in both files.\n\nThe input files need not be sorted, and the lines are output in the order they occur in file1 (followed by the order they occur in file2 for the two “or” operations). This means that the operations are not commutative; “a and b” will not necessarily be the same as “b and a”. To obtain commutative behavior, use the `sort` and `uniq` commands to the result, like this:\n\n```\n$ cat a\none\nten\ntwo\nthree\n$ cat b\nten\ntwelve\none\nthirteen\n$ combine a and b\none\nten\n$ combine b and a\nten\none\n$ combine a and b | sort | uniq\none\nten\n$ combine b and a | sort | uniq\none\nten\n```\n\n### errno\n\nThis tool looks up errno macro names, errno codes, and the corresponding descriptions.\n\n```\n$ errno ENOENT\n\nENOENT 2 No such file or directory\n\n$ errno 100\n\nENETDOWN 100 Network is down\n\n$ errno -l\nEPERM 1 Operation not permitted\nENOENT 2 No such file or directory\nESRCH 3 No such process\nEINTR 4 Interrupted system call\nEIO 5 Input/output error\nENXIO 6 No such device or address\nE2BIG 7 Argument list too long\nENOEXEC 8 Exec format error\nEBADF 9 Bad file descriptor\nECHILD 10 No child processes\nEAGAIN 11 Resource temporarily unavailable\nENOMEM 12 Cannot allocate memory\nEACCES 13 Permission denied\nEFAULT 14 Bad address\nENOTBLK 15 Block device required\nEBUSY 16 Device or resource busy\nEEXIST 17 File exists\nEXDEV 18 Invalid cross-device link\nENODEV 19 No such device\nENOTDIR 20 Not a directory\n(...)\n```\n\nThis is a most valuable tool and a lot better than reading the system header files to figure out the meaning of a particular errno code. If you do a lot UNIX programming you know what I am taking about.\n\n### ifdata\n\nThis tool can be used to check for the existence of a network interface, or to get information about the interface, such as its IP address, all statistics on input and output, number of packets, bytes, errors, drops, incoming and outgoing bit rate, etc.. Unlike `ifconfig` or `ip`, `ifdata` has simple to parse output that is designed to be easily used by a shell script.\n\n```\n$ ifdata -si eth0\n1943188030 8648846 0 16803 0 0 0 0\n$ ifdata -sip eth0\n8649245\n$ ifdata -so eth0\n141940989 953001 0 0 0 0 0 0\n$ ifdata -bips eth0\n3012\n$ ifdata -bops eth0\n66\n```\n\n### ifne\n\nThis tool runs a program if the standard input is not empty. The name stands for “if not empty” and it simply runs the passed command to it, if it receives at least one byte in the `stdin`. You can also use the `-n` flag which causes `ifne` to reverse the operation, e.g., runs the command passed to it if the standard input is empty. Note that if the standard input is not empty, it is passed through ifne in this case. The below command line checks for `core` files on the current directory, and if found (standard inputs is not empty), sends a mail to user root alerting for this fact.\n\n```\n$ find . -name core | ifne mail -s \"Core files found\" root\n```\n\n### isutf8\n\nThis tool checks if a file or standard input are syntactically valid UTF-8. Input is either files named on the command line, or the standard input. Information about files with invalid UTF-8 are printed to standard output, otherwise nothing is printed.\n\n### lckdo\n\n**Note**: Now that package `util-linux` contains a similar command named `flock`, `lckdo` is deprecated, and will be removed from some future version of moreutils.\n\nThe `flock` utility runs a program with a lock held, in order to prevent multiple processes from running in parallel. It locks from within shell scripts or from the command line. It locks a specified file or directory, which is created (assuming appropriate permissions) if it does not already exist. By default, if the lock cannot be immediately acquired, `flock` waits until the lock is available. It can also lock an open file by its file descriptor number. Use it just like `nice` or `nohup`.\n\n```\n(shell1) $ flock /tmp -c cat\n(shell2) $ flock -w .007 /tmp -c echo; /bin/echo $?\n```\n\nThe first command sets an exclusive lock to directory /tmp and the second command will fail.\n\n```\n(shell1) $ flock -s /tmp -c cat\n(shell2) $ flock -s -w .007 /tmp -c echo; /bin/echo $?\n```\n\nThe first command sets a shared lock to directory /tmp and the second command will not fail. Notice that attempting to get exclusive lock with the second command would fail.\n\n```\n$ flock -x local-lock-file echo 'a b c'\n```\n\nThis command grabs the exclusive lock `local-lock-file` before running `echo` with ‘a b c’.\n\n### mispipe\n\nThis command pipes two commands together like the shell does, but unlike piping in the shell, which returns the exit status of the last command; when using mispipe, the exit status of the first command is returned.\n\n**Note**: Modern shells, like `bash` or `zsh`, offer a pipefail option, although that option does not behave the same like mispipe because it makes a failure of **any** command in the pipeline to be returned, not just the exit status of the first. This can be a better alternative to mispipe which only returns the exit status of the first command passed to it.\n\n### parallel\n\nThis tool runs the specified command, passing it a single one of the specified arguments. This is repeated for each argument. Jobs may be run in parallel. The default is to run one job per CPU.\n\n```\n$ parallel sh -c \"echo hi; sleep 2; echo bye\" -- 1 2 3\n```\n\nThe above command runs three subshells that each print a message, delay, and print another message. If your system has multiple CPUs, parallel will run some of the jobs in parallel, which should be clear from the order the messages are output.\n\n```\n$ parallel -j 3 ufraw -o processed -- *.NEF\n```\n\nThe above command runs three ufraw processes at the same time until all of the NEF files have been processed.\n\n```\n$ parallel -j 3 -- ls df \"echo hi\"\n```\n\nThe above command runs three independent commands (`ls`, `df` and the `echo`) in parallel.\n\n**Note**: check the `xargs` command to also run multiple processes at a time, specifically with the `-n` and `-P` flags.\n\n### pee\n\nThis command is similar to how `tee` uses the standard input but for pipes, hence the name. It allows redirecting output to multiple commands at once. Each command is run and fed a copy of the standard input. The output of all commands is sent to `stdout`. While this is similar to `tee`, a copy of the input is not sent to `stdout`, like `tee` does. If that is desired, use `pee cat ...` instead.\n\n```\n$ cat file\n5\n4\n3\n2\n1\n$ cat file | pee 'sort -u > sorted' 'sort -R > unsorted'\n$ cat sorted\n1\n2\n3\n4\n5\n$ cat unsorted\n2\n1\n4\n3\n5\n```\n\nIn this example, `pee` receives two commands: one for sorting the contents of the file, and another for randomly sorting the same file contents, outputting to two different files named sorted and unsorted, respectively.\n\n### sponge\n\nThis tool reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This means that it only writes to the file once the input has been fully read. This allows constructing pipelines that read from and write to the same file, without needing to redirect the output to temporary files.\n\nInstead of using these commands to sort a file contents:\n\n```\n$ sort filename | uniq > temp\n$ mv temp filename\n```\n\nJust use:\n\n```\n$ sort filename | uniq | sponge filename\n```\n\nNo need to redirecting anymore.\n\n### ts\n\nThis tool adds a timestamp to the beginning of each line of input. This is an awesome tool that I like to use specially on a chain of commands where I can measure how long a given command is taking to complete before passing the control to the next command. It is also a great way of logging the time stamps of a programs’ output.\n\n```\n$ cat script.sh\necho \"The first line\"\nsleep 5s\necho \"The last line\"\n$ chmod a+x script.sh\n$ ./script.sh\nThe first line\nThe last line\n$ ./script.sh | ts\nJul 27 15:50:42 The first line\nJul 27 15:50:47 The last line\n```\n\nOne possible use of the `ts` command is to assert whenever a given machine is up and running. Restart a machine and then use `ping` with the `ts` to know exactly when that machine came up to life!\n\n```\n$ ping | ts\nJul 27 15:53:15 PING EE.FF.GG.HH (EE.FF.GG.HH) 56(84) bytes of data.\nJul 27 15:53:15 From AA.BB.CC.DD icmp_seq=3 Destination Net Unreachable\nJul 27 15:53:18 From AA.BB.CC.DD icmp_seq=7 Destination Net Unreachable\n(...)\nJul 27 15:54:08 64 bytes from EE.FF.GG.HH: icmp_seq=57 ttl=127 time=0.673 ms\n```\n\nThe optional format parameter controls how the timestamp is formatted, as used by the C function `strftime`. The default format is `%b %d %H:%M:%S`. In addition to the regular `strftime` conversion specifications, `%.S` and `%.s` are like `%S` and `%s`, but provide sub-second resolution (i.e., “30.00001” and “1301682593.00001”).\n\n### vidir\n\nThis tool allows editing of the contents of a directory in a text editor (check the `$EDITOR` environment variable, because it will be used for the text editor of your choice). If no directory is specified, the current directory is edited. When editing a directory, each item in the directory will appear on its own numbered line. These numbers are how vidir keeps track of what items are changed. Delete lines to remove files from the directory, or edit filenames to rename files. You can also switch pairs of numbers to swap filenames. Note that if “-“ is specified as the directory to edit, it reads a list of filenames from stdin and displays those for editing. Alternatively, a list of files can be specified on the command line.\n\n### vipe\n\nThis tool allows you to run your editor in the middle of a unix pipeline and edit the data that is being piped between programs (once again, check the `$EDITOR` environment variable for your text editor of choice).\n\n```\n$ fortune | vipe | cowsay\n _________________________________________\n/ \"I am your density.\" \\\n| |\n| -- George McFly in \"Back to the Future\" |\n| - This fortune cookie was just edited |\n\\ by Miguel Rentes (in the future!) /\n -----------------------------------------\n \\ ^__^\n \\ (oo)\\_______\n (__)\\ )\\/\\\n ||----w |\n || ||\n```\n\nOn the above command I edited the fortune cookie before passing it to the cowsay command. This is a command that offers you great flexibility and power.\n\n### zrun\n\nThis tool automatically decompresses arguments to command. Prefixing a shell command with `zrun` causes any compressed files that are arguments of the command to be transparently uncompressed to temp files (not pipes) and the uncompressed files fed to the command. This is a quick way to run a command that does not itself support compressed files, without manually uncompressing the files. For example, if you want to `diff` the contents of two archives without having to uncompress them first, use the following command:\n\n```\n$ zrun diff archive1.gz archive2.gz\n```\n\nThis command will uncompress automatically the two .gz files and pass the contents to the diff command. Very cool.\n\nThe following compression types are supported: `gz bz2 Z xz lzma lzo`. If zrun is linked to some name beginning with z, like zprog, and the link is executed, this is equivalent to executing `zrun prog`.\n\nThat’s it! Please don’t forget to read the manpages for greater detail on the above commands. I hope you liked to learn about this fantastic tool package and that it motivated you to use it on your daily UNIX programming tasks! Until next time, have an awesome coding fun!\n","body_html":"

moreutils: the utilities package every UNIX/Linux/Mac OS developer should know

\n

July 27, 2015 · Miguel Rentes

\n

Most UNIX professionals know about the GNU core utilities, from the “used-everyday” ls, cp, ln, rm, touch, tail, wc, etc., to the “not-so-used” shell commands such as tsort, tac, factor, or seq. Among the UNIX utilities available, there is one package that includes some awesome utilities that makes our life’s a lot easier: the moreutils package.

\n\n

Installing

\n

Before describing each of the utilities included, let me tell you that this package is not exclusive for Linux systems. You can install it also under Mac OS and different UNIX flavours.

\n

Under Mac OS, just use the following command:

\n
$ brew install moreutils
\n

Note: There is a conflict between the GNU parallel utility and the shipped parallel on the moreutils package. If you want to install GNU parallel instead of the moreutils’ parallel, use the following commands:

\n
$ brew install moreutils --without-parallel\n$ brew install parallel
\n

You can read more about it here.

\n

Under Linux, use your favourite package manager or the command line to retrieve this package.\nIf you are using Arch Linux all you have to do is:

\n
$ sudo pacman -S moreutils
\n

Under Debian/Ubuntu/Mint a similar command is:

\n
$ sudo apt-get install moreutils
\n

And on RHEL, Fedora, CentOS:

\n
$ sudo yum install moreutils
\n

Don’t forget to check your Linux distribution to know how to install this package if the above commands are not for your particular Linux distribution.

\n

##The moreutils tools

\n

###chronic\nThis tool runs a command , and arranges for its standard out and standard error to only be displayed if the command fails (exits nonzero or crashes). If the command succeeds, any extraneous output will be hidden.

\n

Before discovering this tool I would do something like this to hide a command output:

\n
$ command >& /dev/null
\n

but using chronic is so much simpler, and has the benefit that I know if the command succeeded or not, which I would not know from the line above (all output is redirected to /dev/null).

\n

A common use for chronic is for running a cron job. Rather than trying to keep the command quiet, and having to deal with mails containing accidental output when it succeeds, and not verbose enough output when it fails, you can just run it verbosely always, and use chronic to hide the successful output.

\n
0 1 * * * chronic backup # instead of backup >/dev/null 2>&1
\n

combine

\n

This tool combines the lines in two files using boolean operations.

\n
$ combine file1 and file2\n\n$ combine file1 not file2\n\n$ combine file1 or file2\n\n$ combine file1 xor file2
\n

Depending on the boolean operation specified, the contents will be combined in different ways:

\n\n

The input files need not be sorted, and the lines are output in the order they occur in file1 (followed by the order they occur in file2 for the two “or” operations). This means that the operations are not commutative; “a and b” will not necessarily be the same as “b and a”. To obtain commutative behavior, use the sort and uniq commands to the result, like this:

\n
$ cat a\none\nten\ntwo\nthree\n$ cat b\nten\ntwelve\none\nthirteen\n$ combine a and b\none\nten\n$ combine b and a\nten\none\n$ combine a and b | sort | uniq\none\nten\n$ combine b and a | sort | uniq\none\nten
\n

errno

\n

This tool looks up errno macro names, errno codes, and the corresponding descriptions.

\n
$ errno ENOENT\n\nENOENT 2 No such file or directory\n\n$ errno 100\n\nENETDOWN 100 Network is down\n\n$ errno -l\nEPERM 1 Operation not permitted\nENOENT 2 No such file or directory\nESRCH 3 No such process\nEINTR 4 Interrupted system call\nEIO 5 Input/output error\nENXIO 6 No such device or address\nE2BIG 7 Argument list too long\nENOEXEC 8 Exec format error\nEBADF 9 Bad file descriptor\nECHILD 10 No child processes\nEAGAIN 11 Resource temporarily unavailable\nENOMEM 12 Cannot allocate memory\nEACCES 13 Permission denied\nEFAULT 14 Bad address\nENOTBLK 15 Block device required\nEBUSY 16 Device or resource busy\nEEXIST 17 File exists\nEXDEV 18 Invalid cross-device link\nENODEV 19 No such device\nENOTDIR 20 Not a directory\n(...)
\n

This is a most valuable tool and a lot better than reading the system header files to figure out the meaning of a particular errno code. If you do a lot UNIX programming you know what I am taking about.

\n

ifdata

\n

This tool can be used to check for the existence of a network interface, or to get information about the interface, such as its IP address, all statistics on input and output, number of packets, bytes, errors, drops, incoming and outgoing bit rate, etc.. Unlike ifconfig or ip, ifdata has simple to parse output that is designed to be easily used by a shell script.

\n
$ ifdata -si eth0\n1943188030 8648846 0 16803 0 0 0 0\n$ ifdata -sip eth0\n8649245\n$ ifdata -so eth0\n141940989 953001 0 0 0 0 0 0\n$ ifdata -bips eth0\n3012\n$ ifdata -bops eth0\n66
\n

ifne

\n

This tool runs a program if the standard input is not empty. The name stands for “if not empty” and it simply runs the passed command to it, if it receives at least one byte in the stdin. You can also use the -n flag which causes ifne to reverse the operation, e.g., runs the command passed to it if the standard input is empty. Note that if the standard input is not empty, it is passed through ifne in this case. The below command line checks for core files on the current directory, and if found (standard inputs is not empty), sends a mail to user root alerting for this fact.

\n
$ find . -name core | ifne mail -s "Core files found" root
\n

isutf8

\n

This tool checks if a file or standard input are syntactically valid UTF-8. Input is either files named on the command line, or the standard input. Information about files with invalid UTF-8 are printed to standard output, otherwise nothing is printed.

\n

lckdo

\n

Note: Now that package util-linux contains a similar command named flock, lckdo is deprecated, and will be removed from some future version of moreutils.

\n

The flock utility runs a program with a lock held, in order to prevent multiple processes from running in parallel. It locks from within shell scripts or from the command line. It locks a specified file or directory, which is created (assuming appropriate permissions) if it does not already exist. By default, if the lock cannot be immediately acquired, flock waits until the lock is available. It can also lock an open file by its file descriptor number. Use it just like nice or nohup.

\n
(shell1) $ flock /tmp -c cat\n(shell2) $ flock -w .007 /tmp -c echo; /bin/echo $?
\n

The first command sets an exclusive lock to directory /tmp and the second command will fail.

\n
(shell1) $ flock -s /tmp -c cat\n(shell2) $ flock -s -w .007 /tmp -c echo; /bin/echo $?
\n

The first command sets a shared lock to directory /tmp and the second command will not fail. Notice that attempting to get exclusive lock with the second command would fail.

\n
$ flock -x local-lock-file echo 'a b c'
\n

This command grabs the exclusive lock local-lock-file before running echo with ‘a b c’.

\n

mispipe

\n

This command pipes two commands together like the shell does, but unlike piping in the shell, which returns the exit status of the last command; when using mispipe, the exit status of the first command is returned.

\n

Note: Modern shells, like bash or zsh, offer a pipefail option, although that option does not behave the same like mispipe because it makes a failure of any command in the pipeline to be returned, not just the exit status of the first. This can be a better alternative to mispipe which only returns the exit status of the first command passed to it.

\n

parallel

\n

This tool runs the specified command, passing it a single one of the specified arguments. This is repeated for each argument. Jobs may be run in parallel. The default is to run one job per CPU.

\n
$ parallel sh -c "echo hi; sleep 2; echo bye" -- 1 2 3
\n

The above command runs three subshells that each print a message, delay, and print another message. If your system has multiple CPUs, parallel will run some of the jobs in parallel, which should be clear from the order the messages are output.

\n
$ parallel -j 3 ufraw -o processed -- *.NEF
\n

The above command runs three ufraw processes at the same time until all of the NEF files have been processed.

\n
$ parallel -j 3 -- ls df "echo hi"
\n

The above command runs three independent commands (ls, df and the echo) in parallel.

\n

Note: check the xargs command to also run multiple processes at a time, specifically with the -n and -P flags.

\n

pee

\n

This command is similar to how tee uses the standard input but for pipes, hence the name. It allows redirecting output to multiple commands at once. Each command is run and fed a copy of the standard input. The output of all commands is sent to stdout. While this is similar to tee, a copy of the input is not sent to stdout, like tee does. If that is desired, use pee cat ... instead.

\n
$ cat file\n5\n4\n3\n2\n1\n$ cat file | pee 'sort -u > sorted' 'sort -R > unsorted'\n$ cat sorted\n1\n2\n3\n4\n5\n$ cat unsorted\n2\n1\n4\n3\n5
\n

In this example, pee receives two commands: one for sorting the contents of the file, and another for randomly sorting the same file contents, outputting to two different files named sorted and unsorted, respectively.

\n

sponge

\n

This tool reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This means that it only writes to the file once the input has been fully read. This allows constructing pipelines that read from and write to the same file, without needing to redirect the output to temporary files.

\n

Instead of using these commands to sort a file contents:

\n
$ sort filename | uniq > temp\n$ mv temp filename
\n

Just use:

\n
$ sort filename | uniq | sponge filename
\n

No need to redirecting anymore.

\n

ts

\n

This tool adds a timestamp to the beginning of each line of input. This is an awesome tool that I like to use specially on a chain of commands where I can measure how long a given command is taking to complete before passing the control to the next command. It is also a great way of logging the time stamps of a programs’ output.

\n
$ cat script.sh\necho "The first line"\nsleep 5s\necho "The last line"\n$ chmod a+x script.sh\n$ ./script.sh\nThe first line\nThe last line\n$ ./script.sh | ts\nJul 27 15:50:42 The first line\nJul 27 15:50:47 The last line
\n

One possible use of the ts command is to assert whenever a given machine is up and running. Restart a machine and then use ping with the ts to know exactly when that machine came up to life!

\n
$ ping <IP address or hostname> | ts\nJul 27 15:53:15 PING EE.FF.GG.HH (EE.FF.GG.HH) 56(84) bytes of data.\nJul 27 15:53:15 From AA.BB.CC.DD icmp_seq=3 Destination Net Unreachable\nJul 27 15:53:18 From AA.BB.CC.DD icmp_seq=7 Destination Net Unreachable\n(...)\nJul 27 15:54:08 64 bytes from EE.FF.GG.HH: icmp_seq=57 ttl=127 time=0.673 ms
\n

The optional format parameter controls how the timestamp is formatted, as used by the C function strftime. The default format is %b %d %H:%M:%S. In addition to the regular strftime conversion specifications, %.S and %.s are like %S and %s, but provide sub-second resolution (i.e., “30.00001” and “1301682593.00001”).

\n

vidir

\n

This tool allows editing of the contents of a directory in a text editor (check the $EDITOR environment variable, because it will be used for the text editor of your choice). If no directory is specified, the current directory is edited. When editing a directory, each item in the directory will appear on its own numbered line. These numbers are how vidir keeps track of what items are changed. Delete lines to remove files from the directory, or edit filenames to rename files. You can also switch pairs of numbers to swap filenames. Note that if “-“ is specified as the directory to edit, it reads a list of filenames from stdin and displays those for editing. Alternatively, a list of files can be specified on the command line.

\n

vipe

\n

This tool allows you to run your editor in the middle of a unix pipeline and edit the data that is being piped between programs (once again, check the $EDITOR environment variable for your text editor of choice).

\n
$ fortune | vipe | cowsay\n _________________________________________\n/ "I am your density."                    \\\n|                                         |\n| -- George McFly in "Back to the Future" |\n| - This fortune cookie was just edited   |\n\\ by Miguel Rentes (in the future!)       /\n -----------------------------------------\n        \\   ^__^\n         \\  (oo)\\_______\n            (__)\\       )\\/\\\n                ||----w |\n                ||     ||
\n

On the above command I edited the fortune cookie before passing it to the cowsay command. This is a command that offers you great flexibility and power.

\n

zrun

\n

This tool automatically decompresses arguments to command. Prefixing a shell command with zrun causes any compressed files that are arguments of the command to be transparently uncompressed to temp files (not pipes) and the uncompressed files fed to the command. This is a quick way to run a command that does not itself support compressed files, without manually uncompressing the files. For example, if you want to diff the contents of two archives without having to uncompress them first, use the following command:

\n
$ zrun diff archive1.gz archive2.gz
\n

This command will uncompress automatically the two .gz files and pass the contents to the diff command. Very cool.

\n

The following compression types are supported: gz bz2 Z xz lzma lzo. If zrun is linked to some name beginning with z, like zprog, and the link is executed, this is equivalent to executing zrun prog.

\n

That’s it! Please don’t forget to read the manpages for greater detail on the above commands. I hope you liked to learn about this fantastic tool package and that it motivated you to use it on your daily UNIX programming tasks! Until next time, have an awesome coding fun!

","headings":[{"level":1,"text":"moreutils: the utilities package every UNIX/Linux/Mac OS developer should know","id":"moreutils-the-utilities-package-every-unix-linux-mac-os-develope"},{"level":2,"text":"Installing","id":"installing"},{"level":3,"text":"combine","id":"combine"},{"level":3,"text":"errno","id":"errno"},{"level":3,"text":"ifdata","id":"ifdata"},{"level":3,"text":"ifne","id":"ifne"},{"level":3,"text":"isutf8","id":"isutf8"},{"level":3,"text":"lckdo","id":"lckdo"},{"level":3,"text":"mispipe","id":"mispipe"},{"level":3,"text":"parallel","id":"parallel"},{"level":3,"text":"pee","id":"pee"},{"level":3,"text":"sponge","id":"sponge"},{"level":3,"text":"ts","id":"ts"},{"level":3,"text":"vidir","id":"vidir"},{"level":3,"text":"vipe","id":"vipe"},{"level":3,"text":"zrun","id":"zrun"}]}}