Named pipes in fish shell
Most people who have used the Unix command line should be familiar with pipes. E.g. to get the contents of a directory sorted we could pipe the output of ls in as the input of sort:
$ ls | sort
It is easy to think thought that one can only make simple chains using this principle. But in fact one can pipe the output of multiple processes into a single process. How is this possible? There is only one stdin and one stdout! But a process may read multiple files. And on unix a file does not need to represent a file on disk. It could also represent a pipe. In this case it is called a named pipe.
It is worth remember that on Unix, the file system is just a namespace. It is not the same as what is physically residing on your hard drive. By namespace I mean you a hierarchy of names, which allows you to specify a file by a file path. But the individual names in this name space could be connected to anything: one of your hard drives, a USB stick, A CDROM or a chunk of memory on your computer.
Creating named pipes
A named pipe exists in your filesystem, and looks like a regular file. Except it doesn't exist on your hard drive but in memory in the kernel.
As an example we are going to do the diff of two sorted files. So we are pipeing in the output from two commands, into diff. So first we make two named pipes q and p for diff to read from:
$ mkfifo p q
Next we sort each of the files a.txt and b.txt and send the result into our named pipes:
$ sort a.txt > p; and sort b.txt > q
This will block while waiting for someone to read from the pipes, so we need to open another terminal window to read from p and q.
$ diff p q
Performing process substitution
But it is cumbersome to do it this way each time you need to receive input from the output of multiple commands. So the fish shell has a command psub, perform process substitution.
$ echo (psub) /tmp/.psub.3492.14844
The stdout of psub is the name of a temporarily created named pipe. On stdin it will read into to temporarily created named pipe.
$ cat (sort a.txt | psub)
This will send a.txt sorted into a named pipe, say /tmp/.psub.3492.14844 and then what is inside () will be replaced with the name of the pipe so we get:
$ cat /tmp/.psub.3492.14844
Meaning the net effect is just to output a.txt sorted. That is just a convoluted way of sorting a text file. This get more useful when dealing with multiple input:
$ diff (sort a.txt|psub) (sort b.txt|psub)
Now we manage to pipe the output of the sorting of two files into diff, without making two named pipes in separate stages and deleting them afterwards.
