if you prefer to use cat just to read your pipeline from left to right, you can just start your command with file redirection. It doesn't have to be at the end of the command.
<file grep something | do something else ...
this works in bash too, but if you're using zsh, there're a couple of nice shortcuts
<file
on it's own works as
more file
and
> file
...
...
^D
allows you to put something into the file quickly without firing up the editor. Though
It’s shell specific though, just like using FOO=bar cmd prefixing to make environment changes. Better to use env(1) and keep everything rooted in Unix processes rather than shell syntax, imho.
“Subsetting” a language is a useful practice. Consider the set of language features someone has to know in order to understand your code. If you can make that set smaller (without unreasonably sacrificing functionality), you lower the barrier to entry. More people can read your code.
The typical retort I hear is “but it’s just this one feature, how hard is it to learn just one feature?” But you’re only considering your favorite feature. To the casual code reader, it isn’t just that one feature, it’s everyone else’s favorite obscure feature as well.
Using cat makes your code readable to a wider audience. Using cat has real upside with no downside. Forcing your readers to learn more shell syntax has real downside with no practical upside.
`<file` at the start might not be idiomatic, but when I see it I think it makes perfect sense. It puts the pipeline "in order", starting with redirection from the input (and presumably ending with redirection to the output, if not stdout).
I usually interpret `command <fileA >fileB` as meaning "shove fileA into this command, and shove output into fileB". The "arrows" make visual sense this way.
`<fileA command` OTOH at least visually gives off the impression that it's sending the command's output to the left (changing the prompt, perhaps?)
Better code, yeah I agree. It's way more easily understandable. At least for me though, 99% of my bash use is stuff that will never be seen by anyone else, so the simpler keystrokes one may prove useful. Maybe if I get used to it :)
I don't feel <file is obscure. I've seen shell in that style from coworkers and from open source. Your value judgement against it might just be your experience, rather than something universal.
The point is you do not know if file is a stream, an archive, too big, currently empty or just contains bad data for your current test. So using cat file is a way to standardize debugging or make development easier.
Both of those invocations would give me pause, because grep is a command that can handle file arguments on its own; in fact it can incorporate knowledge of those filenames into its output quite usefully, so I would generally avoid redirects into grep when I could use it to explicitly, directly, open the files in question.
The problem is that according to this argument `cat` should never be used for the thing it was designed for, because if you use it with more than one argument, it would give most Unix users pause, but if you saw it with only one argument they would understand it instantly.
The whole point of the "never use cat" Unix advice is a war between instrumentalism and design purism. Should things be known mostly for what they're useful for, or mostly for what they were made to do? If you understand this, then the war is not soluble but you can at least phrase a third position that will reconcile both sides. If you don't understand this, then the war is over an issue that doesn't even exist.
Hot take: `<file> | abc` and `abc | <file>` should both make sense because a file should be understood by default as a command that reads from and writes to a particular destination, and the shell should take it seriously that `abc | <file>` needs to be easily undoable if the file wasn't chmodded +x appropriately.
So `abc | <file>` would write the stdout of `abc` to the file? Meaning that if you forgot to mark your script as executable, you’ve now lost all your work? I think `abc > <file>` works perfectly fine and pipes don’t need to substitute this behavior.
That's the point is that it's up to the shell to make sure that you don't use your work if you do the wrong thing. Could be as simple as prompting you to make sure -- "Overwrite <filename>? [yn]: " -- or could be something where the old inode is still available under some magic path for a few more commands, or could be that you use special syntax for the command, like "@<path>" is the stdin/stdout command form for the path... or it could be something else entirely, really.
For an instance where there are thousands of bash scripts out there that work around a case where `abc >file` does not work, consider the times when you want to write to a file which you don't have write permissions to, with sudo. my favorite way to do this is with `tee` but I have also seen others: but `>file` is not one of them! Because it's emphatically not part of the algebra of the rest of the shell; the rest of the shell is written in pipes, this one command says "I'm going to make your pipeline terminate, this step will be un-pipeable."
For another example where the algebra doesn't consistently handle the whole system, consider that in a normal language you would consider writing a function which returns a value and also prints debug logs to the process's stdout. Bash can't do this on its own, you have to figure out which /dev/tty is attached to the process and then write your debug log to that TTY, and then your script probably fails in interesting ways when it itself is redirected to a file and there is no TTY that it is attached to anymore...
CAT(1)
concatenate [one or more] files and print on the standard output
A list of length one is still a list. It's best to write code that can handle lists of any size, not to write special cases that handle a fixed number of primitives.
You can swap `cat` for `zcat`, for example, and things still work in just the same way. This would be very awkward to do with input redirection.
Maybe using `cat` like this appeals to people with more of a functional programming exposure.
If we were going to completely change the shell behaviour I think `file > grep` makes more sense than `file | grep`.
People who get up in arms about "useless" uses of cat need to chill out. I read my pipelines left to right. Using a '<' redirection to pull the data in after the command means I have to swap my reading direction back and forth, and why bother with that? I have better things to do with my brain energy.
As others have mentioned, you can put `<` anywhere in the command, including at the very beginning. But, I agree that getting up in arms about this is really dumb.
My main use of cat is for clipboard pasting (if I can't leverage pbpaste or something more appropriate). If I want to get something into a file or get into a pipe as stdin `cat > file` or `cat | cmd`, paste, then end with cmd-D to send an EOF and terminate the cat. For files it circumvents "helpful" formatting you would get pasting into an editor.
> Some people insist that we should never use a cat
Rather, people argue against useless use of cat, which is different. Personally, I'd never argue against it on the command-line even though some do. It's once you commit your pipeline to a script where you may as well remove it.
There are also suitable uses of cat that I don't think anyone would argue against. Besides the obvious use of outputting one or more files, it's also often used with here docs.
The underlying topic for this is that, most shells treat files/redirections as part of the individual commands and not part of the pipelines. Files should be sometimes treated as commands instead, they plug into the fds of a process the same way as commands do.
`cat x | head` attaches `cat x`'s stdout to `head`'s stdin,
`< x head` just attaches `x`'s fd to stdin.
In this way, treating a file like a "light-weight" command (light-weight because no backing process) makes more sense, `open x | head` perhaps
Though I will note that, in a way, bash supports this, as it has a modules system, and one of the pre-existing modules in the source tree is a `cat` replacement, through my lack of understanding of C prevents me from working out if its optimal
From the file you linked, the built-in just copies the input files one by one, sequentially, into stdin. All done in chunks of 4096 bytes, a common page size.
Is it as optimal as just attaching the file descriptor? No, because you will have to pipe the contents.
Is it optimal (or at least optimal enough)? Considering how pipes work, yes.
The logic of the author's argument seems rather convoluted. To me, cat or not is a simple decision: does the first useful command in the pipeline accept input from a file, or only stdin?
You can still just begin with <somefile instead of cat somefile |. Does it really matter? No, but I don’t think pointing out useful and concise built-ins is necessarily asinine.
Yeah, there are still some edge cases where cat is useful, but more often than not I just begin the redirection. I reckon some don't realize you can put it at the beginning, which does sometimes feel more ergonomic in a pipeline.
It's not that it feels awkward to write the redirection first. It's that it feels awkward to write `< file cmd`, with no operator between the filename and the command, and the redirection operator not pointing at the command.
IO redirection doesn't construct a pipe, but piping does.
# Open foo.txt and use it as fd 0 in jq
<foo.txt jq '[1,2,3]'
# Create a pipe(). Use the write end as cat's fd 1 and the read end as jq's fd 0
cat foo.txt | jq '[1,2,3]'
Best thing would be a directed graph where each node is a process or a file. A process has at most one edge going into it (stdin) and at most two coming out (stdout and stderr). A file has exactly one edge touching it.
No, in ordinary usage it doesn't matter, and it's not like I would go out of my way to change someone else's script code.
But if teaching or demonstrating shell features, I'd say that it's important to know that redirection is the most efficient, explicit, and available method.
Indeed, a student could learn "cat file | ..." as a standard idiom and never use an initial redirect, but what happens when they come across one "in the wild"? They should know its proper interpretation, because it can otherwise be a bit jarring to see and difficult to interpret.
It’s a hump curve with virtue signalling in the middle, like a thousand other things in life.
Noob+naive: use cat because don’t know better.
Experienced+smart ass: use shell direction to distinguish oneself from noobs.
Older+wiser: use cat again because it flows left to right, isn’t bash specific like “<file foo”, doesnt special case examples with only one file as input, and allows something other than a file as input or lifting the body of the pipeline into a function without changing the text…
…but mostly because it doesn’t make one come across as a nit picking asshole to the expensive noobs you’ve just spent months trying to hire.
If by that you mean religiously avoiding cat at all cost, no. If by that you mean thinking about what you are actually doing on your commandline I say I do. Which makes me realize that yes, cat is more often than not redundant unless you are actually outputting the concatenation of multiple files.
90% of the time, I care more about someone like that understanding my script than the tiny performance impact of an extra process or 'misusing' a function.
cat concatenates files together. I often use it for that. Who cares if "files" is just one file? Using cat regardless keeps things uniform. I like it. People who get funny about it are just trying to be clever. Yes I know there are other ways to do it. But you understood what cat did, otherwise you wouldn't have commented on it.
I usually start with cat -- later the head doesn't just get replace the cat may too, e.g. replacing it with some command whose output was only temporarily stored in a file for development purposes.
The cat command can be omitted there, as tee reads from standard input by default, even if stdin points to a terminal. I was going to comment an actually useful (and unavoidable in bash) use of cat and ssh, which is to do do nothing with standard input and redirect it to a file:
<file ssh 'cat >file'
And you could just use scp, but I've found clients without scp and servers with the SFTP subsystem disabled.
Speaking of redirections... Here's another fut bit of lesser-known shell:
I.e. You can bind redirections to function definitions that end up applying at call time.Using `cat file.txt | ...` has always felt more natural to me, so I still use it.
If I need to build a long command I've been using the excellent `up` tool to do it, e.g. `cat file.txt | up`
https://github.com/akavel/up
Wow that looks amazing, thanks for telling me about this!
Indeed. A good addition to fzf and bat
if you prefer to use cat just to read your pipeline from left to right, you can just start your command with file redirection. It doesn't have to be at the end of the command.
this works in bash too, but if you're using zsh, there're a couple of nice shortcuts on it's own works as and allows you to put something into the file quickly without firing up the editor. Though will give you nicer line editing experience.more zsh redirection tricks here: https://zsh.sourceforge.io/Doc/Release/Redirection.html#Mult...
Or you could just use cat
Coward, use `tail -c +0` like the rest of us useless command cowboys
It’s shell specific though, just like using FOO=bar cmd prefixing to make environment changes. Better to use env(1) and keep everything rooted in Unix processes rather than shell syntax, imho.
File redirection is specified in POSIX:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
So is FOO=bar cmd.
(I guess it is shell-specific if you include csh in the game, but then what isn't...)
Both of the features that you mention are specified in POSIX, which means that bash, ksh, zsh, sh, ash, dash, etc. implement them.
I find myself googling "OpenGroup shell command language" pretty often to check this sort of thing.
If I saw
It would give me pause. If I saw I would understand it instantly, as would any other Unix user. Therefore the latter is better code.My coworker said the same thing. But now that you know, will it give you pause?
Shell has tricky and arcane corners that are best to avoid, but you still do better to know about them, lest they bite you (shellcheck helps).
I don't think that "I/O redirection can precede the command name" is particularly tricky or arcane.
What bothers me is that it doesn't work with loops:
The shell's grammar is quite a thing.“Subsetting” a language is a useful practice. Consider the set of language features someone has to know in order to understand your code. If you can make that set smaller (without unreasonably sacrificing functionality), you lower the barrier to entry. More people can read your code.
The typical retort I hear is “but it’s just this one feature, how hard is it to learn just one feature?” But you’re only considering your favorite feature. To the casual code reader, it isn’t just that one feature, it’s everyone else’s favorite obscure feature as well.
Using cat makes your code readable to a wider audience. Using cat has real upside with no downside. Forcing your readers to learn more shell syntax has real downside with no practical upside.
`<file` at the start might not be idiomatic, but when I see it I think it makes perfect sense. It puts the pipeline "in order", starting with redirection from the input (and presumably ending with redirection to the output, if not stdout).
I usually interpret `command <fileA >fileB` as meaning "shove fileA into this command, and shove output into fileB". The "arrows" make visual sense this way.
`<fileA command` OTOH at least visually gives off the impression that it's sending the command's output to the left (changing the prompt, perhaps?)
Better code, yeah I agree. It's way more easily understandable. At least for me though, 99% of my bash use is stuff that will never be seen by anyone else, so the simpler keystrokes one may prove useful. Maybe if I get used to it :)
I don't feel <file is obscure. I've seen shell in that style from coworkers and from open source. Your value judgement against it might just be your experience, rather than something universal.
For sure many people don't know what it does
Furthermore, a little memory lapse or typo, and, oops! >byebyefile
It's definitely more obscure than cat, and even those familiar might be confused with it at the beginning of the line
Or just use
The point is you do not know if file is a stream, an archive, too big, currently empty or just contains bad data for your current test. So using cat file is a way to standardize debugging or make development easier.
Also, you can replace cat with pv to get a progress bar. Or replace cat with socat to stream stdin from a network socket. Etc etc.
Both of those invocations would give me pause, because grep is a command that can handle file arguments on its own; in fact it can incorporate knowledge of those filenames into its output quite usefully, so I would generally avoid redirects into grep when I could use it to explicitly, directly, open the files in question.
The problem is that according to this argument `cat` should never be used for the thing it was designed for, because if you use it with more than one argument, it would give most Unix users pause, but if you saw it with only one argument they would understand it instantly.
The whole point of the "never use cat" Unix advice is a war between instrumentalism and design purism. Should things be known mostly for what they're useful for, or mostly for what they were made to do? If you understand this, then the war is not soluble but you can at least phrase a third position that will reconcile both sides. If you don't understand this, then the war is over an issue that doesn't even exist.
Hot take: `<file> | abc` and `abc | <file>` should both make sense because a file should be understood by default as a command that reads from and writes to a particular destination, and the shell should take it seriously that `abc | <file>` needs to be easily undoable if the file wasn't chmodded +x appropriately.
So `abc | <file>` would write the stdout of `abc` to the file? Meaning that if you forgot to mark your script as executable, you’ve now lost all your work? I think `abc > <file>` works perfectly fine and pipes don’t need to substitute this behavior.
That's the point is that it's up to the shell to make sure that you don't use your work if you do the wrong thing. Could be as simple as prompting you to make sure -- "Overwrite <filename>? [yn]: " -- or could be something where the old inode is still available under some magic path for a few more commands, or could be that you use special syntax for the command, like "@<path>" is the stdin/stdout command form for the path... or it could be something else entirely, really.
For an instance where there are thousands of bash scripts out there that work around a case where `abc >file` does not work, consider the times when you want to write to a file which you don't have write permissions to, with sudo. my favorite way to do this is with `tee` but I have also seen others: but `>file` is not one of them! Because it's emphatically not part of the algebra of the rest of the shell; the rest of the shell is written in pipes, this one command says "I'm going to make your pipeline terminate, this step will be un-pipeable."
For another example where the algebra doesn't consistently handle the whole system, consider that in a normal language you would consider writing a function which returns a value and also prints debug logs to the process's stdout. Bash can't do this on its own, you have to figure out which /dev/tty is attached to the process and then write your debug log to that TTY, and then your script probably fails in interesting ways when it itself is redirected to a file and there is no TTY that it is attached to anymore...
Can you explain what fou mean by <file> | abc? I have no clue what that would mean. Further what does undoable mean in a shell context?
It is the thing it was designed for:
A list of length one is still a list. It's best to write code that can handle lists of any size, not to write special cases that handle a fixed number of primitives.You can swap `cat` for `zcat`, for example, and things still work in just the same way. This would be very awkward to do with input redirection.
Maybe using `cat` like this appeals to people with more of a functional programming exposure.
If we were going to completely change the shell behaviour I think `file > grep` makes more sense than `file | grep`.
What if you want to grep a bash script file?
People who get up in arms about "useless" uses of cat need to chill out. I read my pipelines left to right. Using a '<' redirection to pull the data in after the command means I have to swap my reading direction back and forth, and why bother with that? I have better things to do with my brain energy.
As others have mentioned, you can put `<` anywhere in the command, including at the very beginning. But, I agree that getting up in arms about this is really dumb.
cat all the things. i cat | grep and im not ashamed of it! cat | less - why not.? its not going to melt my pc more or less if i cat | more | less...
You may miss some optimizations for cat | tail. And you can't --follow a cat, obviously!
It's the cat's meow!
> Because for me it'd only be a local maxima.
Nitpick: this should be „maximum“, maxima is the plural.
Note the 'local'. If the output has several humps, multiple minima/maxima are possible.
Yes, but "it" and "a" in the original sentence are singular.
My main use of cat is for clipboard pasting (if I can't leverage pbpaste or something more appropriate). If I want to get something into a file or get into a pipe as stdin `cat > file` or `cat | cmd`, paste, then end with cmd-D to send an EOF and terminate the cat. For files it circumvents "helpful" formatting you would get pasting into an editor.
if you're on os x, you can also do
and if you're using zsh, just is sufficient, without cat> Some people insist that we should never use a cat
Rather, people argue against useless use of cat, which is different. Personally, I'd never argue against it on the command-line even though some do. It's once you commit your pipeline to a script where you may as well remove it.
There are also suitable uses of cat that I don't think anyone would argue against. Besides the obvious use of outputting one or more files, it's also often used with here docs.
The underlying topic for this is that, most shells treat files/redirections as part of the individual commands and not part of the pipelines. Files should be sometimes treated as commands instead, they plug into the fds of a process the same way as commands do.
`cat x | head` attaches `cat x`'s stdout to `head`'s stdin, `< x head` just attaches `x`'s fd to stdin. In this way, treating a file like a "light-weight" command (light-weight because no backing process) makes more sense, `open x | head` perhaps
Though I will note that, in a way, bash supports this, as it has a modules system, and one of the pre-existing modules in the source tree is a `cat` replacement, through my lack of understanding of C prevents me from working out if its optimal
https://git.savannah.gnu.org/cgit/bash.git/tree/examples/loa...
From the file you linked, the built-in just copies the input files one by one, sequentially, into stdin. All done in chunks of 4096 bytes, a common page size.
Is it as optimal as just attaching the file descriptor? No, because you will have to pipe the contents. Is it optimal (or at least optimal enough)? Considering how pipes work, yes.
"A la test driven development. "
I would not call that test driven development (where are the testcases?)
Rather more like REPL driven programming.
Whether there are formal examples or not, the thing you're doing during REPL driven programming is testing. It's just rapid, iterative and informal.
TDD refers to a specific kind of testing methodology, not just any and all testing. Manual testing is not TDD.
The logic of the author's argument seems rather convoluted. To me, cat or not is a simple decision: does the first useful command in the pipeline accept input from a file, or only stdin?
https://news.ycombinator.com/item?id=39457875 another one
A better example is ‘pv -l’. Seeing cat in a heavily-used pipeline means pv is not available.
You can still just begin with <somefile instead of cat somefile |. Does it really matter? No, but I don’t think pointing out useful and concise built-ins is necessarily asinine.
Yeah, there are still some edge cases where cat is useful, but more often than not I just begin the redirection. I reckon some don't realize you can put it at the beginning, which does sometimes feel more ergonomic in a pipeline.
It's not that it feels awkward to write the redirection first. It's that it feels awkward to write `< file cmd`, with no operator between the filename and the command, and the redirection operator not pointing at the command.
A pipeline without pipes connecting is just a field of shit
IO redirection doesn't construct a pipe, but piping does.
Best thing would be a directed graph where each node is a process or a file. A process has at most one edge going into it (stdin) and at most two coming out (stdout and stderr). A file has exactly one edge touching it.Hard to do in text, though. Would look like lisp.
Can you? In bash I end up with no input at all if I start the line with '<somefile'.
Ahem, try this: <file cat
No, in ordinary usage it doesn't matter, and it's not like I would go out of my way to change someone else's script code.
But if teaching or demonstrating shell features, I'd say that it's important to know that redirection is the most efficient, explicit, and available method.
Indeed, a student could learn "cat file | ..." as a standard idiom and never use an initial redirect, but what happens when they come across one "in the wild"? They should know its proper interpretation, because it can otherwise be a bit jarring to see and difficult to interpret.
I honestly use tac more then cat
I'm not sure I have ever used tac for real. Could you give an example?
Reading a log file from most recent entry for example
Cat *log|grep error|tac for example
Tail -500 log | tac
When you want to tail a file, but you want it in descending order and you’re not sure how much of it you want:
I'll just stick with tail -f until they implement tac -f
I guess the difference is that I expect things to read top to bottom, so I'd do that as
Fair enough, if that’s how you want it to read. less on its own is certainly quite capable.
In my experience it's best to do
Why?
Moar cat!
Does anyone take the "don't use cat" stuff seriously?
It’s a hump curve with virtue signalling in the middle, like a thousand other things in life.
Noob+naive: use cat because don’t know better.
Experienced+smart ass: use shell direction to distinguish oneself from noobs.
Older+wiser: use cat again because it flows left to right, isn’t bash specific like “<file foo”, doesnt special case examples with only one file as input, and allows something other than a file as input or lifting the body of the pipeline into a function without changing the text…
…but mostly because it doesn’t make one come across as a nit picking asshole to the expensive noobs you’ve just spent months trying to hire.
“<file foo” is POSIX: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V... (see the absence of mention that redirection needs to be after the command)
Depends what you mean by seriously.
If by that you mean religiously avoiding cat at all cost, no. If by that you mean thinking about what you are actually doing on your commandline I say I do. Which makes me realize that yes, cat is more often than not redundant unless you are actually outputting the concatenation of multiple files.
I tend to agree with this sentiment but this year I came across two situations where the use of cat is actually harmful and not just useless: https://github.com/jonjohnsonjr/til/blob/main/post/readat.md...
I still instinctively start my pipelines with cat half the time, but now I have complicated feelings about it.
It's more of a symptom of someone who doesn't understand what's actually going on, i.e. cargo cult scripting.
Specifically, it indicates some combination of:
1. isn't aware of processes/pipes
2. isn't aware of cat's "primary" functionality
3. isn't aware of shell input redirection
4. isn't aware that shell input redirection can be put before the command
90% of the time, I care more about someone like that understanding my script than the tiny performance impact of an extra process or 'misusing' a function.
ShellCheck does (did): https://www.shellcheck.net/wiki/SC2002
Well, this is the internet!
We had just gotten over tabs vs spaces after barely a thousand years of noticing the other side is objectively wrong.
Something new that you're definitely doing wrong to blog and argue and bicker about; it can't be helped!
cat concatenates files together. I often use it for that. Who cares if "files" is just one file? Using cat regardless keeps things uniform. I like it. People who get funny about it are just trying to be clever. Yes I know there are other ways to do it. But you understood what cat did, otherwise you wouldn't have commented on it.
literally what cat stands for. conCATenate.
I usually start with cat -- later the head doesn't just get replace the cat may too, e.g. replacing it with some command whose output was only temporarily stored in a file for development purposes.
and a cat is also good against mice
I'm a believer in the ditt-k-pow, DTTCPW, the Dumbest Thing That Could Possibly Work.
What, exactly, are the arguments against `cat`?
The argument is that it's redundant because 'cat f | head' does the same as 'head f'
I don't mind either way but I find starting with a cat and reading LTR to be very easy to understand.
The cat command can be omitted there, as tee reads from standard input by default, even if stdin points to a terminal. I was going to comment an actually useful (and unavoidable in bash) use of cat and ssh, which is to do do nothing with standard input and redirect it to a file:
And you could just use scp, but I've found clients without scp and servers with the SFTP subsystem disabled.I want do paste from my clipboard, not copy another file.
I know, it's intentionally unrelated. But if you read my first sentence, you can do what you are interested in without using cat.
And you will be able to paste from your clipboard or write anything you want. Without cat or piping.[dead]