Dumb question time!
I'm writing a bash script that outputs some progress info to stdout (stuff like "Doing this... ok", "Doing that... ok") before outputing a list of names (to stdout too).

I'd like to be able to pipe this list of names to a second script for processing, by doing a simple :
~$ script1 | script2

Unfortunately, as you may have guessed, the progress info is piped as well, and is not displayed on the screen.
Is it considered a bad practice to redirect that progress info to stderr so it is not sent into the pipe ?

Is there any "design pattern" for this kind of usecase, where you want to be able to choose what to display and what to pipe to a program that accepts input from stdin ?

stderr is for error output only. The usual solution is a CLI option to control the verbosity level of the program. Default is usually only what's strictly necessary, and increase the verbosity e.g. if a "-v" flag is given.

not sure if it's a bad practice per se. I've seen it used like that before.

honestly if it works it works!

@Fast-Nop while that's true, that's not really the solution to OPs problem. If he just adds a verbosity flag then using the script with -v will make it unpipe-able, which is kinda annoying... it would mean he would have to choose between either seeing the progress, or getting the pipe working. not a good choice to make :/

I admit I don't know what the best solution here is, but using stderr doesn't sound like the worst workaround in this case.

@Hazarth Maybe using -v to print progress on stderr as compromise?

@Fast-Nop Or a -q mode for only printing required info and printing everything by default?
Explicit > Implicit

@Avyy That's also an option. I guess it depends on which use case is the more prevalent one and hence should be served with the default.

I have no idea how but some bash tools detect whether their output is piped or not and adapt it accordingly.

For example grep does not print any color info through pipes unless you explicitly enforce it.

Other than that I'd also just add a flag to toggle the verbosity. Flags like -q and -v are very common for that, and sometimes -p/--progress.

@deadlyRants The script can check if the terminal output is going to a tty.

First of all, bash is single threaded unless explicitly written with process forks.

Second of all, you're looking for a tee not a pipe.

You can tee the output to go to stdout and your second script.

Just to clarify a few things.

I disagree with @Fast-Nop :)

stderr is meant for error and progress messages.

http://manpages.org/stderr

"Diagnostic output" to quote the manpage.

Which makes sense.

Eg. think of functions in bash:

function what_is_my_name() {
local name=${1:-'Joker'}
if [[ "${name}" == "Joker" ]]; then
printf "Oh no....\n" >&2
fi
printf "My name is %s\n" "${name}"
return 0
}

The result will be stdout - and as in bash a return should represent an exit code, you will have to use either stderr for anything else or open a separate file descriptor for it.

Thing with separate file descriptor: No one knows this without reading the shell script. Is in my opinion the worst solution, as using something additional to stderr and stdout is "unexpected".

Hence using stderr makes most sense.

Plus you can easily solve your problem with bash and process substitution.

generator > 1>(pacman) 2> >(poopoo)

As an example, pacman and poopoo would be commands.

Pacman gets stdout, poopoo stderr.

How does curl do it?

@lbfalvy

https://github.com/curl/curl/...

Will be appended to global->errors.

https://github.com/curl/curl/...

Should be stderr, though I'm too lazy to lookup how global is initialized.

How about janky stuff with \b? It used to work back in the day. Just hope you like spinning /

I think there's also something in /dev that refers to the closest parent VTT and you can dump into it but that's not very flexible and is in direct opposition of the unix principles.

I don't know if this applies to your requirements, but instead of adding a bunch of update messages you could simply enable the "debug" output with "set -x" at the beginning of your script.

Definitely not pretty but enough for most cases.

Thanks for all your answers.

I think I'll settle on using stderr since

1. I don't really consider those messages as "output", as the real expected ouput of this script is the list of names, and they are indeed more used as diagnosis messages;

2. It's not a very important script, but I wanted to use it to hone my scripting skills.

3. I don't have the motivation to add the conditions that -v, -q, -p or -d options would imply, but I'll remember them if I need them is a future script.

Also, I looked at the built-in command read :

```read -p "Enter the variable value please: " my_var```

and it seems that the prompt is written in stderr, so if bash uses stderr like this, I don't see why I couldn't.

@sariel, could you elaborate please ? I really don't see what tee would have done better than a pipe in this case.

@lbfalvy curl detects whether its stdout is a tty, and if it's not, it prints progress to stderr.

@gat0r What do you mean by "janky stuff with \b"?