:: Re: [DNG] OT: redirecting piped / p…
Kezdőlap
Delete this message
Reply to this message
Szerző: Ludovic Bellière
Dátum:  
Címzett: Florian Zieboll
CC: [DNG]
Tárgy: Re: [DNG] OT: redirecting piped / processed stderr to a file
Reading stdbuf man page, it tells us that 'dd' is unaffected by its
settings because 'dd' doesn't use streams:

        NOTE: If COMMAND adjusts the buffering of its standard streams
        ('tee' does for example) then that will override corresponding
        changes by 'stdbuf'. Also some filters (like 'dd' and 'cat' etc.)
        don't use streams for I/O, and are thus unaffected by 'stdbuf'
        settings.


Reading the pipe man page ('man 7 pipe') tells us that there are two
ends to a pipe: read and write. Each of them are affected by the buffer
settings. Most likely 'rsync' outputs is being buffered before landing
on 'tr'. Looking at how you use stdbuff, you seem to only define the
output (write end) of the 'tr' command.

Lastly, it would be unwise to disable the buffering unless it is only
done temporarily. You have nothing to gain from it except brevity on
feedback latency. Instead of disabling buffering altogether, you can opt
to use line buffers (option 'L') instead of the default 4k block buffer.
That would, presumably, give you a line by line output.


Cheers,
                 Ludovic



On Mon, 14 Mar 2022, Florian Zieboll via Dng wrote:

>
>Thank you very much, adding 'stdbuf -o0' before 'tr' indeed solved the issue - and saves my sanity :-)
>
>But I still do not understand, why this happens only with 'dd' and not with e.g. 'rsync', as it is tr's output, which I have to unbuffer...
>
>$ rsync -az --info=progress2 $SOURCE $TARGET/ | tr '\r' '\n' > /tmp/logmailer & logmailer.sh
>
>vs.
>
>$ dd if=/dev/zero of=/dev/null bs=4M count=1000 status=progress 2>&1 | tr '\r' '\n' > /tmp/logmailer & logmailer.sh