Pipe

管道緩衝區有多大?

  • August 15, 2021

作為評論,我很困惑為什麼makefile中的“| true”與“|| true”使用者cjm寫道:

另一個需要避免的原因| true是,如果該命令產生了足夠的輸出來填滿管道緩衝區,它將阻塞等待 true 來讀取它。

我們有什麼方法可以找出管道緩衝區的大小嗎?

管道緩衝區的容量因係統而異(甚至在同一系統上也可能不同)。我不確定是否有一種快速、簡單且跨平台的方法來查找管道的容量。

例如,Mac OS X 預設使用 16384 字節的容量,但如果對管道進行大量寫入,則可以切換到 65336 字節的容量,或者如果已經有太多核心記憶體,則會切換到單個系統頁面的容量被管道緩衝區使用(參見xnu/bsd/sys/pipe.h, 和xnu/bsd/kern/sys_pipe.c; 因為這些來自 FreeBSD,同樣的行為也可能發生在那裡)。

一個 Linux *pipe(7)*手冊頁說,自 Linux 2.6.11 以來管道容量為 65536 字節,而在此之前的單個系統頁面(例如(32 位)x86 系統上的 4096 字節)。程式碼(include/linux/pipe_fs_i.hfs/pipe.c)似乎使用了 16 個系統頁面(即,如果系統頁面為 4 KiB,則為 64 KiB),但每個管道的緩衝區可以通過管道上的fcntl進行調整(最大容量預設為 1048576字節,但可以通過/proc/sys/fs/pipe-max-size)) 更改。


這是我用來測試系統上的管道容量的一個小bash / perl組合:

#!/bin/bash
test $# -ge 1 || { echo "usage: $0 write-size [wait-time]"; exit 1; }
test $# -ge 2 || set -- "$@" 1
bytes_written=$(
{
   exec 3>&1
   {
       perl -e '
           $size = $ARGV[0];
           $block = q(a) x $size;
           $num_written = 0;
           sub report { print STDERR $num_written * $size, qq(\n); }
           report; while (defined syswrite STDOUT, $block) {
               $num_written++; report;
           }
       ' "$1" 2>&3
   } | (sleep "$2"; exec 0<&-);
} | tail -1
)
printf "write size: %10d; bytes successfully before error: %d\n" \
   "$1" "$bytes_written"

這是我發現在 Mac OS X 10.6.7 系統上以各種寫入大小執行它的結果(請注意大於 16KiB 的寫入更改):

% /bin/bash -c 'for p in {0..18}; do /tmp/ts.sh $((2 ** $p)) 0.5; done'
write size:          1; bytes successfully before error: 16384
write size:          2; bytes successfully before error: 16384
write size:          4; bytes successfully before error: 16384
write size:          8; bytes successfully before error: 16384
write size:         16; bytes successfully before error: 16384
write size:         32; bytes successfully before error: 16384
write size:         64; bytes successfully before error: 16384
write size:        128; bytes successfully before error: 16384
write size:        256; bytes successfully before error: 16384
write size:        512; bytes successfully before error: 16384
write size:       1024; bytes successfully before error: 16384
write size:       2048; bytes successfully before error: 16384
write size:       4096; bytes successfully before error: 16384
write size:       8192; bytes successfully before error: 16384
write size:      16384; bytes successfully before error: 16384
write size:      32768; bytes successfully before error: 65536
write size:      65536; bytes successfully before error: 65536
write size:     131072; bytes successfully before error: 0
write size:     262144; bytes successfully before error: 0

Linux 3.19 上的相同腳本:

/bin/bash -c 'for p in {0..18}; do /tmp/ts.sh $((2 ** $p)) 0.5; done'
write size:          1; bytes successfully before error: 65536
write size:          2; bytes successfully before error: 65536
write size:          4; bytes successfully before error: 65536
write size:          8; bytes successfully before error: 65536
write size:         16; bytes successfully before error: 65536
write size:         32; bytes successfully before error: 65536
write size:         64; bytes successfully before error: 65536
write size:        128; bytes successfully before error: 65536
write size:        256; bytes successfully before error: 65536
write size:        512; bytes successfully before error: 65536
write size:       1024; bytes successfully before error: 65536
write size:       2048; bytes successfully before error: 65536
write size:       4096; bytes successfully before error: 65536
write size:       8192; bytes successfully before error: 65536
write size:      16384; bytes successfully before error: 65536
write size:      32768; bytes successfully before error: 65536
write size:      65536; bytes successfully before error: 65536
write size:     131072; bytes successfully before error: 0
write size:     262144; bytes successfully before error: 0

注意:PIPE_BUF在 C 標頭檔中定義的值(以及 的pathconf_PC_PIPE_BUF)沒有指定管道的容量,而是可以原子寫入的最大字節數(參見POSIX write(2))。

引用自include/linux/pipe_fs_i.h

/* Differs from PIPE_BUF in that PIPE_SIZE is the length of the actual
  memory allocation, whereas PIPE_BUF makes atomicity guarantees.  */

引用自:https://unix.stackexchange.com/questions/11946