為什麼 kworker 在 Linux 3.0.0-12-server 上消耗這麼多資源？

上週五，我將我的 Ubuntu 伺服器升級到 11.10，它現在使用 3.0.0-12-server 核心執行。從那時起，整體性能急劇下降。在升級之前，系統負載約為 0.3，但目前在具有 16GB RAM（10GB 空閒，未使用交換）的 8 核 CPU 系統上為 22-30。

我要責怪 BTRFS 文件系統驅動程序和底層 MD 陣列，因為

$$ md1_raid1 $$和$$ btrfs-transacti $$消耗了大量資源。但是所有的[數學處理錯誤]$$ kworker/: $$消耗更多。 sar自周五以來不斷輸出類似的內容：

11:25:01        CPU     %user     %nice   %system   %iowait    %steal     %idle 
11:35:01        all      1,55      0,00     70,98      8,99      0,00     18,48 
11:45:01        all      1,51      0,00     68,29     10,67      0,00     19,53 
11:55:01        all      1,40      0,00     65,52     13,53      0,00     19,55 
12:05:01        all      0,95      0,00     66,23     10,73      0,00     22,10 

And iostat confirms a very poor write rate:

sda             129,26      3059,12       614,31  258226022   51855269          
sdb              98,78        24,28      3495,05    2049471  295023077          
md1             191,96       202,63       611,95   17104003   51656068          
md0               0,01         0,02         0,00       1980        109          

The question is: How can I track down why the kworker threads consume so many resources (and which one)? Or better: Is this a known issue with the 3.0 kernel, and can I tweak it with kernel parameters?

Edit:

I updated the Kernel to the brand new version 3.1 as recommended by the BTRFS developers. But unfortunately this didn’t change anything.

I found this thread on lkml that answers your question a little. (It seems even Linus himself was puzzled as to how to find out the origin of those threads.)

Basically, there are two ways of doing this:

$ echo workqueue:workqueue_queue_work > /sys/kernel/debug/tracing/set_event
$ cat /sys/kernel/debug/tracing/trace_pipe > out.txt
(wait a few secs)

For this you will need ftrace to be compiled in your kernel, and to enable it with:

mount -t debugfs nodev /sys/kernel/debug

More information on the function tracer facilities of Linux is available in the ftrace.txt documentation.

This will output what threads are all doing, and is useful for tracing multiple small jobs.

cat /proc/THE_OFFENDING_KWORKER/stack

This will output the stack of a single thread doing a lot of work. It may allow you to find out what caused this specific thread to hog the CPU (for example). THE_OFFENDING_KWORKER is the pid of the kworker in the process list.

引用自：https://unix.stackexchange.com/questions/22851