FIO 基準測試如何設置 IOdepth？

注意： 我的問題源於另一個 U&L Q - fio 中的 iodepth 到底是什麼？

---

我想知道FIO如何在內部設置 I/O 深度。即，當我們執行它時，我們送出給 FIO 的參數之一是“IOdepth”（--iodepth=）。FIO內部如何與底層作業系統控制這個參數？

以下是我們用來執行 FIO 基準測試的命令範例：

$ sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=randwrite --refill_buffers \
--norandommap --randrepeat=0 --ioengine=libaio --bs=8K **--iodepth=72** --numjobs=256 \
--time_based --runtime=600 --allow_mounted_write=1 --group_reporting --name=benchtest
基準測試：（g=0）：rw=randwrite，bs=8K-8K/8K-8K/8K-8K，ioengine=libaio，**iodepth=72**

如本例所示，“iodepth”的值可以更改。因此，fio 將該值傳遞給作業系統。那麼 FIO 是如何做到這一點的呢？

如果你想解決一個實際問題：如果我想寫一個像fio這樣的基準程序，我將如何控制IO隊列深度？

fio 正在傳遞這個值

$$ depth $$到作業系統

和

因此，fio 將該值傳遞給作業系統。那麼 FIO 是如何做到這一點的呢？

這裡可能有一個誤解：fio 不是直接將深度參數傳遞給作業系統。如果可能，fio 會嘗試iodepth使用給定的 ioengine 送出指定的 I/O。如果達到該深度，則fio 將等待（一些）未完成的 I/O 完成，然後再嘗試送出更多 I/O…

FIO基準如何設置

$$ io $$深度？

這取決於 ioengine，取決於 fio 中的iodepth 到底是什麼？, https://serverfault.com/questions/923487/what-does-iodepth-in-fio-tests-really-mean-is-it-the-queue-depth和https://www.spinics.net/lists /fio/msg07191.html。沒有固定的小例子，解釋的太多了。

有一點，除了閱讀和理解 fio 本身的程式碼外，別無他法…… Fio 有用於送出 I/O 的主循環（參見https://github.com/axboe/fio/blob/fio- 3.8/後端.c#L1055）：

static void do_io(struct thread_data *td, uint64_t *bytes_done)
{
   [...]
   while ((td->o.read_iolog_file && !flist_empty(&td->io_log_list)) ||
       (!flist_empty(&td->trim_list)) || !io_issue_bytes_exceeded(td) ||
       td->o.time_based) {
       [...]
       } else {
           ret = io_u_submit(td, io_u);

           if (should_check_rate(td))
               td->rate_next_io_time[ddir] = usec_for_io(td, ddir);

           if (io_queue_event(td, io_u, &ret, ddir, &bytes_issued, 0, &comp_time))
               break;

           /*
            * See if we need to complete some commands. Note that
            * we can get BUSY even without IO queued, if the
            * system is resource starved.
            */
reap:
           full = queue_full(td) ||
               (ret == FIO_Q_BUSY && td->cur_depth);
           if (full || io_in_polling(td))
               ret = wait_for_completions(td, &comp_time);
       }
       [...]
   }
   [...]
}

The ioengine’s queuing routine is invoked by a call chain from io_u_submit(). Assuming the ioengine is asynchronous, it may choose to just “queue” the I/Os within fio and then at a later point submit the whole lot down in one go (usually as a result of its getevents() function being called from a wait_for_completions() call chain). However we’ll leave tracing through fio’s code as an exercise for the reader.

if I want to write a benchmark program like fio, how would I control the IO queue depth?

You would need to mimic one of fio’s (asynchronous) ioengines and have an event loop that was capable of (asynchronously) submitting I/Os AND checking for their completion. Once you had such a thing the idea that you only submit up to a particular depth would be easy - if at any point you have outstanding uncompleted I/O that matches (or exceeds if you aren’t checking one by one) the chosen depth then you need to wait for something to be completed before you submit more.

You may find aio-stress.c in the Linux Test Project easier to understand/modify than fio if you’re making a toy benchmark.

引用自：https://unix.stackexchange.com/questions/460491