This is an old revision of the document!
What is a Wait Channel?
Threads in the kernel can wait for resources using various APIs such as tsleep() or cv_wait(). The resource being waited on is associated with a pointer in the kernel's virtual address space. These pointers are known as wait channels. In addition to the raw pointer itself, each thread stores a reference to a short string describing the wait channel. This string is known as a wait message.
Finding Wait Channels
There are various ways of determining which wait channel a thread is waiting on. The raw pointer can be displayed via the “nwchan” keyword for ps(1):
> ps O nwchan PID NWCHAN TT STAT TIME COMMAND 6515 - v0 Z 0:00.19 <defunct> 6517 fffff8000deffa18 v0 IW 0:00.00 -tcsh (tcsh) 6545 fffff8000dbea970 v0 IW+ 0:00.00 /bin/sh /usr/local/bin/startx ...
The wait message is available via the “wchan” keyword:
> ps O wchan PID WCHAN TT STAT TIME COMMAND 6515 - v0 Z 0:00.19 <defunct> 6517 pause v0 IW 0:00.00 -tcsh (tcsh) 6545 wait v0 IW+ 0:00.00 /bin/sh /usr/local/bin/startx ...
The ps(1) command also includes another keyword called “mwchan”. This keyword displays the wait message for a thread blocked on a wait channel. For a thread blocked on a mutex, read-mostly lock or read/write lock, the name of the lock is displayed after a “*” prefix. Note that other locks such as lockmanager and shared/exclusive locks are implemented using wait channels, so threads blocked on one of those locks display the lock name as the wait message without a “*” prefix.
In addition, the STATE column in top(1) displays the same data as the “mwchan” keyword for threads blocked on either a wait channel or a lock. Wait messages generally use lower-case letters in contrast to the non-waiting status messages displayed in this column such as CPU x, RUN, or WAIT.
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 64K CPU3 3 275.1H 92.09% idle{idle: cpu
11 root 155 ki31 0K 64K RUN 2 273.6H 89.45% idle{idle: cpu
11 root 155 ki31 0K 64K RUN 0 273.6H 87.89% idle{idle: cpu
11 root 155 ki31 0K 64K RUN 1 273.6H 86.77% idle{idle: cpu
6692 john 39 0 445M 119M CPU1 1 267:50 22.07% akonadi_imap_r
6637 john 33 0 217M 73660K CPU2 2 158:27 15.19% akonadiserver{
6638 john 27 0 242M 153M sbwait 0 204:18 7.86% mysqld{mysqld}
6692 john 24 0 445M 119M select 0 3:22 4.49% akonadi_imap_r
6559 john 21 0 3330M 101M select 3 127:07 0.88% Xorg
6622 john 20 0 613M 93880K select 3 85:02 0.29% kwin{kwin}
6629 john 20 0 849M 154M select 3 49:13 0.10% kdeinit4{kdein
6638 john 20 0 242M 153M sbwait 2 219:54 0.00% mysqld{mysqld}
7546 john 20 0 505M 86916K select 3 48:41 0.00% kdeinit4{kdein
12 root -96 - 0K 320K WAIT 3 42:18 0.00% intr{irq264: v
1765 root 20 0 28876K 4180K select 1 37:53 0.00% hald-addon-mou
8 root 16 - 0K 16K syncer 3 32:32 0.00% syncer
6692 john 20 0 445M 119M select 3 17:17 0.00% akonadi_imap_r
Common Wait Channels
This is a table of some of the common wait channel messages one might see in a system and a brief description of what state the associated thread is in (for example, which resource a thread is waiting on). Note that this list is not exhaustive.
| Wait Channel | Description |
|---|---|
| biord | Waiting for a disk read operation to complete |
| biowr | Waiting for a disk write operation to complete |
| nfsreq | Waiting for a reply to an NFS request |
| wdrain | Waiting for pending writes to a file to drain to disk |
| flushbuf | Waiting for pending writes to a file to drain to disk so the pages can be flushed from RAM via posix_fadvise(2) |
| sbwait | Waiting for data to arrive on a socket, e.g. a blocking read(2) on a socket |
| kqread | Waiting for an event to arrive on a kqueue |
| select | Waiting for an event to trigger on a file descriptor via select(2) or poll(2), usually a socket |
| piperd | Waiting for data to arrive on a pipe, i.e. a blocking read(2) |
| pipewr | Waiting for room to write data into a pipe, i.e. a blocking write(2) |
| fifoor | Waiting in a blocking read-only open of a FIFO (O_RDONLY) for another process to open a writable descriptor on this FIFO (O_RDWR or O_WRONLY) |
| fifoow | Waiting in a blocking write-only open of a FIFO (O_WRONLY) for another process to open a readable descriptor on this FIFO (O_RDWR or O_RDONLY) |
| wait | Waiting for a child process to exit, e.g. a shell waiting for a command to complete |
| ttyin | Waiting to read data from a terminal, e.g. idle shell for an ssh session or xterm |
| ttyout | Waiting to write data to a terminal |
| pause | Waiting for a signal (tcsh uses this instead of “wait”) |
| nanslp | Waiting for a specified amount of time to pass, e.g. “sleep 1” |
| ucond | Waiting for a userland condition variable (pthread_cond_wait()) |
| getblk | Waiting to access file data that is being accessed by another thread |
| ufs | Waiting to read from a file or directory on a UFS filesystem that is being written to by another thread, or waiting to write to a local file or directory that is being read or written by another thread |
| nfs | Same as “ufs” but for an NFS file or directory. Often if you see this it's because the other thread is blocked in “nfsreq”. |