Event queuing via epoll/kqueue and IOCP – Understanding OS-Backed Event Queues, System Calls, and Cross-Platform Abstractions

This is a sort of hybrid of the previous approaches. In the case of a network call, the call itself will be non-blocking. However, instead of polling the handle regularly, we can add that handle to an event queue, and we can do that with thousands of handles with very little overhead.

As programmers, we now have a new choice. We can either query the queue with regular intervals to check if any of the events we added have changed status or we can make a blocking call to the queue, telling the OS that we want to be woken up when at least one event in our queue has changed status so that the task that was waiting for that specific event can continue.

This allows us to only yield control to the OS when there is no more work to do and all tasks are waiting for an event to occur before they can progress. We can decide exactly when we want to issue such a blocking call ourselves.

Note

We will not cover methods such as poll and select. Most operating systems have methods that are older and not widely used in modern async runtimes today. Just know that there are other calls we can make that essentially seek to give the same flexibility as the event queues we just discussed.

Readiness-based event queues

epoll and kqueue are known as readiness-based event queues, which means they let you know when an action is ready to be performed. An example of this is a socket that is ready to be read from.

To give an idea about how this works in practice, we can take a look at what happens when we read data from a socket using epoll/kqueue:

1. We create an event queue by calling the syscall epoll_create or kqueue.
2. We ask the OS for a file descriptor representing a network socket.
3. Through another syscall, we register an interest in Read events on this socket. It’s important that we also inform the OS that we’ll be expecting to receive a notification when the event is ready in the event queue we created in step 1.
4. Next, we call epoll_wait or kevent to wait for an event. This will block (suspend) the thread it’s called on.
5. When the event is ready, our thread is unblocked (resumed) and we return from our wait call with data about the event that occurred.
6. We call read on the socket we created in step 2.

Figure 3.1 – A simplified view of the epoll and kqueue flow