Linux Blocking IO

IO tasks

An IO task is a process in which a program reads data from a file or writes data to a file, a process that generally requires the CPU to control it. For example, a network IO task.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

//创建socket
int s = socket(AF_INET, SOCK_STREAM, 0);   
//绑定
bind(s, ...)
//监听
listen(s, ...)
//接受客户端连接
int c = accept(s, ...)
//接收客户端数据
recv(c, ...);
//将数据打印出来
printf(...)

Or a local file IO task.

1
2
3
4

//打开文件，获取fd
open(...);
//读取内容
read(...);

In some cases, IO operations cannot be completed immediately. For example, in network IO, the process calls accept to try to receive a client connection, but at this time there is no client initiating the connection, so how does the process handle the execution of the accept function? At this point, the concept of blocking and non-blocking comes into play.

Blocking and interrupting programs

Blocking

When the CPU executes process tasks, it divides the computing resources by time slice, and the processes that need to use the CPU are queued in the work queue, and the CPU takes turns to take out processes from the queue, allocate time slices and execute tasks. Suppose at this time process A executes the accept function, but the client does not initiate a connection, then if nothing is done, process A will remain in the work queue and keep getting stuck at the place where the accept function is executed, and the CPU usage is 100%.

So at this point the system needs to adopt some strategies to avoid meaningless computational resource consumption.

When the accept function is executed, but the client does not initiate a connection, the system removes process A from the work queue and puts it in the wait queue of accept’s socketfd. At this point, the CPU executes the task without allocating any time slice to process A. We call process A a blocking state. So when does the execution of process A resume?

Interrupting a process

After the client initiates the connection and the NIC receives the data, it executes an interrupt program to interrupt the current CPU task, which does two things: 1. copies the data received by the NIC to the kernel buffer (i.e., the socket’s receive buffer) and 2. puts process A back in the execution queue. when the CPU executes the process task again, the accept function gets the client connection data and the program continues to execute normally. The program continues to execute normally.

Now to understand blocking a little more.

• It is inefficient for a single process, once it enters the blocking state, the process cannot perform other tasks.
• Efficient for the system as a whole, allowing processes that do not need to continue to execute to enter a blocking state, giving up CPU and increasing the processing power of the system.

Some operations involving blocking

There are quite a few operations that can cause a process to block, here is a small list of those covered in this article, more will be sorted out later.

• accept function
• read function
• write function
• connect function

Example

Listening

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

int ListenINETPort(){
    //监听的 inet 地址结构体
    struct sockaddr_in inet_addr;
    inet_addr.sin_family = AF_INET;
    inet_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    inet_addr.sin_port = htons(8080);
    //创建监听socket，常用类型 SOCK_STREAM-tcp SOCK_DGRAM-udp SOCK_RAW-icmp，可以叠加SOCK_NONBLOCK将socket设置为非阻塞模式，如 SOCK_STREAM|SOCK_NONBLOCK
    int sfd = socket(AF_INET, SOCK_STREAM, 0);
    //inet地址绑定到socket
    if( bind(sfd, (struct sockaddr *)&inet_addr, sizeof(inet_addr)) == -1 ) handle_error("bind inet error");
    //监听，第二个参数定义积压的连接数，例如这里定义为2，那么超过2个积压后，客户端可以收到 ECONNREFUSED 错误
    if(listen(sfd, LISTEN_BACKLOG) == -1) handle_error("listen");
    return sfd;
}

Receive connections and read data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

int AcceptAndRecvINET(int sfd){
    //接收连接的循环
    while(1){
        //创建接收的socket地址
        sockaddr_in client_addr;
        socklen_t client_addr_length = sizeof(client_addr);
        //接收客户端连接
        int cfd = accept(sfd, (struct sockaddr*)&client_addr, &client_addr_length);
        if(cfd == -1){
            printf("accept fail: %d", errno);
            return 0;
        }
        printf("accept connect success: %d", cfd);
        //定义一个缓冲区数据块，用于接收客户端的数据
        char buf[BUFFER_SIZE];
        //接收数据的循环
        while (1)
        {
            bzero(buf, 0);
            //接收客户端的数据，最后一个参数是flags，用于控制接收消息的模式
            int recvbytes = recv(cfd, buf, sizeof(buf), 0);
            //客户端断开
            if(recvbytes == 0){
                printf("client is disconnect: %d \n", cfd);
                break;
            }
            //接收错误，继续
            if(recvbytes < 0){
                printf("recv error: %d", errno);
                continue;
            }
            //打印结果
            printf("client->server: %s \n", buf);
            //原样返回
            send(cfd, buf, sizeof(buf), 0);
        }
        close(cfd);
    }
    close(sfd);
}

Start the server-side program and start multiple client programs to initiate connections and then send data.

As you can see, with blocking io, a process can only process one connection at a time, and other connections need to wait for the previous connection to be disconnected before they can be processed.