coroutine A long time ago I knew this thing, but Java did not, so I did not know much about it, recently in learning Python see coroutine, make a record.

Concepts

Speaking of coroutine is generally associated with processes and threads, usually please compare the three as follows.

  • Process: an instance of program execution, a process contains at least one thread, and switching between different processes is costly.
  • Thread: the basic unit of CPU scheduling, an entity of a process, the context switching cost of threads is smaller than that of processes.
  • coroutine : a user-state lightweight thread, a thread can contain more than one coroutine .

The biggest advantage of coroutine is its extremely high execution efficiency. Because subroutine switching is not thread switching, but controlled by the program itself, there is no overhead of thread switching, and the more threads there are compared to multiple threads, the more obvious the performance advantage of coroutine is.

The second major advantage is that there is no need for the lock mechanism of multi-threading, because there is only one thread, and there is no conflict of writing variables at the same time. In coroutine, the shared resources are controlled without locking, and only the state needs to be judged, so the execution efficiency is much higher than multi-threading.

Coroutine in Python

generators generator and yield keywords

If a function definition contains the yield keyword, then it is a generator function.

The syntax of yield is that it pauses the function here at yield and returns the value of the expression after yield (default is None) until it is called again by the next() method, which continues from the last paused yield code. When there is no way to continue next(), an exception is thrown, which can be handled by the for loop.

Each generator can execute the send() method, which sends data to the yield statement inside the generator; Python’s support for coroutines is implemented through the generator.

Look at an example of a producer and a consumer, where the producer produces a message and then yields to the consumer to consume it, and the consumer executes and jumps back to the producer to continue producing, all within a single thread.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

def consumer():
    r = 'start task...'
    while True:
        # When a yield statement is returned, execution continues from the last returned yield statement
        n = yield r
        if not n:
            return
        print('[CONSUMER] Consuming %s...' % n)
        r = 'success'

def produce(c):
    # The first time the generator is run, use the send() function, passing None parameter to start the generator
    print(c.send(None))
    n = 0
    while n < 3:
        n = n + 1
        print('[PRODUCER] Producing %s...' % n)
        r = c.send(n)
        print('[PRODUCER] Consumer return: %s' % r)
    c.close()

c = consumer()
produce(c)

Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

start task...
[PRODUCER] Producing 1...
[CONSUMER] Consuming 1...
[PRODUCER] Consumer return: success
[PRODUCER] Producing 2...
[CONSUMER] Consuming 2...
[PRODUCER] Consumer return: success
[PRODUCER] Producing 3...
[CONSUMER] Consuming 3...
[PRODUCER] Consumer return: success

@asyncio.coroutine and yield from

@asyncio.coroutine marks a generator as a coroutine, and yield from waits for the return of another coroutine. asyncio is a time-loop-based asynchronous IO module introduced in Python 3.4.

The programming model of asyncio is a message loop. We get a reference to an EventLoop directly from the asyncio module, and then throw the coroutine that needs to be executed into the EventLoop to execute it, which implements asynchronous IO.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import asyncio, datetime

@asyncio.coroutine
def task(name):
    print("Start: {} Time: {}".format(name, datetime.datetime.now()))
    # Simulation of delayed IO tasks, such as network requests or writing files, etc.
    yield from asyncio.sleep(2)
    print('continue other task: ', name)
    print("End: {} Time: {}".format(name, datetime.datetime.now()))

loop = asyncio.get_event_loop()
tasks = [task('t1'), task('t2')] 
# loop.run_until_complete(asyncio.wait(tasks))
# wait and gather return different values
loop.run_until_complete(asyncio.gather(*tasks))
loop.close()

Output.

1
2
3
4
5
6
7
8

/coroutin/test.py:6: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
  def task(name):
Start: t1 Time: 2022-02-26 00:43:01.195497
Start: t2 Time: 2022-02-26 00:43:01.196419
continue other task:  t1
End: t1 Time: 2022-02-26 00:43:03.199804
continue other task:  t2
End: t2 Time: 2022-02-26 00:43:03.200136

The environment here is Python 3.8, and you can see the warning that @coroutine is not recommended for newer versions, and that you can use async def to define the coroutine.

async and await

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import asyncio, datetime

async def task(name):
    print("Start: {} Time: {}".format(name, datetime.datetime.now()))
    await asyncio.wait([ioTask(name)])
    print('continue other task: ', name)
    print("End: {} Time: {}".format(name, datetime.datetime.now()))

async def ioTask(name):
    # Simulation of delayed IO tasks, such as network requests or writing files, etc.
    print('execute io task: ', name)
    await asyncio.sleep(2)

loop = asyncio.get_event_loop()
tasks = [task('t1'), task('t2')] 
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

Output.

1
2
3
4
5
6
7
8

Start: t2 Time: 2022-02-26 00:46:01.575044
Start: t1 Time: 2022-02-26 00:46:01.575539
execute io task:  t2
execute io task:  t1
continue other task:  t2
End: t2 Time: 2022-02-26 00:46:03.580642
continue other task:  t1
End: t1 Time: 2022-02-26 00:46:03.580710

coroutine is suitable for IO-intensive, not CPU-intensive applications.

Why Java doesn’t have a coroutine

In the current release of Java, there is no coroutine yet. The usual point of using coroutine is to save the overhead of creating and switching threads, but there have always been other solutions in Java, such as

  • the availability of non-blocking I/O client-server frameworks such as Netty.
  • thread pools solve the overhead of thread creation and destruction.
  • The JDK also has complete tools like JUC for asynchronous programming.

The biggest benefit of using coroutine in other languages is that it is simple and elegant to write, synchronous to write and asynchronous to run. In contrast Java’s asynchronous threads are much more complex to write, and Java has always been criticized for being too cumbersome.

Java is also currently pushing the development of a coroutine library, the Loom project, which is still in development.