In this article, we will briefly introduce Generators, and then focus on the operation mechanism of Generators and its implementation in ES5 with my experience in C#.
A simple Generator function example.
The above code defines a generator function that is not executed immediately when the generator function example() is called, but instead returns a generator object. Whenever the .next() method of the generator object is called, the function will run to the next yield expression, return the result of the expression, and pause itself. When the end of the generator function is reached, the value of done is true and the value of value is undefined. we will call the above example() function a generator function, and the difference between the two is as follows
- Normal functions use function declaration, generator functions use function* declaration.
- Normal functions use return, generator functions use yield.
- Normal functions are in run to completion mode, i.e., they are executed until all statements of the function are completed, during which time other code statements are not executed; generator functions are in run-pause-run mode, i.e., generator functions can be paused once or more during function execution and resumed later, during which allows other code statements to be executed
Generators in C
Generators are not a new concept, I was first introduced to them when I was learning to use C#, which introduced the yield keyword from version 2.0, making it easier to create enumerated numbers and enumerable types. The difference is that instead of calling them generators Generators in C#, they are called iterators.
This article will not cover the C# enumerable class IEnumerable and the enumerator IEnumerator, for that we recommend reading the chapter “C# 4.0 Illustrated Tutorial”.
C# Iterator Introduction
Let’s start with an example where the following method declaration implements an iterator that generates and returns an enumeration number.
The method definition is very close to the ES6 Generators definition in that it declares a generic enumerable type that returns an int type, and the method body returns the value and suspends itself via a yield return statement.
Iterators are used to create classes of enumerable types.
The above code will produce the following output.
C# Iterator Principle
Net runtime feature, but rather a syntactic sugar that is compiled into simple IL code by the C# compiler when the code is compiled.
Continuing with the above example, we can see through the Reflector decompiler tool that the compiler generates an internal class for us with the following declaration.
The original Example() method returns only an instance of YieldEnumerator and passes the initial state -2 to itself and its referrer, each iterator saving one state indication.
- -2: initialized to the iterable class Enumerable
- -1: End of iteration
- 0: initialized to iterator Enumerator
- 1-n: yield return index value in the original Example() method
The code in the Example() method is converted to YieldingEnumerator.MoveNext(), and in our example the converted code is as follows.
Using the above code transformation, the compiler generates a state machine for us and it is based on this state machine model that the properties of the yield keyword are implemented.
The iterator state machine model can be shown in the following figure.
- Before is the initial state of the iterator
- Running is the state entered after calling MoveNext. In this state, the enumerator detects and sets the position of the next item. When yield return, yield break or end of iteration is encountered, this state is exited
- Suspended is the state where the state machine waits for the next call to MoveNext.
- After is the state where the iteration ends
By reading the above, we understand the use of Generator in C#, and by looking at the IL code generated by the compiler, we know that the compiler generates an internal class to hold the context information, and then converts the yield return expression into a switch case, which implements the yield keyword through the state machine model.
How Generators work in ES5
The Regenerator tool already implements the above idea, and with the Regenerator tool we can already use generator functions in native ES5. In this section, we’ll analyze the Regenerator implementation to get a deeper understanding of how Generators work.
This online address allows you to easily view the converted code, still using the article as an initial example.
After conversion, it is as follows.
As you can see from the converted code, similar to the C# compiler’s conversion of yield return expressions, Regenerator rewrites the yield expressions in the generator functions as switch cases, and uses context110 in each case to preserve the current context state of the function.
In addition to the switch case, the iterator function example is wrapped by regeneratorRuntime.mark and returns an iterator object wrapped by regeneratorRuntime.wrap.
The example is wrapped in the following object by mark wrapping.
When the generator function example() is called, an iterator object wrapped by the wrap function is returned.
The returned iterator object is shown below.
When calling the iterator object iter.next() method, the _invoke method is executed because of the following code, and according to the previous wrap method code, the makeInvokeMethod(innerFn, self, context); method of the iterator object is finally called.
The makeInvokeMethod method has a lot of content, so here is a partial analysis. First, we find that the generator initializes its state to “Suspended Start”.
makeInvokeMethod returns the invoke function, and when we execute the .next method, the following statement in the invoke method is actually called.
In the tryCatch method, fn is the converted example$ method and arg is the context object context, because the reference to context inside the invoke function forms a closure reference, so the context context is maintained throughout the iteration.
The tryCatch method actually calls the example$ method, which enters the converted switch case and executes the code logic. If the result is a normal type value, we wrap it in an iterable object format and update the generator state to GenStateCompleted or GenStateSuspendedYield.
By analyzing the Regenerator transformed generator code and tool source code, we have explored the operation of the generator, which wraps the generator function with tool functions, adding methods such as next/return. It also wraps the returned generator object so that calls to methods such as next end up in a state machine model consisting of switch case. In addition, the closure technique is used to preserve the generator function context information.
The above process is basically the same as the implementation of the yield keyword in C#, which uses the compile conversion idea, the state machine model, and the preservation of function context information to realize the new language features brought by the new yield keyword.