In a previous article, I wonder if you have noticed that all example code executes with an environment variable
ASSUME_NO_MOVING_GC_UNSAFE_RISK_IT_WITH added in front, like the following:
What is going on here? What happens if you don’t add this environment variable? Let’s try:
We see that the program panic! We see that the error message of panic mentions the package
go4.org/unsafe/assume-no-moving-gc, so obviously the problem is here, so what exactly does the package
assume-no-moving-gc do? What exactly is the purpose of this package? Why does gorgonia.org/tensor rely on this package? This is beyond the scope of the previous article, so I didn’t mention it. In this article, I’ll take a look at the unsafe-assume-no-moving-gc package with you.
2. unsafe-assume-no-moving-gc What exactly is the package?
The canonical import path for the package unsafe-assume-no-moving-gc is go4.org/unsafe/assume-no-moving-gc, which is obviously an open source package from the organization go4.org. Let’s take a look at the go4.org homepage (as follows):
The home page of this site is very “simple”, the biggest value is to explain the origin of go4: gopher harmonic . go4.org open source some Go packages, this can be seen in its official github site.
There are not many projects, and the number of Stars is not much, but looking through the contributor of a random project, we can see former Googler, former Go core team member, and designer of the net/http package Brad Fitzpatrick (bradfitz) and core contributor of Go runtime Josh Bleecher Snyder (josharian). Now both seem to be working in the startup tailscale, do based on the wireguard protocol remote security control platform (simple understanding is the VPN platform). tailscale gathered a handful of Go language original core development, go4.org is their open source some misc go package. And unsafe-assume-no-moving-gc this package is one of them.
So what exactly does this package do? Let’s move on to the following.
3. how unsafe-assume-no-moving-gc works
unsafe-assume-no-moving-gc is a very simple package.
In addition to the test source file, it has only two source files, assume-no-moving-gc.go and untested.go. When you open these two source files, you will find that this package does not even provide any API. so what exactly does this package do? Here is the README of this package.
If your Go package wants to declare that it plays
unsafegames that only work if the Go runtime’s garbage collector is not a moving collector, then add:
import _ "go4.org/unsafe/assume-no-moving-gc"
Then your program will explode if that’s no longer the case. (Users can override the explosion with a scary sounding environment variable.)
This also gives us a way to find all the really gross unsafe packages.
The general idea is that if your code uses the unsafe tip in Go, then your program will work fine provided that the Go runtime garbage collector is not a collector with a migration mechanism.
A collector with a migration mechanism may move some heap objects to other memory addresses during GC recovery. If your application imports the unsafe-assume-no-moving-gc package, it can alert you with a “program startup crash” behavior when Go GC supports migration.
Let’s look at an example:
After go mod tidy, run the source file using Go version 1.20:
Since the current GC in the latest Go 1.20.x version is not a GC with a migration mechanism, running the above program with Go 1.20 will not result in a panic.
Let’s roll back the unsafe-assume-no-moving-gc package to a previous version, for example: v0.0.0-20230221090011-e4bae7ad2296, and then run main.go again.
From the output panic error message, we see that go4.org/unsafe/assume-no-moving-gc has not been upgraded to a version that can be trusted with go 1.20, so running the program with Go 1.20 may be risky. If you can confirm that there will be no problems, you can avoid panic with the environment variable ASSUME_NO_MOVING_GC_UNSAFE_RISK_IT_WITH=go1.20, like this output below.
So how does the unsafe-assume-no-moving-gc package do the above “detection”? The trick is in the source file untested.go. We download the go4.org/unsafe/assume-no-moving-gc source code and “roll back” it to the commit time of 1025295fd063.
Check out untested.go:
This file has two features:
- Uses the build constraint: // +build go1.18, which means that this source file will only be compiled if you are using Go 1.18 and higher.
- Contains the init function, which will be executed when your code imports the assume_no_moving_gc package, creating a “side effect”.
Note: For the usage of build constraint, see go help buildconstraint.
Thus, when we run the above main.go with go 1.20, untested.go will be compiled and the init function will be executed since go 1.20 is larger than go 1.18. If the environment variable corresponding to the constant env (“ASSUME_NO_MOVING_GC_UNSAFE_RISK_IT_WITH”) is not set, then the init function will go to panic, which will cause the program to exit and output the panic message.
Now we switch the version of the assume_no_moving_gc package back to the latest version, which has the following build constraint in untested.go.
This means that the untested.go file will only be compiled if you are using Go 1.21 or above. If we run main.go with go 1.20, we will not “trigger” the side effects of the init function in untested.go, so main.go will runs normally.
Note: As of go 1.20, Go GC still does not move the heap object.
Before understanding the unsafe-assume-no-moving-gc package, I “consulted” ChatGPT about the package’s functionality, and ChatGPT replied as follows:
As you can see, ChatGPT is basically talking nonsense.
unsafe-assume-no-moving-gc is only for GC migration of heap objects, but not for stack address migration. We know that stack addresses change in Go because the initial stack of a goroutine is only 2KB. Once the initial stack of a goroutine is 2KB, Go runtime will extend the stack, i.e., allocate a larger address range for the goroutine’s stack, and then migrate the variables on the original stack to the new stack, so that the addresses of the variables on the original stack will change.
However, if your Go source code uses unsafe tips and relies on the address of the heap object, then it is recommended that you import the unsafe-assume-no-moving-gc package. But be careful, as the latest version of go is released, you have to update the dependent unsafe-assume-no-moving-gc version in time. Otherwise programs that depend on your package will be alerted with panic when users use the latest version of go.