The Go language added support for modular programming and a built-in module-based dependency management tool in version 1.11, released in August 2018. modules in the Go language are collections of packages in a file tree, where the
go.mod file contained in the module root directory defines the module’s import path, the Go language version, and other dependency requirements for the module. Each module’s dependency requirements are listed as a separate module path and the corresponding module version is specified, and only modules that meet all dependency requirements can be successfully built.
With the modular programming capabilities that come with Go, there is no need to put Go code into
$GOPATH/src, which is the old GOPATH mode; in fact, we can create Go projects and initialize Go modules using
go mod init in any directory outside of
Note: For compatibility, Go commands can still be run in the old GOPATH mode in Go 1.11 and 1.12. Starting with Go 1.13, module mode (
GO111MODULE=on) will be the default mode.
The history of GOPATH
Before the Go language supported modular programming, our Go projects generally needed to use the GOPATH pattern, which means that the Go code needed to be placed in
$GOPATH/src. A typical GOPATH directory structure contains the following three subfolders.
go get command to get the dependencies will also automatically download them to
However, there is a problem with GOPATH mode. When we use
go get to get a dependency without specifying a version, the dependency code downloaded by default will be the latest version, and if project A and B depend on two incompatible versions of project C, the GOPATH path with only one version of C will not be able to satisfy both project A and B’s dependency needs. This is a tricky flaw, and in fact, the GOPATH pattern has not been officially recommended since Go 1.13.
In addition, as the Go language becomes more and more popular, dependency packages become more and more abundant, and the issue of dependency management becomes a focus for developers.
The GOPATH pattern has given rise to many version management tools, but the basic idea is to maintain a separate copy of the dependencies for each project.
- the vendor feature was first officially introduced in Go 1.5: a
vendor/directory is created for each Go project to store copies of the project’s required version dependencies
- The community has developed various version management tools based on the vendor feature. Some popular ones are govendor, and godep, which was officially recognized before.
Go dependency management tools are abundant, but there are incompatibilities between different versions of tools, and there are also learning costs for all kinds of tools. So the Go community proposed the vgo scheme, and with the gradual development and maturity of vgo, Go 1.11 released the Go module function based on this scheme, and integrated it into the official Go language tools.
Environment variables related to Go modularity support
Using the built-in modularity features of the Go language, there are six environment variables that developers need to care about, which can be listed using the
go env command.
This environment variable controls whether the use of the Go module feature is enabled. The optional values and descriptions are shown in the table below.
|on||Enable Go modularity mode, recommended|
|off||Use GOPATH mode, disable Go modularity mode, not recommended|
All you need to do to enable Go modular mode is to set the
GO111MODULE environment variable to
This environment variable is used to set a proxy for Go module dependency downloads so that Go can quickly pull module dependencies directly through the proxy site when pulling them. The default value is:
Due to the Chinese government’s network blocking, you can’t access go’s official services directly in China. So you need to set the proxy address to enable Go modular mode, and the common mirror proxy addresses in China are as follows.
The value of the
GOPROXY environment variable is a comma-separated list of Go module proxies, allowing multiple module proxies to be set; if you want to turn off proxies, you can set it to
The default value of
direct followed by a comma is used to tell Go that it will download directly from the source address of the dependency. For example, when a Go mirror proxy in the list of values returns
410, Go automatically tries the next one in the list, and if the next value is
direct goes to the source address to download the dependent module.
Set up the Go module proxy.
The full name of this environment variable is GO checkSUM DataBase, and as it literally says, it is used to ensure that the module version data is not tampered with when pulling module dependencies, and to abort the pull immediately and report an error if inconsistencies are found. The default value of the
GOSUMDB environment variable is:
sum.golang.org. If it is set to
off, Go commands will be prevented from verifying module dependencies when pulling them later, which is not recommended.
It is also inaccessible in China. The good thing is that after setting the proxy address
GOSUMDBcan also be accessed
4. GONOPROXY & GONOSUMDB & GOPRIVATE
These three environment variables are mainly used when a Go project depends on a private module, for example, some dependencies exist in a private git repository, and the mirror proxy set directly with
GOPROXY or the check site set with
GOSUMDB will not be able to access the corresponding private repository. In this case, you need to set these dependency modules to pull checksums directly without going through the mirror proxy site. In fact, for private dependency modules, the best practice is to set the
GOPRIVATE environment variable directly, whose value will be the default for
Their value is a module path prefix separated by an English comma
,, i.e. more than one can be set, e.g.
This setting means that module dependencies prefixed with
github.com/eabc/def will be considered private and will not be pulled directly from the mirror proxy site.
Go Modular Manipulation Commands
go mod initcreates a new module and initializes the
go.modfile that describes it
go testand other package build commands add new dependencies to
go list -m allprints all dependencies of the current module
go getto change the version of the required dependency (or add a new one)
go mod tidyRemove unused dependencies
go mod vendorCopy the correct version of the module dependency from the module to the vendor directory of the project
Initialize the module
We can create Go projects and initialize Go modules using
go mod initin any directory other than
example.com/greetingsis not only the module identifier, but also the import path of the module, which will be used as a common prefix when other Go projects refer to a package under this module, plus the relative path of the package to the root of the module.
The above command will generate a
go.mod file in the current directory after successful execution.
In fact, in Go modularity mode, the dependency information is automatically recorded in the
go.mod file after the module dependency is fetched using the
go get command.
go get command downloads the latest dependency version by default, but you can also specify the tag or commit id for versioning with
@, for example.
Pulling module dependencies with the
go get command will cache the results in the
$GOPATH/pkg/sumdb directories, where the module dependencies are stored in the
$GOPATH/pkg/mod directory in the
After pulling the module dependencies, the
go.mod file will look like this.
indirectcomment indicates that the module is an indirect dependency, meaning that no explicit reference to the module is found in the
importstatement in the current application. Use the
go getcommand to pull module dependencies directly, rather than using
go buildto automatically pull module dependencies based on
Syntax keywords that can be used in the
go.mod file and their meanings.
moduledefines the module path of the current project
goIdentifies the Go language version of the current module
requireindicates the version of the module that depends on it
excludeto exclude a specific module version from use, if a version of the module dependency has a serious bug, you need to explicitly exclude a version, e.g.
exclude github.com/google/uuid v1.1.0means don’t use
replacereplaces the module dependency declared in
require, using another module dependency and its version
Usage scenarios for the
Usage scenario 1: replacing the dependency package of require
go.modfile for the current project is as follows.
Execute the command
go list -m allto list all the dependencies of the current module.
Add this line of configuration under the go.mod file.
replace github.com/google/uuid v1.1.1 => github.com/google/uuid v1.1.0
Execute the command
go list -m allagain to list all the dependencies of the current module.
Found that the module dependency that finally takes effect has changed from
github.com/google/uuid v1.1.0. Generally speaking, this scenario is not used much, and it works the same as modifying
requiredirectly, with the following prerequisites.
- the currently referenced module dependency is valid
- The package name and version on the left side of the
replacecommand must be the corresponding package name and version contained in
Usage scenario 2: Replacing Unable-to-Download Packages
Due to network limitations in China, some module dependencies cannot be downloaded, such as all of the dependencies under
golang.org. The good thing is that these dependencies have mirrors on GitHub, so you can use the mirrors on GitHub to replace the dependencies under the
For example, if you use the
golang.org/x/netpackage in your project.
This way the project will download the
golang.org/x/netpackage from GitHub when it is compiled, and the
golang.org/x/net/xxxwill not need to be changed in the source code.
Usage scenario 3: Debugging Dependency Packages
When debugging a dependency package you can use
replaceto modify the dependency as follows.
uuidto replace the dependency package, at this time, we can modify
. /uuiddirectory to debug. Besides using relative paths, you can also use absolute paths, or even use your own forked repositories.
Usage scenario 4: Prohibition of being dependent
The last scenario where the
replace directive is used is when you don’t want your module to be directly referenced, such as in the
go.mod file of a k8s project, where the
require section has a large number of
Since none of the above dependencies exist for
v0.0.0, other projects that depend directly on
k8s.io/kubernetes will not be able to pull it successfully because the version cannot be found.
Because the k8s core repository does not want to be used directly as a module dependency, other projects can use other subcomponents of the k8s project. k8s hides the dependency version number from the outside, and its real dependencies are specified by
After we build and execute commands like
go build or
go test we will also find a
go.sum file generated in the root of the project. Its main purpose is to check for downloaded module dependencies. In practice, module dependencies can be tampered with during download, and dependencies cached locally can also be tampered with, so a single
go.mod file does not guarantee a consistent build. The
go.sum file was introduced on top of
go.mod to record the hash value of each dependency package. When building a Go project, if the local dependency package hash checksum does not match the one recorded in the
go.sum file, the build will be rejected.
go.mod file is shown below.
Normally, each dependency version will contain two records, the first being the hash of the dependency version as a whole, and the second being the hash of the
go.mod file in the dependency version only, or the first record if the dependency version does not have a
So how is
go get command in the root of the Go project and it will update the
go.sum files simultaneously.
go.mod records the dependency names and their versions, e.g.
go.sum file records the hash of the dependency package (along with the hash of
go.mod in the dependency package). Before updating
go.sum, the Go command also queries the server indicated by the
GOSUMDB environment variable to get an authoritative hash of the dependency version after downloading the dependency to ensure that the downloaded dependency is real and reliable. If the dependency package version hash computed by the Go command does not match the hash given by the
GOSUMDB server, the Go command will refuse to execute further and will not update the
Use and update of Go modules
We first initialize the Go project with the
go mod init command and set the module import path. Then, as we write the code, we write the imported packages, etc. Commands like
go build or
go mod tidy will automatically download the imported packages and update the
go.sum files, and then use a version management tool like Git to commit and manage the project source code and the updated
go.sum files. This ensures that you get consistent dependencies for each build.
minor version update
go get -ucommand will update a module to depend on the latest minor version, for example, it will update
1.1.0something like that; the
go get -u=patchcommand will get the latest patch update, for example, it will update
If our Go project is using dependency package version
1.0.0and the module dependency has just been updated to version
1.0.1, any of the following commands will update our to module dependency version
Major version update
In general, major versions are completely different from minor versions, and major versions can break backwards compatibility. Thus, from the perspective of Go modules, a major version is a completely different package. Two incompatible versions of a library are essentially two different libraries. When you encounter a major version update, you can still use tag to update from
2.0.0, but be sure to pay attention to the backwards compatibility issue.