I believe that all developers writing Go language projects must be using Go module now, and Go module files are written in /go/pkg/mod directory by default, so when you want to string CI/CD process, because it is not under the project path, each container cannot share the /go/pkg/mod path, resulting in repeated downloads of third-party packages. In fact, the cross-container solution can be solved by Drone’s Temporary Volumes, but eventually, when you want to finish the compilation process, you can pack the final mod directory and save it for the next CI/CD deployment process, and if you can change the /go/pkg/mod path, you can dynamically adjust the directory structure. The following is an explanation of the Drone deployment tool.

GOMODCACHE Environment Variables

Go 1.15 started to support GOMODCACHE environment variable, the default is GOPATH[0]/pkg/mod, now this path can be corrected by environment variable. This tutorial will use the meltwater/drone-cache package to complete the cache mechanism of the go module to speed up each subsequent CI/CD deployment. To do the cache, we need to create Temporary Volumes in the pipeline first.

1
2
3

volumes:
- name: cache
  temp: {}

With this temporary space, you can see the same files in containers that are not synchronized with each other. Next, set up the steps to compile the Go project.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

- name: build
  pull: always
  image: golang:1.15-rc
  commands:
  - make build
  environment:
    CGO_ENABLED: 0
    GOMODCACHE: '/drone/src/pkg.mod'
    GOCACHE: '/drone/src/pkg.build'
  when:
    event:
      exclude:
      - tag
  volumes:
  - name: cache
    path: /go

Here you can notice that since Go 1.15 is not yet released (expected 2020/08), we have used the rc version first. Here you can notice that we modified GOMODCACHE to put the original mod content in /drone/src/pkg.mod and /drone/src is the project directory, so remember to set a directory name that will not be used, please use the absolute path.

Using build cache

In the CI/CD compilation process, the first step is to download the remotely backed up mod file to the container and uncompress it to the path specified by GOMODCACHE.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

- name: restore-cache
  image: meltwater/drone-cache
  environment:
    AWS_ACCESS_KEY_ID:
      from_secret: aws_access_key_id
    AWS_SECRET_ACCESS_KEY:
      from_secret: aws_secret_access_key
  pull: always
  settings:
    debug: true
    restore: true
    cache_key: '{{ .Repo.Name }}_{{ checksum "go.mod" }}_{{ checksum "go.sum" }}_{{ arch }}_{{ os }}'
    bucket: drone-cache-demo
    region: ap-northeast-1
    local_root: /
    archive_format: gzip
    mount:
      - pkg.mod
      - pkg.build
  volumes:
  - name: cache
    path: /go

Here you can see that I am using AWS S3 as the backend storage, you can also change it via SFTP or other methods. In archive_format please select gzip to make the file smaller. Then you will go through a series of Go processes, such as testing, compiling, packaging … etc., and finally pkg.mod will be packaged and uploaded to AWS S3.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

- name: rebuild-cache
  image: meltwater/drone-cache
  pull: always
  environment:
    AWS_ACCESS_KEY_ID:
      from_secret: aws_access_key_id
    AWS_SECRET_ACCESS_KEY:
      from_secret: aws_secret_access_key
  settings:
    rebuild: true
    cache_key: '{{ .Repo.Name }}_{{ checksum "go.mod" }}_{{ checksum "go.sum" }}_{{ arch }}_{{ os }}'
    bucket: drone-cache-demo
    region: ap-northeast-1
    archive_format: gzip
    mount:
      - pkg.mod
      - pkg.build
  volumes:
  - name: cache
    path: /go

Please refer to the code here.