Intro

Golang is an amazing language, its syntax is not very complicated, and the compiled program is a Binary, you don’t need to install any extra runtime to run, if you want to write a small program like Hello World, just follow the tutorial and you can write it quickly. If you want to write a slightly more complex Web application, just find a Web framework, such as Gin, Fiber, etc., which also has a similar experience to ExpressJS.

Of course, Go also has a lot of incredible points, for example, it does not have a centralized package management platform like pypi, npm, etc., and everyone who writes Golang’s program is importing a bunch of stuff from github.com, for example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import (
    "fmt"
    "math"
    "regexp"
    "strconv"
    "unicode"

    "github.com/pingcap/errors"
    "github.com/pingcap/tidb/parser/ast"
    "github.com/pingcap/tidb/parser/auth"
    "github.com/pingcap/tidb/parser/charset"
    "github.com/pingcap/tidb/parser/mysql"
    "github.com/pingcap/tidb/parser/terror"
    "github.com/pingcap/tidb/parser/types"
)

I don’t really know how to import these packages when GitHub is gone, and there are also issues with GOPATH that cause a lot of people to put all their Go projects under /home/User/go/ for development, which is very inexplicable.

WebP Server Go Upgrade

Since Go version 1.17 is EOL, we upgrade the build version of WebP Server Go. Since a Golang upgrade is generally a matter of changing an image, we upgrade the build image from.

1

FROM golang:1.17.4-alpine as builder

to

1

FROM golang:1.19.0-alpine as builder

It was very simple, and then very quickly, CI told us that the image build had failed.

https://github.com/webp-sh/webp_server_go/runs/7784058473?check_suite_focus=true

1
2
3
4

#14 [builder 6/6] RUN cd /build && sed -i "s|.\/pics|/opt/pics|g" config.json      && sed -i "s|""|"/opt/exhaust"|g" config.json      && sed -i 's/127.0.0.1/0.0.0.0/g' config.json      && go build -ldflags="-s -w" -o webp-server .
#14 0.395 error obtaining VCS status: exit status 128
#14 0.395   Use -buildvcs=false to disable VCS stamping.
#14 ERROR: process "/bin/sh -c cd /build && sed -i \"s|.\\/pics|${IMG_PATH}|g\" config.json      && sed -i \"s|\\\"\\\"|\\\"${EXHAUST_PATH}\\\"|g\" config.json      && sed -i 's/127.0.0.1/0.0.0.0/g' config.json      && go build -ldflags=\"-s -w\" -o webp-server ." did not complete successfully: exit code: 1

From the Release Note for Go 1.18 you will find this passage.

The go command now embeds version control information in binaries. It includes the currently checked-out revision, commit time, and a flag indicating whether edited or untracked files are present. Version control information is embedded if the go command is invoked in a directory within a Git, Mercurial, Fossil, or Bazaar repository, and the main package and its containing main module are in the same repository. This information may be omitted using the flag -buildvcs=false.

As you can see from Go 1.18 onwards, Go comes with Version control, which means that if you are using go build in a Git directory that has a broken .git directory, you will encounter the above problem, but why is there a broken .git directory in the build environment?

It started when someone added a .dockerignore file, and because of a lack of skill, our .dockerignore was initially written as .git/*, resulting in the directory itself not being Ignore’d. And it’s hard to see this problem, after all, no one will go to the Dockerfile when building the image and add ls -a to see what’s being passed in.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

#0 0.071 .                                                                                                                                                     
#0 0.071 ..                                                                                                                                                    
#0 0.071 .dockerignore                                                                                                                                         
#0 0.071 .git
#0 0.071 .github
#0 0.071 .gitignore
#0 0.071 .idea
#0 0.071 Dockerfile
#0 0.071 builds
#0 0.071 config.go
#0 0.071 config.json
#0 0.071 coverage.txt
#0 0.071 encoder.go
#0 0.071 encoder_test.go
#0 0.071 exhaust
#0 0.071 go.mod
#0 0.071 go.sum
#0 0.071 helper.go
#0 0.071 helper_test.go
#0 0.071 pics
#0 0.071 prefetch.go
#0 0.071 prefetch_test.go
#0 0.071 remote-raw
#0 0.071 router.go
#0 0.071 router_test.go
#0 0.071 scripts
#0 0.071 update.go
#0 0.071 update_test.go
#0 0.071 webp-server.go
#0 0.071 webp-server_test.go
#0 0.161 error obtaining VCS status: exit status 128
#0 0.161        Use -buildvcs=false to disable VCS stamping.
------

Since .git/* is written, there will be an empty .git directory in the build environment, and if you add a git status to the Dockerfile you can see that

1
2

 > [builder 6/7] RUN cd /build && git status:
#0 0.078 fatal: not a git repository (or any of the parent directories): .git

So here’s where the above error comes in.

Once we know what’s going on we can target the problem. If you are building your application in containers like we are, you can just add the following line to .dockerignore.

1

  .git

This lets Docker build without copying the .git directory directly to the build environment. Or you can add a line rm -rf .git to the Dockerfile. Of course, this is a bit weird, and if the local .git directory is too large COPY . will get very stuck.

If you’re building your app outside of a container and you’re having problems with your Git repository (whether it’s a permissions issue or whatever), add a -buildvcs=false to your build parameters and you’re good to go.

Epilogue

The problem we are having is made up of two issues.

  1. the .dockerignore file was not written correctly in the first place.
  2. Instead of reading the full release note when upgrading, we changed the image version for the build as a matter of course (of course the -buildvcs is also a bit strange to have enabled by default).

The bad thing is that it blocked our development for a while, but the good thing is that it helped us to find a problem that we had not noticed before, maybe this is the “industrial grade language”.

Ref

  • https://nova.moe/solve-go-buildvcs/