String concatenation is indispensable in everyday coding, and the most commonly used is the native concatenation method (+=). However, its performance is fine for a small number of splices, but for a large number of string splices, other more efficient methods should be used.

This article first lists several string concatenation methods commonly used in Golang, and then benchmarks them, so that after reading this article, we can have a basic understanding of the applicability of various concatenation methods.

1 How many ways are there for string concatenation?

We are inevitably asked in Golang usage, “How many ways are there to splice strings?” . Here is a list of them.

a) Native concatenation (+=)

The native concatenation method uses the + operator to splice two strings directly.

The following code uses += to splice and reassign strings.

1
2

var s string
s += "hello"

Why is this approach not efficient? Because string is immutable in Golang, the value of s has to be taken down (copied from the top), then spliced with a string, and then the new value (a brand new string) is assigned to s again after the calculation, while the old value of s will wait for the garbage collector. Since each splice is iteratively copied from the beginning, it involves more computation and memory allocation.

The time complexity of this approach is O(N^2).

b) bytes.Buffer

Buffer is a variable-length byte buffer. It uses slice internally to store bytes (buf []byte).

1
2

buf := bytes.NewBufferString("hello")
buf.WriteString(" world") // fmt.Fprint(buf, " world")

When using WriteString for string concatenation, it dynamically extends the slice length according to the situation and copies the string to be spliced into the buffer using the built-in slice memory copy function. Since it is a variable-length slice, each time you splice, you do not need to re-copy the old part, but only append the part to be spliced to the end, so it has higher performance than the native concatenation method.

The time complexity of this method is O(N).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

// WriteString appends the contents of s to the buffer, growing the buffer as
// needed. The return value n is the length of s; err is always nil. If the
// buffer becomes too large, WriteString will panic with ErrTooLarge.
func (b *Buffer) WriteString(s string) (n int, err error) {
    b.lastRead = opInvalid
    m, ok := b.tryGrowByReslice(len(s))
    if !ok {
        m = b.grow(len(s))
    }
    return copy(b.buf[m:], s), nil
}

c) strings.Builder

The strings.Builder also uses byte slice internally for storage.

1
2

var builder strings.Builder
builder.WriteString("hello") // fmt.Fprint(&builder, "hello")

When using WriteString for string stitching, the built-in append function is called and only the string to be stitched is merged into the buffer. It is also very efficient.

1
2
3
4
5
6
7

// WriteString appends the contents of s to b's buffer.
// It returns the length of s and a nil error.
func (b *Builder) WriteString(s string) (int, error) {
    b.copyCheck()
    b.buf = append(b.buf, s...)
    return len(s), nil
}

d) Built-in copy function

The built-in copy function supports copying a source slice to a target slice, and since the underlying representation of a string is []byte, you can also use this function for string concatenation. However, the limitation is that the length of the byte slice needs to be known in advance.

1
2
3
4

bytes := make([]byte, 11)
size := copy(bytes[0:], "hello")
copy(bytes[size:], " world")
fmt.Println(string(bytes))

The built-in copy function supports copying from one slice to another (it supports copying a string to []byte), and the return value is the length of the copied element.

The return value is the length of the element copied. Each time it splices, it only needs to append the string to be spliced to the end of the slice, which is also very efficient.

1
2
3
4
5
6

// The copy built-in function copies elements from a source slice into a
// destination slice. (As a special case, it also will copy bytes from a
// string to a slice of bytes.) The source and destination may overlap. Copy
// returns the number of elements copied, which will be the minimum of
// len(src) and len(dst).
func copy(dst, src []Type) int

e) strings.Join

If you want to put the parts of a string slice ([]string) into a single string, you can use strings.Join to do so.

1

s := strings.Join([]string{"hello world"}, "")

Its internal implementation is also done using bytes. So it is also very efficient.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

// Join concatenates the elements of its first argument to create a single string. The separator
// string sep is placed between elements in the resulting string.
func Join(elems []string, sep string) string {
    ...
    var b Builder
    b.Grow(n)
    b.WriteString(elems[0])
    for _, s := range elems[1:] {
        b.WriteString(sep)
        b.WriteString(s)
    }
    return b.String()
}

2 Benchmarking

The string concatenation methods described above are assembled into a test file, string_test.go, for benchmarking. (Because strings.Join requires a pre-generated []string, it is not used in the same scenario as the other methods, so it is not included in this test)

The benchmark test will use each method to splice a string s 1000 times.

string_test.go Source code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

package string_test

import (
    "bytes"
    "strings"
    "testing"
)

var (
    concatSteps = 1000
    subStr      = "s"
    expectedStr = strings.Repeat(subStr, concatSteps)
)

func BenchmarkConcat(b *testing.B) {
    for n := 0; n < b.N; n++ {
        var s string
        for i := 0; i < concatSteps; i++ {
            s += subStr
        }
        if s != expectedStr {
            b.Errorf("unexpected result, got: %s, want: %s", s, expectedStr)
        }
    }
}

func BenchmarkBuffer(b *testing.B) {
    for n := 0; n < b.N; n++ {
        var buffer bytes.Buffer
        for i := 0; i < concatSteps; i++ {
            buffer.WriteString(subStr)
        }
        if buffer.String() != expectedStr {
            b.Errorf("unexpected result, got: %s, want: %s", buffer.String(), expectedStr)
        }
    }
}

func BenchmarkBuilder(b *testing.B) {
    for n := 0; n < b.N; n++ {
        var builder strings.Builder
        for i := 0; i < concatSteps; i++ {
            builder.WriteString(subStr)
        }
        if builder.String() != expectedStr {
            b.Errorf("unexcepted result, got: %s, want: %s", builder.String(), expectedStr)
        }
    }
}

func BenchmarkCopy(b *testing.B) {
    for n := 0; n < b.N; n++ {
        bytes := make([]byte, len(subStr)*concatSteps)
        c := 0
        for i := 0; i < concatSteps; i++ {
            c += copy(bytes[c:], subStr)
        }
        if string(bytes) != expectedStr {
            b.Errorf("unexpected result, got: %s, want: %s", string(bytes), expectedStr)
        }
    }
}

Execute the Benchmark test command.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

$ go test -benchmem -bench .

goos: darwin
goarch: amd64
pkg: github.com/olzhy/test
cpu: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
BenchmarkConcat-4           7750            148143 ns/op          530274 B/op        999 allocs/op
BenchmarkBuffer-4         161848              7151 ns/op            3248 B/op          6 allocs/op
BenchmarkBuilder-4        212043              5406 ns/op            2040 B/op          8 allocs/op
BenchmarkCopy-4           281827              4208 ns/op            1024 B/op          1 allocs/op
PASS
ok      github.com/olzhy/test   5.773s

You can see that the built-in copy function with strings.Builder is the most efficient way, bytes.Buffer is the next most inefficient, and the native concatenation way is the most inefficient.