Go Reflect Performance

Go reflect package provides the ability to get the type and value of an object at runtime, which can help us to abstract and simplify the code, achieve dynamic data acquisition and method invocation, improve development efficiency and readability, and make up for Go’s ability to handle data uniformly in the absence of generics.

With reflect, we can achieve the ability to get object types, object fields, object methods, get tag information of struct, dynamically create objects, whether objects implement specific interfaces, convert objects, get and set object values, call Select branches dynamically, etc. It looks good, but we all know one thing: there is a performance cost to using reflect!

Test

The use of reflect in Java also has an impact on performance, but unlike Java reflect, Java does not distinguish between Type and Value types, so at least in Java we can pre-cache the corresponding reflect objects to reduce the impact of reflection on performance, but there is no way to pre-cache reflect in Go, because the Type type does not contain the runtime value of the object, you must go through ValueOf and runtime instance objects to get the Value object.

Reflection generation and acquisition of objects will add additional code instructions, and will also involve interface{} boxing/unboxing operations, and may increase the generation of temporary objects in the middle, so performance degradation is certain, but how much can be reduced, or the data to speak.

Of course, the different methods used by reflect, and the different types of objects, will more or less affect the performance of the test data, we will take a common struct type as an example.

package test
import (
	"reflect"
	"testing"
)
type Student struct {
	Name  string
	Age   int
	Class string
	Score int
}
func BenchmarkReflect_New(b *testing.B) {
	var s *Student
	sv := reflect.TypeOf(Student{})
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		sn := reflect.New(sv)
		s, _ = sn.Interface().(*Student)
	}
	_ = s
}
func BenchmarkDirect_New(b *testing.B) {
	var s *Student
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		s = new(Student)
	}
	_ = s
}
func BenchmarkReflect_Set(b *testing.B) {
	var s *Student
	sv := reflect.TypeOf(Student{})
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		sn := reflect.New(sv)
		s = sn.Interface().(*Student)
		s.Name = "Jerry"
		s.Age = 18
		s.Class = "20005"
		s.Score = 100
	}
}
func BenchmarkReflect_SetFieldByName(b *testing.B) {
	sv := reflect.TypeOf(Student{})
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		sn := reflect.New(sv).Elem()
		sn.FieldByName("Name").SetString("Jerry")
		sn.FieldByName("Age").SetInt(18)
		sn.FieldByName("Class").SetString("20005")
		sn.FieldByName("Score").SetInt(100)
	}
}
func BenchmarkReflect_SetFieldByIndex(b *testing.B) {
	sv := reflect.TypeOf(Student{})
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		sn := reflect.New(sv).Elem() 
		sn.Field(0).SetString("Jerry")
		sn.Field(1).SetInt(18)
		sn.Field(2).SetString("20005")
		sn.Field(3).SetInt(100)
	}
}
func BenchmarkDirect_Set(b *testing.B) {
	var s *Student
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		s = new(Student)
		s.Name = "Jerry"
		s.Age = 18
		s.Class = "20005"
		s.Score = 100
	}
}

Test results:

BenchmarkReflect_New-4               	20000000	        70.0 ns/op	      48 B/op	       1 allocs/op
BenchmarkDirect_New-4                	30000000	        45.6 ns/op	      48 B/op	       1 allocs/op
BenchmarkReflect_Set-4               	20000000	        73.6 ns/op	      48 B/op	       1 allocs/op
BenchmarkReflect_SetFieldByName-4    	 3000000	       492 ns/op	      80 B/op	       5 allocs/op
BenchmarkReflect_SetFieldByIndex-4   	20000000	       111 ns/op	      48 B/op	       1 allocs/op
BenchmarkDirect_Set-4                   30000000	        43.1 ns/op	      48 B/op	       1 allocs/op

Test results

We conducted tests for two functions.

Creation of objects (struct)
Assignment of object fields

For object creation, it takes 70 nanoseconds to generate the object through reflection, while it takes 45.6 nanoseconds to directly new the object, which is a big performance difference.

For the field assignment, there are four test cases.

Reflect_Set: Generate an object through reflection, convert it to an actual object, and call the object’s fields directly for assignment, takes 73.6 nanoseconds
Reflect_SetFieldByName: Generate the object by reflection and assign it by FieldByName, takes 492 nanoseconds
Reflect_SetFieldByIndex: Generates an object by reflection and assigns it by Field, takes 111 nanoseconds
Direct_Set: Assignment by calling the field of the object directly, in 43.1 nanoseconds

The main difference between the performance of Reflect_Set and Direct_Set is still in the generation of the object, because the assignment methods for the fields are the same afterwards, which is also consistent with the results of the test case for object creation.

If the assignment is done by reflection, the performance degradation is very strong and the time consumption increases exponentially. Interestingly, the FieldByName method is several times more powerful than the Field method, because FieldByName has an extra loop to find the field, although it still ends up calling Field for the assignment.

unc (v Value) FieldByName(name string) Value {
	v.mustBe(Struct)
	if f, ok := v.typ.FieldByName(name); ok {
		return v.FieldByIndex(f.Index)
	}
	return Value{}
}
func (v Value) FieldByIndex(index []int) Value {
	if len(index) == 1 {
		return v.Field(index[0])
	}
	v.mustBe(Struct)
	for i, x := range index {
		if i > 0 {
			if v.Kind() == Ptr && v.typ.Elem().Kind() == Struct {
				if v.IsNil() {
					panic("reflect: indirection through nil pointer to embedded struct")
				}
				v = v.Elem()
			}
		}
		v = v.Field(x)
	}
	return v
}

Optimization

From the results of the above tests, reflection can affect the performance of object generation and field assignment, but it does simplify the code and provide a unified code for business logic, such as the standard library json coding and decoding, rpc service registration and invocation, some ORM frameworks such as gorm, etc., are used to process data through reflection, which is to be able to handle common types.

// https://github.com/golang/go/blob/master/src/encoding/json/decode.go#L946
  ......
      case reflect.String:
	v.SetString(string(s))
case reflect.Interface:
	if v.NumMethod() == 0 {
		v.Set(reflect.ValueOf(string(s)))
	} else {
		d.saveError(&UnmarshalTypeError{Value: "string", Type: v.Type(), Offset: int64(d.readIndex())})
          }
  ......

// https://github.com/jinzhu/gorm/blob/master/scope.go#L495
     for fieldIndex, field := range selectFields {
if field.DBName == column {
	if field.Field.Kind() == reflect.Ptr {
		values[index] = field.Field.Addr().Interface()
	} else {
		reflectValue := reflect.New(reflect.PtrTo(field.Struct.Type))
		reflectValue.Elem().Set(field.Field.Addr())
		values[index] = reflectValue.Interface()
		resetFields[index] = field
	}
	selectedColumnsMap[column] = offset + fieldIndex
	if field.IsNormal {
		break
	}
}
     }

In our pursuit of high performance scenarios, we may need to avoid reflection calls as much as possible, such as unmarshal for json data, easyjson avoids using reflection by means of generators.

func (v *Student) UnmarshalJSON(data []byte) error {
	r := jlexer.Lexer{Data: data}
	easyjson4a74e62dDecodeGitABCReflect(&r, v)
	return r.Error()
}
func (v *Student) UnmarshalEasyJSON(l *jlexer.Lexer) {
	easyjson4a74e62dDecodeGitABCReflect(l, v)
}
func easyjson4a74e62dDecodeGitABCReflect(in *jlexer.Lexer, out *Student) {
	isTopLevel := in.IsStart()
	if in.IsNull() {
		if isTopLevel {
			in.Consumed()
		}
		in.Skip()
		return
	}
	in.Delim('{')
	for !in.IsDelim('}') {
		key := in.UnsafeString()
		in.WantColon()
		if in.IsNull() {
			in.Skip()
			in.WantComma()
			continue
		}
		switch key {
		case "Name":
			out.Name = string(in.String())
		case "Age":
			out.Age = int(in.Int())
		case "Class":
			out.Class = string(in.String())
		case "Score":
			out.Score = int(in.Int())
		default:
			in.SkipRecursive()
		}
		in.WantComma()
	}
	in.Delim('}')
	if isTopLevel {
		in.Consumed()
	}
}

Some other codec libraries also provide this method of avoiding the use of reflection to improve performance.

Performance impact of “crate/uncrate” operations

func DirectInvoke(s *Student) {
	s.Name = "Jerry"
	s.Age = 18
	s.Class = "20005"
	s.Score = 100
}
func InterfaceInvoke(i interface{}) {
	s := i.(*Student)
	s.Name = "Jerry"
	s.Age = 18
	s.Class = "20005"
	s.Score = 100
}
func BenchmarkDirectInvoke(b *testing.B) {
	s := new(Student)
	for i := 0; i < b.N; i++ {
		DirectInvoke(s)
	}
	_ = s
}
func BenchmarkInterfaceInvoke(b *testing.B) {
	s := new(Student)
	for i := 0; i < b.N; i++ {
		InterfaceInvoke(s)
	}
	_ = s
}

Test results:

1
2

BenchmarkDirectInvoke-4              	300000000	         5.60 ns/op	       0 B/op	       0 allocs/op
BenchmarkInterfaceInvoke-4           	200000000	         6.64 ns/op

You can see that converting concrete objects to interface{} (and the reverse operation) does have a small performance impact, but it doesn’t seem to be a big one.

Reference https://colobu.com/2019/01/29/go-reflect-performance/

Table of Contents

Test

Test results

Optimization

Performance impact of “crate/uncrate” operations