In the process of writing code, sometimes we need to do some bulk queries/operations, which often involve chunking a large array or slice.

For example, if we have an array of ids and we want to request an interface to query information based on the ids, the interface supports batch queries, but the maximum number of queries per query is 100. the best practice is to take up to 100 ids from the array each time and do a batch query until the array is traversed.

This operation is not complicated and can be achieved simply with a loop, but every time you encounter this scenario you need to write the code once, a little write spit. So I want to write a function that splits []T into [][]T as needed.

But go’s generics are not here yet, so I have to use reflection to do it.

Results

Pass in []T , T can be of any type, chunked by the specified size, and returns [][]T.

The example divides [0,1,2,3,4,5,6,7,8,9] into [0,1,2], [3,4,5], [6,7,8], [9]

https://github.com/kirito41dd/xslice

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

package main

import (
	"fmt"
	"github.com/kirito41dd/xslice"
)

func main() {
	s := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
	i := xslice.SplitToChunks(s, 3)
	ss := i.([][]int)
	fmt.Println(ss) // [[0 1 2] [3 4 5] [6 7 8] [9]]
}

Code implementation

Using reflection naturally requires the use of: interface{}

Welcome to use: github.com/kirito41dd/xslice, and will continue to expand its functionality if we encounter other scenarios in the future.

Still looking forward to the early arrival of go generic

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

func SplitToChunks(slice interface{}, chunkSize int) interface{} {
	sliceType := reflect.TypeOf(slice)
	sliceVal := reflect.ValueOf(slice)
	length := sliceVal.Len()
	if sliceType.Kind() != reflect.Slice {
		panic("parameter must be []T")
	}
	n := 0
	if length%chunkSize > 0 {
		n = 1
	}
	SST := reflect.MakeSlice(reflect.SliceOf(sliceType), 0, length/chunkSize+n)
	st, ed := 0, 0
	for st < length {
		ed = st + chunkSize
		if ed > length {
			ed = length
		}
		SST = reflect.Append(SST, sliceVal.Slice(st, ed))
		st = ed
	}
	return SST.Interface()
}