There is an amazing thing about Go in the file IO scenario. When opening a file, instead of an interface, it returns a pointer to an
This means that the concept of Go’s filesystem is directly related to the concept of the OS’s filesystem. You have to pass in a file path, and you have to actually go and open an OS file.
If you don’t use an interface, but instead strongly relate it to a specific type, it will lead to poor scalability. For example, if all the OS packages are used, then the operations are strongly bound to the OS file system.
Go’s designers have always been concerned about this, but there is nothing they can do about it. Because users are already using it, Go’s promise is to be forward-compatible, so directly modifying the original semantics and interface will not work.
What to do?
Go 1.16 gives us the answer. Go gives us
io.FS. Go’s intention is to make another layer of FS abstraction at the level of its own language, so that it can be decoupled from the OS’s FS.
io.FS can be any odd-shaped FS, as long as you implement the specified FS interface. The next step is to look at some of the core changes brought by Go 1.16.
Some people say that Go is already 1.19, why look at 1.16?
io/fs was introduced in Go 1.16. There is one major change in io.
What has changed about io in Go 1.16?
- A new package for
io/fswas added, abstracting an FS out.
embedpackage uses this abstraction.
- Regulates the contents of
Let’s look at them one by one.
Why does Go abstract FS?
As mentioned earlier, the concept of Go’s filesystem is directly related to the concept of OS’s filesystem. This creates an inconvenience for scalability. Most importantly, Go has identified a need for a different file system than OS, namely embed FS.
embed is a feature provided by Go to package files into a binary, which is also a file system-like requirement. But it is not a filesystem directly on the OS (the vfs thing).
So in Go 1.16 it came along. FS definition was introduced, and embed uses this layer of abstraction directly.
Go’s implementers are strong and recommend small interfaces. That is, minimalist, atomic interface semantics. You can see the strong power in the definition of io/fs.
This, in its simplest form, is FS. This is what the file system looks like in its simplest form, with just an Open method that returns a file.
In other words, Go understands the file system, as long as it can implement an Open method and return a File interface, the File only needs to implement the Stat, Read, Close methods.
Did you find that the OS FS already meets the conditions? So, Go’s FS can be OS’s FS, and naturally, it can be any other implementation.
Go extends the
io.FS interface to add filesystem functionality. For example, adding
ReadDir is a file system
ReadDirFS with read directories.
Add a Glob method and you have a file system with path wildcard queries.
Add a Stat and it becomes a path-searching filesystem.
The definition of these very classic file systems is already done by Go in
How to use io.FS
Our goal is to implement an FS for Go, which is already defined in
io.FS. We just need to write a structure that implements its methods, and then you can say it’s an FS.
There is a lot of room for imagination here, for example, it can be an OS FS, a memory FS, a hash FS, and so on. There are many examples on the web. But the standard library already has one of the best examples, and that is embed FS.
Let’s look at how
embed implements an embedded filesystem. embed is implemented in the embed/embed.go file, which is very concise.
First, a structure FS is defined in the embed package, which will be the concrete implementation of
The FS structure inside embed only needs to implement the Open method.
Open above, in the case of a file, returns an openFile structure as a concrete implementation of the
As above, you only need to implement the
Close methods. This is a complete, Go-level implementation of FS.
You can use the embed file system as follows.
In the above example, a hello.txt file in the current directory is packed into a binary file when it is compiled. You can read it out when the program starts.
Note: The variable f, the compiler will arrange to fill it. It has a value when the process is started.
Go 1.16 Other changes to IO
In addition to the io/fs and embed fs mentioned above, Go has made some more precise adjustments to the structure of the previous io to classify it. The stuff in the previous hodgepodge of io/ioutil has been split out. It has been moved to the corresponding io and os packages. For compatibility, the ioutil package was not removed directly, but imported. For example.
- Discard is moved to the io library implementation
- ReadAll has been moved to the io library implementation.
- NopCloser has been moved to the io library implementation.
- ReadFile moved to os library implementation
- WriteFile moved to os library implementation
Basically ioutil package is hollowed out. go 1.16 is not deleted yet just for compatibility.
io.FSinterface is intended to be decoupled from the OS’s FS. This gives the programmer more room for imagination.
- embed FS has a typical FS interface, but it is not located directly on the OS file system. So it is ideal for the first practice with io.FS.
- FS to manage the files, so that it can be decoupled from the OS and easy to do unit testing.
- ioutil can be used sparingly, as its functionality has been moved to a more explicit package implementation.