The Go standard library provides convenient methods to run external commands easily. Generally we use the methods under the os/exec package to execute external commands and interact with external commands.
os/exec wraps the os.StartProcess method for easier access to input and output, providing I/O pipe and other features. I will dedicate two articles to Go’s methods for starting new processes and executing external commands, this is the first one, dedicated to the
os/exec library provides methods similar to those defined in the
POSIX standard for the C language, but provides further encapsulation to make it easier to use. I will introduce you to each of them next.
Run a command
The simplest way to run an external command is to call the Command method, you need to pass in the program to be executed and parameters, it will return a
*Cmd data structure, representing an external command to be executed, the main call to its
CombinedOutput method after this object can not be reused, and generally we will not reuse this object, but generate a new one when needed.
The following code is to execute the
ls command and pass it the parameter
If you execute this command, there is no output in the console, in fact the command has been executed, only our Go program does not capture and process the output, so there is no output in the console.
Run method will execute the external command and wait for the command to finish, if the command is executed normally without errors and the return code is 0, then Run returns err == nil, then it returns an
*ExitError, sometimes you need to read cmd.
If you don’t want to wait, then you can call the Start method. If Start succeeds, the Process and
ProcessState fields will be set. You can check
ProcessState.Exited() to determine if the program has exited. If you want to block again and wait for the program to finish, you can call the Wait method.
In fact, the
Run method is implemented using the
Display the output of external commands
The Cmd command contains input and output fields that you can set to enable customization of input and output:
Stdin is null, then the process will read from
Stdin is an
*os.File object, then it will read from this file.
os.Stdin, then it will read from standard input like command line.
Stderr represent the standard output and error output of the external program process, and if they are null, then they are output to the null device.
Stderr are *os.File objects, then the data is output to a file.
Stdout and Stderr are set to os.Stdout and os.Stderr respectively, it will output to the command line.
Let’s adapt the previous example to show the command output.
By default the working path of the process is the folder where the process is called, but you can also specify it manually, for example we specify the working path as the root path.
External program path
Cmd.Path is the path of the program to be executed, if it is a relative path, then it calculates the relative path based on
Cmd.Dir. If the program is already in the system $PATH path, then you can write the program name directly.
Set environment variables
Cmd also has a field called
Env, which is used to set the environment variable of the process in the format
Env is empty, then the new process will use the environment variables of the caller process.
For example, in the following example, we set the
myvar variable, you can comment out the line
cmd.Enc = ... that line to compare the results.
The underlying Process and ProcessState
As I mentioned above,
os/exec is a wrapped convenience library, and at the bottom it is implemented using
os.StartProcess, so you can get the underlying Process object and ProcessState object, which represent the process and the process state respectively
Determine if an external command exists
Sometimes you need to check if an external command exists when you execute it, you can use the
If the incoming argument contains a path separator, then it will look for the program based on the relative or absolute path of
Cmd.Dir. If it does not contain a path separator, then it will look for the file from the
PATH environment variable
Get command results
Cmd provides the
Output() method to get the bytes of the command execution result if the command is executed correctly:
If there is an error with the command, the error message can be obtained with
Combining Stdout and Stderr
If you want to have one way to get the output regardless of errors, you can call the
CombinedOutput() method, which will return either normal or error output, and a second return err to indicate whether an error was executed
The implementation of the
CombinedOutput method is also very simple, in fact it is implemented by sharing the same
Read Stdout and Stderr separately
Once we understand the implementation of
CombinedOutput, we can set
Stderr respectively, to achieve independent reads.
Show command execution progress
Since we have been able to use our own
io.Writer to set
Stdout/Stderr, we can do something richer.
For example, if we use the
curl command to download a large file, we can display the size of the downloaded data in real time:
While the previous examples demonstrated handling
Output, this next example shows how to set up
wc command reads the
main.go file and counts how many lines it has in total.
You can use the output of one command as input to the next command, and so on, to string multiple commands into a pipe.
StdoutPipe methods to get the pipe object.
For example, the following command takes the output of
cat main.go as input to the
wc -l command:
First, the pipeline is created, and then the Start method and Wait method of each command are called in turn.
Generic Pipe method
Here is a more generic way to create a Cmd pipeline.
If you execute it via the bash command, you can use the bash pipe feature, which is simpler to write.
When the parent process ends before the child process does, then the child process is called an orphan process and the ppid of the child process is set to 1 at this time.
When the main program exits, this curl sub-process will keep downloading and its ppid is set to 1. You can use ps to view the process information, for example in a Mac.
Kill child processes when the program exits
If we want to kill the child process started by the program when it exits, then a stupid way is to get the Process object of the child process and call its Kill method to kill it. But unfortunately it can’t kill the Sun process.
For Linux systems, you can kill the grandchild process as well with the following settings.
Or more generally, use the following settings:
Pass the file opened by the parent process to the child process
In addition to the standard input and output of 0,1,2 files, you can also pass files from the parent process to the child process via the
One of the more common scenarios is graceful restart, where the new process inherits the
net.Listener that the old process is listening to, so that the network connection does not need to be closed and reopened.
For example, one of the earliest articles introducing the go graceful restart technique, Graceful Restart in Golang, has the following code