go Command Line Parses flag Package View Command Source via Subcommand

Posted by kevinc on Sun, 08 Dec 2019 03:30:21 +0100

Last article Describes how to extend a new type of support in flag.This article describes how to use flag to implement subcommands. Overall, this is the core of this series. The first two articles are just padding.

The first two articles are linked as follows:

A quick way to parse flag packages from the Go command line
New Extended Type for Go Command Line Resolution flag Package

Hopefully, after reading this article, if you read the implementation source of the go command, you won't lose your way, at least in the overall structure.

FlagSet

Before formally introducing the implementation of subcommands, learn about a type in the flag package, FlagSet, which represents a command.

From the components of a command, a command is made up of command name, option Flag, and parameters.Similar to the following:

$ cmd --flag1 --flag2 -f=flag3 arg1 arg2 arg3

The definition of FlagSet also fits this point, as follows:

type FlagSet struct {
    // Help information for printing commands
    Usage func()

    // Command Name
    name          string
    parsed        bool
    // Actual incoming Flag
    actual        map[string]*Flag
    // Flag to be used, added to the form via Flag.Var()
    formal        map[string]*Flag

    // Parameter, Parse parses the []string passed in from the command line,
    // The first one that does not satisfy the Flag rule (such as not-or--at the beginning),
    // From this location, after all
    args          []string // arguments after flags
    // There are three options for how errors are handled:
    // ContinueOnError Continue
    // ExitOnError Exit
    // PanicOnError     panic
    errorHandling ErrorHandling
    output        io.Writer // nil means stderr; use out() accessor
}

The containing field has the command name, the option Flag has the form and actual, and the parameter args.

If someone says that FlagSet is the core of the command line implementation, it's more acceptable.It has not been mentioned before, mainly because flag packages are further encapsulated on FlagSet to simplify the processing of the command line and can be used simply without regard to its existence.

flag defines a global FlagSet type variable, CommandLine, that represents the entire command line.It can be said that CommandLine is a special case of FlagSet and has a fixed usage pattern, so a set of default functions can be provided on it.

Some of the functions you've used before, such as the following.

func BoolVar(p *bool, name string, value bool, usage string) {
    CommandLine.Var(newBoolValue(value, p), name, usage)
}

func Bool(name string, value bool, usage string) *bool {
    return CommandLine.Bool(name, value, usage)
}

func Parse() {
    // Ignore errors; CommandLine is set for ExitOnError.
    CommandLine.Parse(os.Args[1:])
}

More, not all of them are listed here.

Next, let's take off this layer and comb through the entire processing of the command line.

Process Interpretation

The whole process of using CommandLine consists of three main parts: getting the command name, defining the actual options in the command, and parsing options.

The command name was specified when the CommandLine was created, as follows:

CommandLine = NewFlagSet(os.Args[0], ExitOnError)

The name is specified by os.Args[0], the first parameter on the command line.In addition to the command name, ExitOnError specifies how to handle errors.

Next comes the definition of the Flag that will actually be used in the command.

The core code is FlagSet.Var(), as follows:

func (f *FlagSet) Var(value Value, name string, usage string) {
    // Remember the default value as a string; it won't change.
    flag := &Flag{name, usage, value, value.String()}

    // ...
    // Omit some code
    // ...

    if f.formal == nil {
        f.formal = make(map[string]*Flag)
    }
    f.formal[name] = flag
}

Previously used flag.BoolVar and flag.Bool both save Flag to FlagSet.formal through CommandLine.Var(), FlagSet.Var(), so that the value can be successfully set to the defined variable when parsing later.

The last step is to parse the option Flag from the command line.Since CommandLine represents the entire command line, its options and parameters must be resolved from os.Args[1:].

The code for flag.Parse is as follows:

func Parse() {
    // Ignore errors; CommandLine is set for ExitOnError.
    CommandLine.Parse(os.Args[1:])
}

The main point now is to understand the parsing rules for options and parameters in flag, such as gvg-v list, by which rules do -v be a Flag and list a parameter?

If you continue to chase down Parse's source code, Flag's parsing rules will be found in FlagSet.parseOne.

func (f *FlagSet) ParseOne()
    if len(f.args) == 0 {
        return false, nil
    }
    s := f.args[0]
    if len(s) < 2 || s[0] != '-' {
        return false, nil
    }
    numMinuses := 1
    if s[1] == '-' {
        numMinuses++
        if len(s) == 2 { // "--" terminates the flags
            f.args = f.args[1:]
            return false, nil
        }
    }
    // ...
}

Parsing Flag is terminated in three cases when the command line argument is completely parsed, that is, len(f.args) == 0, or less than 2 in length, but the first character is not -, or the parameter length is equal to 2, and the second character is -.Subsequent content continues to be treated as command line parameters.

Without subcommands, the command parsing is almost complete, and business code development follows.What if CommandLine has subcommands?

Subcommand

There is little difference between subcommands and CommandLine in form or logic.Formally, subcommands also contain options and parameters, and logically, they have the same rules of resolution as CommandLine for options and parameters.

A command line containing subcommands in the following form:

$ cmd --flag1 --flag2 subcmd --subflag1 --subflag2 arg1 arg2

As you can see from the above, if CommandLine contains subcommands, you can understand that there are no parameters in it, because the first parameter of CommandLine is the name of the subcommand, and the subsequent parameters are resolved to the option parameters of the subcommand.

Now the implementation of the subcommand is very simple. Create a new FlagSet and reprocess the parameters in the CommandLine as described earlier.

First, get CommandLine.Arg(0) and check to see if the corresponding subcommand exists.

func main() {
    flag.Parse()
    if h {
        flag.Usage()
        return
    }

    cmdName := flag.Arg(0)
    switch cmdName {
    case "list":
        _ = list.Exec(cmdName, flag.Args()[1:])
    case "install":
        _ = install.Exec(cmdName, flag.Args()[1:])
    }
}

The implementation of the subcommand is defined in another package, taking the list command as an example.The code is as follows:

var flagSet *flag.FlagSet

var origin string

func init() {
    flagSet = flag.NewFlagSet("list", flag.ExitOnError)
    val := newStringEnumValue("installed", &origin, []string{"installed", "local", "remote"})
    flagSet.Var(
        val, "origin",
        "the origin of version information, such as installed, local, remote",
    )
}

In the code above, a FlagSet for the list subcommand is defined, and an option Flag, origin is added to the Init method.

Run functions are code that actually executes business logic.

func Run(args []string) error {
    if err := flagSet.Parse(args); err != nil {
        return err
    }

    fmt.Println("list --oriign", origin)
    return nil
}

The final Exec function combines the Init and Run functions and is provided for the main call.

func Run(name string, args []string) error {
    Init(name)
    if err := Run(args); err != nil {
        return err
    }

    return nil
}

The parsing of the command line is complete, and if subcommands have subcommands, the logic of processing remains the same.Now you can start writing business code in Run functions.

Go Command

Now read the implementation code for the Go command.

Since the code written by the big guys is handcrafted based on flag packages, it's a bit less readable without any framework.

Source at go/src/cmd/go/cmd/main.go Next, all the commands supported by Go are initialized with the base.Go variable, as follows:

base.Go.Commands = []*base.Command{
    bug.CmdBug,
    work.CmdBuild,
    clean.CmdClean,
    doc.CmdDoc,
    envcmd.CmdEnv,
    fix.CmdFix,
    fmtcmd.CmdFmt,
    generate.CmdGenerate,
    modget.CmdGet,
    work.CmdInstall,
    list.CmdList,
    modcmd.CmdMod,
    run.CmdRun,
    test.CmdTest,
    tool.CmdTool,
    version.CmdVersion,
    vet.CmdVet,

    help.HelpBuildmode,
    help.HelpC,
    help.HelpCache,
    help.HelpEnvironment,
    help.HelpFileType,
    modload.HelpGoMod,
    help.HelpGopath,
    get.HelpGopathGet,
    modfetch.HelpGoproxy,
    help.HelpImportPath,
    modload.HelpModules,
    modget.HelpModuleGet,
    modfetch.HelpModuleAuth,
    modfetch.HelpModulePrivate,
    help.HelpPackages,
    test.HelpTestflag,
    test.HelpTestfunc,
}

Both the go command and its subcommands are of type *base.Command.Take a look at the definition of *base.Command.

type Command struct {
    Run func(cmd *Command, args []string)

    UsageLine string
    Short string
    Long string

    Flag flag.FlagSet

    CustomFlags bool

    Commands []*Command
}

There are three main fields: Run, which handles business logic, FlagSet, command line parsing, and []*Command, which supports subcommands.

Let's look at the core logic in the main function.The following:

BigCmdLoop:
for bigCmd := base.Go; ; {
    for _, cmd := range bigCmd.Commands {
        // ...
        // Main Logic Code
        // ...
    }

    // print the help information
    helpArg := ""
    if i := strings.LastIndex(cfg.CmdName, " "); i >= 0 {
        helpArg = " " + cfg.CmdName[:i]
    }
    fmt.Fprintf(os.Stderr, "go %s: unknown command\nRun 'go help%s' for usage.\n", cfg.CmdName, helpArg)
    base.SetExitStatus(2)
    base.Exit()
}

Starting with the top-level base.Go, iterate through all subcommands of Go, and print help information if no corresponding command is available.

The main logic code omitted is as follows:

for _, cmd := range bigCmd.Commands {
    // If no command is found, continue with the next cycle
    if cmd.Name() != args[0] {
        continue
    }
    // Check for subcommands
    if len(cmd.Commands) > 0 {
        // Set bigCmd to current command
        // For example, go tool compile, cmd is compile
        bigCmd = cmd
        args = args[1:]
        // If there are no command parameters, the command rules are not met and help information is printed.
        if len(args) == 0 {
            help.PrintUsage(os.Stderr, bigCmd)
            base.SetExitStatus(2)
            base.Exit()
        }
        // Print help information for this command if the command name is help
        if args[0] == "help" {
            // Accept 'go mod help' and 'go mod help foo' for 'go help mod' and 'go help mod foo'.
            help.Help(os.Stdout, append(strings.Split(cfg.CmdName, " "), args[1:]...))
            return
        }
        // Continue processing subcommands
        cfg.CmdName += " " + args[0]
        continue BigCmdLoop
    }
    if !cmd.Runnable() {
        continue
    }
    cmd.Flag.Usage = func() { cmd.Usage() }
    if cmd.CustomFlags {
        // Parse Parameters and Options Flag
        // Custom Processing Rules
        args = args[1:]
    } else {
        // Processing through methods provided by FlagSet
        base.SetFromGOFLAGS(cmd.Flag)
        cmd.Flag.Parse(args[1:])
        args = cmd.Flag.Args()
    }

    // Execute business logic
    cmd.Run(cmd, args)
    base.Exit()
    return
}

There are several main parts, namely, finding commands, checking for subcommands, parsing options and parameters, and finally executing commands.

Use cmd.Name()!= args[0] to determine if a command was found and continue executing down if found.

Check the existence of subcommands by len(cmd.Commands), the presence of a bigCmd override, and the compliance of the command line, such as checking that len(args[1:]) is 0, indicating that the incoming command line does not provide subcommands.If everything is ready, proceed to the next loop through continue to execute the processing of the subcommands.

Next comes the parsing of command options and parameters.Processing rules can be customized or handled directly using FlagSet.Parse.

Finally, cmd.Run is called to perform logical processing.

summary

This paper introduces how to implement subcommands in Go by flag. From the structure of FlagSet, the processing logic of FlagSet is combed through the CommandLine provided by default in the flag package.Basically, the related functions of subcommands are implemented.

Finally, this paper analyses how go in Go source code is implemented using flag.It's slightly more difficult to read because it's bare writing using flag packages only.This article is only an introduction, at least to help you not get lost in the big direction, there are more details to dig.

Topics: Go less