abstract

stay Last article In, we talked about how to implement an Http server in Golang.But in the end, we can find that although DefaultServeMux can do routing distribution, its function is also imperfect.

Routing distribution by DefaultServeMux is not a RESTful style API, and we cannot define the method required for requests or add query parameters to the API path.Second, we also want to make routing more efficient.

So in this article, we'll analyze the httprouter package to see how it achieves the functions we mentioned above at the source level.Also, for the most important prefix tree in this package, this article will explain it in a graphical way.

1 Use

We also start with how to use it and look at httprouter from the top down.Let's start with a small example from the official documentation:

package main

import (
    "fmt"
    "net/http"
    "log"

    "github.com/julienschmidt/httprouter"
)

func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    fmt.Fprint(w, "Welcome!\n")
}

func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}

func main() {
    router := httprouter.New()
    router.GET("/", Index)
    router.GET("/hello/:name", Hello)

    log.Fatal(http.ListenAndServe(":8080", router))
}

In fact, we can see that this is similar to using the net/http package that comes with Golang.They all register URI s and functions first, in other words, to match routes to processors.

When registering, use router.XXX method to register corresponding methods, such as GET, POST, etc.

After registration, use http.ListenAndServe to start listening.

As to why, we'll cover it in more detail in later chapters, so for now, you just need to know what to do first.

2 Create

Let's start with the first line of code, where we define and declare a Router.Let's look at the structure of this Router, omitting other properties that are not relevant to this article:

type Router struct {
	//This is the prefix tree, which records the corresponding route
	trees map[string]*node
	
	//Maximum number of parameters recorded
	maxParams  uint16

}

After creating this Router structure, we used the router.XXX method to register routes.Continue to see how routes are registered:

func (r *Router) GET(path string, handle Handle) {
	r.Handle(http.MethodGet, path, handle)
}

func (r *Router) POST(path string, handle Handle) {
	r.Handle(http.MethodPost, path, handle)
}

...

There is also a long list of methods here, all of which are the same, called

r.Handle(http.MethodPost, path, handle)

This method.Let's have a look:

func (r *Router) Handle(method, path string, handle Handle) {
	...
	if r.trees == nil {
		r.trees = make(map[string]*node)
	}

	root := r.trees[method]
	if root == nil {
		root = new(node)
		r.trees[method] = root

		r.globalAllowed = r.allowed("*", "")
	}

	root.addRoute(path, handle)
	...
}

In this method, a lot of details are also omitted.We'll just look at what's relevant to this article.As we can see, in this method, if the tree has not been initialized, the prefix tree is initialized first.

Then we noticed that the tree is a map structure.That is, a method corresponds to a tree.Then, corresponding to the tree, call the addRoute method to save the URI and the corresponding Handle.

3 Prefix Tree

3.1 Definition

Simply put, it is what you are looking for, as long as you follow a path of this tree, you can find it.

For example, in the search engine, you entered a Cai:

He would have these associations, or he could be interpreted as a prefix tree.

Another example is:

In the prefix tree of this GET method, the following routes are included:

• /wow/awesome
• /test
• /hello/world
• /hello/china
• /hello/chinese

When it comes to building this tree, you should understand that any two nodes that have the same prefix will be merged into one node.

3.2 Diagram Construction

The addRoute method mentioned above is the insertion method of this prefix tree.Assuming the present number is empty, I am going to illustrate the construction of this tree here.

Suppose we need to insert three routes:

• /hello/world
• /hello/china
• /hello/chinese

(1) Insert/hello/world

Because the tree is empty at this time, you can insert it directly:

(2) Insert/hello/china

At this point, it is found that/hello/world and/hello/china have the same prefix/hello/.

The original/hello/world node is then split, and the node to be inserted/hello/china is truncated to act as a child of/hello/world.

(3) Insert/hello/chinese

At this point, we need to insert/hello/Chinese, but we found that, /hello/chinese and node/hello/have a common prefix/hello/, so we'll look at the children of/hello/this node.

Notice that there is an attribute in the node called indices.It records the first letters of the child nodes of this node so that we can find them easily.For example, for this / hello / node, its indices value is wc.And the node we're going to insert is / hello/chinese, except for the public prefix, the first letter of Chinese is also c, so we're going to go into this node called china.

Did we find that at this point, it's back to where we started inserting / hello/china?At that time the public prefix was/hello/, and now the public prefix is chin.

So we also truncate chin as a node and a as a child of this node.Also, ese is used as a child node.

3.3 Summary Construction Algorithms

At this point, the build is over.Let's summarize the algorithm.

Specific annotated code will be given at the end of this article, and you can look at it yourself if you want to learn more.Understand the process here:

(1) If the tree is empty, insert it directly
(2) Otherwise, find out if the current node has a common prefix with the URI to insert
(3) If there is no public prefix, insert it directly
(4) If there is a common prefix, determine if the current node needs to be split
(5) If splitting is required, use the public part as the parent and the rest as the child
(6) If splitting is not required, look for children with the same prefix
(7) If the prefix is the same, skip to (4)
(8) If no prefix is the same, insert it directly
(9) At the last node, put the Handle corresponding to this route

But when we got here, some students asked: How can I route here without parameters?

In fact, as long as you understand the above process, the same is true with parameters.The logic is that before each insert, the path of the node being inserted is scanned for parameters (that is, whether the scan has or has*).If you have parameters, set the wildChild property of the current node to true, then set the parameter section to a new child node.

4 Monitoring

After we finish registering routes, let's talk about monitoring routes.

stay Last article We mentioned this in our content:

type serverHandler struct {
	srv *Server
}

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
	handler := sh.srv.Handler
	if handler == nil {
		handler = DefaultServeMux
	}
	if req.RequestURI == "*" && req.Method == "OPTIONS" {
		handler = globalOptionsHandler{}
	}
	handler.ServeHTTP(rw, req)
}

At that time, we mentioned that if we don't pass in any of the Andle methods, Golang will use the default DefaultServeMux method to process requests.Now that we've passed in a router, we'll use it to process the request.

Therefore, router also implements the ServeHTTP method.Let's take a look (some steps are also omitted):

func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	...
	path := req.URL.Path

	if root := r.trees[req.Method]; root != nil {
		if handle, ps, tsr := root.getValue(path, r.getParams); handle != nil {
			if ps != nil {
				handle(w, req, *ps)
				r.putParams(ps)
			} else {
				handle(w, req, nil)
			}
			return
		} 
	}
    ...
    // Handle 404
	if r.NotFound != nil {
		r.NotFound.ServeHTTP(w, req)
	} else {
		http.NotFound(w, req)
	}
}

Here, we select the prefix tree corresponding to the request method and call the getValue method.

Explain this method briefly: In this method, the path in the current path and node is matched continuously until the Handle method corresponding to this route is found.

Note that during this period, if the route is RESTful and contains parameters, it will be saved in Param, where the Param structure is as follows:

type Param struct {
	Key   string
	Value string
}

If no corresponding route is found, the subsequent 404 method is called.

5 Processing

At this point, it's almost the same as before.

This function is called after the corresponding Handle for the route is obtained.

The only difference with the Handler used in the net/http package before is that the Handle here encapsulates the parameters obtained from the API.

type Handle func(http.ResponseWriter, *http.Request, Params)

6 Write at the end

Thank you for seeing here~

So far, httprouter has been introduced, the most critical is the construction of prefix tree.In the above I have simulated a prefix tree building process in a graphical and textual way, hoping that you can understand how the prefix tree works.Of course, if you have any questions, you can also leave a message or communicate with me in a micro-letter~

Of course, if you're not satisfied with this, you can see the appendix below, with the full code comments for the prefix tree.

Of course, the author is just getting started.So there may be a lot of omissions.If there are any inadequate explanations or errors in your understanding during the reading process, please leave a message to correct them.

Thanks again~

PS: If you have other questions, you can also find the author on the public number.Also, all articles will be updated on the Public Number at the first time. Welcome to visit the author ~

7 Source reading

7.1 Tree Structure

type node struct {
	
	path      string    //URI of the current node
	indices   string    //First letter of child node
	wildChild bool      //Is Child Node a Parameter Node
	nType     nodeType  //Node Stereotypes
	priority  uint32    //weight
	children  []*node   //Child Node
	handle    Handle    //processor
}

7.2 addRoute

func (n *node) addRoute(path string, handle Handle) {

	fullPath := path
	n.priority++

	// If this is an empty tree, insert it directly
	if len(n.path) == 0 && len(n.indices) == 0 {

		//This method actually inserts a path at the n node, but handles the parameters
		//Detailed implementation will be shown later
		n.insertChild(path, fullPath, handle)
		n.nType = root
		return
	}

	//Set a flag
walk:
	for {
		// Find the longest prefix in the current node path and the path to insert
		// i is the first different subscript
		i := longestCommonPrefix(path, n.path)

		// The same part is shorter than the path recorded by this node at this time
		// That is, the current node needs to be split
		if i < len(n.path) {
			child := node{

				// Set different parts as a slice as child nodes
				path:      n.path[i:],
				wildChild: n.wildChild,
				nType:     static,
				indices:   n.indices,
				children:  n.children,
				handle:    n.handle,
				priority:  n.priority - 1,
			}

			// Make the new node a child of this node
			n.children = []*node{&child}
			// Add the first letter of this node to indices
			// The goal is to find faster
			n.indices = string([]byte{n.path[i]})
			n.path = path[:i]
			n.handle = nil
			n.wildChild = false
		}

		// At this point the same part is only part of the new URI
		// So set the different parts behind the path to a new node
		if i < len(path) {
			path = path[i:]

			// In this case, if the child node of n is parameterized
			if n.wildChild {
				n = n.children[0]
				n.priority++

				// Determine if it will be illegal
				if len(path) >= len(n.path) && n.path == path[:len(n.path)] &&
					n.nType != catchAll &&
					(len(n.path) >= len(path) || path[len(n.path)] == '/') {
					continue walk
				} else {
					pathSeg := path
					if n.nType != catchAll {
						pathSeg = strings.SplitN(pathSeg, "/", 2)[0]
					}
					prefix := fullPath[:strings.Index(fullPath, pathSeg)] + n.path
					panic("'" + pathSeg +
						"' in new path '" + fullPath +
						"' conflicts with existing wildcard '" + n.path +
						"' in existing prefix '" + prefix +
						"'")
				}
			}

			// Record the first intercepted path
			idxc := path[0]

			// If the child node of n is parameterized at this time
			if n.nType == param && idxc == '/' && len(n.children) == 1 {
				n = n.children[0]
				n.priority++
				continue walk
			}

			// This step checks to see if the split path should be merged into the child nodes
			// For example, see the illustration above
			// If so, set this child node to n and start a new cycle
			for i, c := range []byte(n.indices) {
				if c == idxc {
					// This part is to adjust the higher weight first character to the front
					i = n.incrementChildPrio(i)
					n = n.children[i]
					continue walk
				}
			}

			// If this node does not need to be merged
			if idxc != ':' && idxc != '*' {
				// Add the first letter of this node to the indices of n
				n.indices += string([]byte{idxc})
				child := &node{}
				n.children = append(n.children, child)
				n.incrementChildPrio(len(n.indices) - 1)
				// Create a new node
				n = child
			}
			// Insert this node
			n.insertChild(path, fullPath, handle)
			return
		}

		// Insert directly into the current node
		if n.handle != nil {
			panic("a handle is already registered for path '" + fullPath + "'")
		}
		n.handle = handle
		return
	}
}

7.3 insertChild

func (n *node) insertChild(path, fullPath string, handle Handle) {
	for {
		// This method is used to find out if the path contains parameters
		wildcard, i, valid := findWildcard(path)
		// Without parameters, jump out of the loop and look at the last two lines
		if i < 0 {
			break
		}

		// Conditional Check
		if !valid {
			panic("only one wildcard per path segment is allowed, has: '" +
				wildcard + "' in path '" + fullPath + "'")
		}

		// Similarly determine legality
		if len(wildcard) < 2 {
			panic("wildcards must be named with a non-empty name in path '" + fullPath + "'")
		}

		if len(n.children) > 0 {
			panic("wildcard segment '" + wildcard +
				"' conflicts with existing children in path '" + fullPath + "'")
		}

		// If the first bit of the parameter is `:`, then this is a parameter type
		if wildcard[0] == ':' {
			if i > 0 {
				// Set the current path to the part before the parameter
				n.path = path[:i]
				// Ready to use the part after the parameter as a new node
				path = path[i:]
			}

			//Then the parameter part is used as the new node
			n.wildChild = true
			child := &node{
				nType: param,
				path:  wildcard,
			}
			n.children = []*node{child}
			n = child
			n.priority++

			// This means that the path does not end after the parameter
			if len(wildcard) < len(path) {
				// Divide the part after the parameter into another node and continue continues
				path = path[len(wildcard):]
				child := &node{
					priority: 1,
				}
				n.children = []*node{child}
				n = child
				continue
			}

			// Set Processor In
			n.handle = handle
			return

		} else { // Another case
			if i+len(wildcard) != len(path) {
				panic("catch-all routes are only allowed at the end of the path in path '" + fullPath + "'")
			}

			if len(n.path) > 0 && n.path[len(n.path)-1] == '/' {
				panic("catch-all conflicts with existing handle for the path segment root in path '" + fullPath + "'")
			}

			// Determine if there is one/
			i--
			if path[i] != '/' {
				panic("no / before catch-all in path '" + fullPath + "'")
			}

			n.path = path[:i]

			// Set a child node of catchAll type
			child := &node{
				wildChild: true,
				nType:     catchAll,
			}
			n.children = []*node{child}
			n.indices = string('/')
			n = child
			n.priority++

			// Set the latter parameter section to the new node
			child = &node{
				path:     path[i:],
				nType:    catchAll,
				handle:   handle,
				priority: 1,
			}
			n.children = []*node{child}

			return
		}
	}

	// For the first part, if there are no parameters in this path, set it directly
	n.path = path
	n.handle = handle
}

The key ways are all over here. Applause the people who see it!

This part of the understanding will be more difficult and may need to be read a few more times.

If there is still something difficult to understand, please leave a message to exchange, or come directly to the public number to find me~