The etymology of Gin a strong alcohol, comes from the Netherlands.

In Go, there is a frequently mentioned web framework, which is Gin, which has the characteristics of high performance and flexible customization. Since it is so promising, before understanding it in depth, you might as well Let’s take a look at what he is based on.

Consider the origin

According to the description of the Git author, the high performance of Gin is generated by a framework called httprouter. Looking at the source, we first target httprouter and start from HTTP routing , as an introduction to the Gin framework.

Route

First sort out the concept of routing:
Probably means to achieve interconnection by forwarding data packets, such as the common physical router in life, which refers to the distribution of information flow between the internal network and the external network.

Similarly, the software layer also has the concept of routing, which is generally exposed on the business layer and used to forward requests to the appropriate logical processor.

The router matches incoming requests by the request method and the path.

In the application of the program, it is common for the external server to send the external request to the Nginx gateway, and then routing (forwarding) to the internal service or the “control layer” of the internal service, such as Java’s springMVC, Go’s native router, etc. forward different requests to different business layers.
Or to be more specific, calling a method with the same name with different parameters, such as Java overloading, can also be understood as the program is routed to different methods according to different parameters.

httprouter

method usage:

On the README document of Git, httprouter shows one of its common functions straight to the point,
Start an HTTP server, listen on port 8080, and perform parameter parsing on the request. It is only a few lines of code. When I first saw this implementation, I really felt that the implementation of go is quite elegant.

router.GET("/", Index)
router.GET("/hello/:name", Hello)

func Hello(w http.ResponseWriter, r *http.Request) {
    // get the parameters by the context of the http.Request structure
    params := httprouter.ParamsFromContext(r.Context())
    fmt.Fprintf(w, "hello, %s!\n", params.ByName("name"))
}

http.ListenAndServe(":8080", router)

interface implement

After observing how to create a listener, before digging into how this elegance is encapsulated and implemented, we must first understand that in native Go, each Router``` routing structure implements `http.Handlerinterface,Handlerhas only one method body, which isServerHTTP```, which has only one function, that is, to process requests and respond.

type Handler interface {
	ServeHTTP(ResponseWriter, *Request)
}

Go is more inclined to KISS Pattern or single responsibility, the function of each interface is singled, and if necessary we can combine them again, use combination instead of inheritance, and it will be used as a code specificationsin the future.

net\http\server

// The HandlerFunc type is an adapter to allow the use of
// ordinary functions as HTTP handlers. If f is a function
// with the appropriate signature, HandlerFunc(f) is a
// Handler that calls f.
type HandlerFunc func(ResponseWriter, *Request)

// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
	f(w, r)
}

It can be seen that in the Go native library, the ServeHTTP() implementation body HandlerFunc is a func function type, and the specific implementation is to directly apply HandlerFunc Is there a kind of “I realize myself”.

All in all, Let’s put aside the encapsulation of third-party libraries and review the standard library. If we want to use the native http\server package to build an HTTP processing logic, we can generally use:
Way 1:

  1. define a method with signature as(ResponseWriter, *Request)
  2. register it to http.Serveras a member call Handler
  3. call ListenAndServer to listen on http.Server

Way 2:

  1. define a struct and implement it with ServeHTTP(w http.ResponseWriter, req *http.Request)
  2. register it to http.Serveras a member call Handler
  3. call ListenAndServer to listen on http.Server

Code Example:

// Way 1
func SelfHandler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprint(w, "<h1>Hello!</h1>")
}

// Way 2
type HelloHandler struct {
}

// write your implement logic
func (* HelloHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	fmt.Fprint(w, "<h1>Hello!</h1>")
}

s := &http.Server{
	Addr:           ":8080",
	// Way 1
	Handler:        SelfHandler,
	// Way 2
	// Handler:      &HelloHandler{},
	ReadTimeout:    10 * time.Second,
	WriteTimeout:   10 * time.Second,
	MaxHeaderBytes: 1 << 20,
}
s.ListenAndServe()

Take a shot
The above is the common usage of the Go standard library to implement http services. Now let’s expand. If we need to get parameters through the url, such as Get request, localhost:8080/abc/1 Q: How do we get abc or 1?
A:In fact, there is a relatively crude method

  • Use the Parse() function of net/url to extract the paragraph after 8080
  • Use stings.split(ul, "/")
  • Parameter ranges using subscripts

Code Example

func TestStartHelloWithHttp(t *testing.T)  {
	//fmt.Println(path.Base(ul))

	ul := `https://localhost:8080/pixeldin/123`
	parse, e := url.Parse(ul)
	if e != nil {
		log.Fatalf("%v", e)
	}
	//fmt.Println(parse.Path)	// "/pixeldin/123"
	name := GetParamFromUrl(parse.Path, 1)
	id := GetParamFromUrl(parse.Path, 2)
	fmt.Println("name: " + name + ", id: " + id)	
}

// specify the subscript to return the value of the relative url
func GetParamFromUrl(base string, index int) (ps string) {
	kv := strings.Split(base, "/")
	assert(index < len(kv), errors.New("index out of range."))
	return kv[index]
}

func assert(ok bool, err error)  {
	if !ok {
		panic(err)
	}
}

Output:

name: pixeldin, id: 123

This method feels quite violent, and it needs to remember the location and corresponding value of each parameter, and multiple URLs cannot be managed uniformly, and each addressing is traversal. Although the Go standard library also provides some general functions.

Such as the following example:
GET request url: https://localhost:8080/?key=hello
we can use *http.Request to obtain, this request method is to declare the key-value pair in the url, and then the background extracts according to the request key.

// Referenced from:https://golangcode.com/get-a-url-parameter-from-a-request/
func handler(w http.ResponseWriter, r *http.Request) {

    keys, ok := r.URL.Query()["key"]
    
    if !ok || len(keys[0]) < 1 {
        log.Println("Url Param 'key' is missing")
        return
    }

    // Query()["key"] will return an array of items, 
    // we only want the single item.
    key := keys[0]

    log.Println("Url Param 'key' is: " + string(key))
}

I believe that we all prefers the example listed at the beginning, including several mainstream frameworks that we are used to now, all tend to use the location of the url to search for parameters. Of course, the advantages of httprouter are definitely not only here, but here only as a An entry point to understand httprouter.

router.GET("/hello/:name", Hello)
router.GET("/hello/*name", HelloWorld)

Later, we will track the underlying implementation after they are encapsulated and how to plan url parameters.


How httprouter implement the ServerHTTP()

As mentioned earlier, all routing structures implement the ServeHTTP() method of the http.Handler interface. Let’s take a look at the implementation method of httprouter based on it.

julienschmidt\httprouter

In httprouter, the implementation structure of ServeHTTP() is called *Router, which encapsulates the tree structure used to retrieve url, several commonly used Boolean options, and several default handlers are also implemented based on http.Handler, and its implementation is as follows:

// ServeHTTP makes the router implement the http.Handler interface.
func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	if r.PanicHandler != nil {
		defer r.recv(w, req)
	}

	path := req.URL.Path

	if root := r.trees[req.Method]; root != nil {
	    // getValue() returns the processing method and parameter list
		if handle, ps, tsr := root.getValue(path); handle != nil {
			handle(w, req, ps)
			return
		} else if req.Method != http.MethodConnect && path != "/" {
			//...
		}
	}

	if req.Method == http.MethodOptions && r.HandleOPTIONS {
		// Handle OPTIONS requests
	} else if r.HandleMethodNotAllowed { // Handle 405
		// execute default handler...
	}

	// Handle 404
	if r.NotFound != nil {
		r.NotFound.ServeHTTP(w, req)
	} else {
		http.NotFound(w, req)
	}
}

At this point, it can be roughly guessed that it injects the processing method into the internal trees structure, uses the incoming url to perform matching searches in the trees, and executes the execution chain accordingly.
It can be guessed that this Router.trees contains handle and corresponding parameters, and then we enter its routing index function to see how it implements url matching and parameter parsing.

Sneaking in
This trees exists in the tree.go source file, which is actually a map key-value pair, The key is HTTP methods (such as GET/HEAD/POST/PUT, etc.), and the method is the node on which the current method is bound to the method

I added some comments to the source code, believe it will more easy for you to understand.

// Handle registers a new request handle with the given path and method.
//
// For GET, POST, PUT, PATCH and DELETE requests the respective shortcut
// functions can be used.
// ...
func (r *Router) Handle(method, path string, handle Handle) {
	if len(path) < 1 || path[0] != '/' {
		panic("path must begin with '/' in path '" + path + "'")
	}

	if r.trees == nil {
		r.trees = make(map[string]*node)
	}

    // bind http methods root node, method can be GET/POST/PUT, etc.
	root := r.trees[method]
	if root == nil {
		root = new(node)
		r.trees[method] = root

		r.globalAllowed = r.allowed("*", "")
	}
    
	root.addRoute(path, handle)
}

The value of the ``Router.treesmap key-value pair is a node``` structure, each HTTP METHOD is a root node, and the most important path allocation is in these nodes the addRoute() function, eventually those paths with the same prefix will be bound to the same branch direction of the tree, which directly improves the efficiency of the index.

Below I will list a few more important members of node:

type node struct {
	path      string
	// identifies whether the path is followed by ':', which is used for parameter judgment
	wildChild bool
	/* type of the current node, the default is 0,
	(root/param/catchAll) Identify separately 
	(root/with parameters/full path) */
	nType     nodeType
	maxParams uint8
	// the priority of the current node, the more child nodes hanging on it, the higher the priority
	priority  uint32
	indices   string
	// child nodes that satisfy the prefix can be extended
	children  []*node
	// the processing logic block bound to the current node
	handle    Handle
}

The more child nodes, or the root nodes with more handle methods bound, the higher the priority is. The author consciously prioritizes each registration completion. Quoting the author’s note:

This helps in two ways:

  • Nodes which are part of the most routing paths are evaluated first. This helps to make as much routes as possible to be reachable as fast as possible.
  • It is some sort of cost compensation. The longest reachable path (highest cost) can always be evaluated first. The following scheme visualizes the tree structure. Nodes are evaluated from top to bottom and from left to right.

Nodes with high priority are conducive to the rapid positioning of handles. I believe it is easier to understand. In reality, densely populated areas are often crossroads, similar to transportation hubs. Matching based on the prefix tree allows addressing to start at a dense location, which helps to improve efficiency.

Step by step
Let’s first register the GET processing logic provided by several authors with the router, and then start debugging to see what changes to the tree members as the url is added.

router.Handle("GET", "/user/ab/", func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
	//do nothing, just add path+handler
})

router.Handle("GET", "/user/abc/", func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
	//do nothing, just add path+handler
})

router.Handle(http.MethodGet, "/user/query/:name", func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
	routed = true
	want := httprouter.Params{httprouter.Param{"name", "gopher"}}
	if !reflect.DeepEqual(ps, want) {
		t.Fatalf("wrong wildcard values: want %v, got %v", want, ps)
	}
})

The above operation takes [RESTful GET method, url path, anonymous function handler] as the parameters of router.Handler(), router.Handler( ) operation has been briefly analyzed above. The main nodes are divided into the addRoute() function. Let’s briefly go over its logic.

// put the current url and processing logic on the current node
func (n *node) addRoute(path string, handle Handle) {
	fullPath := path
	n.priority++
	// extract the number of current url parameters
	numParams := countParams(path)

	// if the current node already has a registered link
	if len(n.path) > 0 || len(n.children) > 0 {
	walk:
		for {
			if numParams > n.maxParams {
				n.maxParams = numParams
			}
			
			// This also implies that the common prefix contains no ':' or '*'
			// since the existing key can't contain those chars.
			i := 0
			max := min(len(path), len(n.path))
			for i < max && path[i] == n.path[i] {
				i++
			}

			/*
			    If the matching length of the incoming url is greater than the existing url of the current node, create a child node
			    Like a /user/ab/ with func1,
			    
			    then receive a /user/abc/ with func2, then need to create child nodes of /user/ab/ and c/
			    the tree will look like this:
        			    |-/user/ab
        			    |--------|-/ func1  
        			    |--------|-c/ func2
        			    
			    and now, a new map comming, /user/a/ with func3,
			    then the final tree will be adjusted to:
        		Priority 3 |-/user/a 
        		Priority 2 |--------|-b 
        		Priority 1 |----------|-/ func1
        		Priority 1 |----------|-c/ func2
        		Priority 1 |--------|-/ func3
			    
		    */
			if i < len(n.path) {
				child := node{
					path:      n.path[i:],
					wildChild: n.wildChild,
					nType:     static,
					indices:   n.indices,
					children:  n.children,
					handle:    n.handle,
					priority:  n.priority - 1,
				}

				// traverse the child nodes and take the highest priority as the priority of the parent node
				for i := range child.children {
					if child.children[i].maxParams > child.maxParams {
						child.maxParams = child.children[i].maxParams
					}
				}

				n.children = []*node{&child}
				// []byte for proper unicode char conversion, see #65
				n.indices = string([]byte{n.path[i]})
				n.path = path[:i]
				n.handle = nil
				n.wildChild = false
			}

			// Make new node a child of this node
			if i < len(path) {
				path = path[i:]

				if n.wildChild {
					n = n.children[0]
					n.priority++

					// Update maxParams of the child node
					if numParams > n.maxParams {
						n.maxParams = numParams
					}
					numParams--

					// Check if the wildcard matches
					if len(path) >= len(n.path) && n.path == path[:len(n.path)] &&
						// Adding a child to a catchAll is not possible
						n.nType != catchAll &&
						// Check for longer wildcard, e.g. :name and :names
						(len(n.path) >= len(path) || path[len(n.path)] == '/') {
						continue walk
					} else {
						// Wildcard conflict
						var pathSeg string
						if n.nType == catchAll {
							pathSeg = path
						} else {
							pathSeg = strings.SplitN(path, "/", 2)[0]
						}
						prefix := fullPath[:strings.Index(fullPath, pathSeg)] + n.path
						panic("'" + pathSeg +
							"' in new path '" + fullPath +
							"' conflicts with existing wildcard '" + n.path +
							"' in existing prefix '" + prefix +
							"'")
					}
				}

				c := path[0]

				// slash after param
				if n.nType == param && c == '/' && len(n.children) == 1 {
					n = n.children[0]
					n.priority++
					continue walk
				}

				// Check if a child with the next path byte exists
				for i := 0; i < len(n.indices); i++ {
					if c == n.indices[i] {
					    // increase the priority of the current node and make a position adjustment
						i = n.incrementChildPrio(i)
						n = n.children[i]
						continue walk
					}
				}

				// Otherwise insert it
				if c != ':' && c != '*' {
					// []byte for proper unicode char conversion, see #65
					n.indices += string([]byte{c})
					child := &node{
						maxParams: numParams,
					}
					n.children = append(n.children, child)
					n.incrementChildPrio(len(n.indices) - 1)
					n = child
				}
				n.insertChild(numParams, path, fullPath, handle)
				return

			} else if i == len(path) { // Make node a (in-path) leaf
				if n.handle != nil {
					panic("a handle is already registered for path '" + fullPath + "'")
				}
				n.handle = handle
			}
			return
		}
	} else { // Empty tree
		n.insertChild(numParams, path, fullPath, handle)
		n.nType = root
	}
}

The general idea above is to register each handle func() on a url prefix tree, and branch according to the same matching degree of url prefixes to improve routing efficiency.

Parameter Lookup
Next, let’s see how httprouter encapsulates the parameter param in the context:
It is not difficult to guess that when the branch is divided, the pre-received parameters are extracted by judging the keyword “:”. These parameters are stored in string (dictionary) key-value pairs, and the bottom layer is stored in a Param structure:

type Param struct {
	Key   string
	Value string
}

The concept of context is also very common in other languages, such as application-context in the Java Spring framework, which is used to manage some global properties throughout the program life cycle.
The context of Go also has multiple implementations in different frameworks. Here we first understand that the top-level context of the Go program is background(), which is the source of all subcontexts, similar to the Linux system’s init() process.

Let’s take an example first, show the usage of context parameters in Go:

func TestContext(t *testing.T) {	
	ctx := context.Background()	
	valueCtx := context.WithValue(ctx, "hello", "pixel")
	value := valueCtx.Value("hello")
	if value != nil {
		fmt.Printf("Params type: %v, value: %v.\n", reflect.TypeOf(value), value)
	}
}

Output:

Params type: string, value: pixel.

In httprouter, the context encapsulated in http.request is actually a valueCtx, the type is the same as valueCtx in our chestnut above, the framework provides A method to get the Params key-value pair from the context.

func ParamsFromContext(ctx context.Context) Params {
	p, _ := ctx.Value(ParamsKey).(Params)
	return p
}

Using the returned Params, we can get our target value according to the key

params.ByName(key)

After tracing, Params comes from a function called getValue(path string) (handle Handle, p Params, tsr bool), remember the ServeHTTP() implemented by the *Router route listed above?

// part of the ServeHTTP() function
if root := r.trees[req.Method]; root != nil {    
	if handle, ps, tsr := root.getValue(path); handle != nil {
	    // matching execution, the handle here is the anonymous function above
		handle(w, req, ps)
		return
	} else if req.Method != http.MethodConnect && path != "/" {
		// ...
	}
}

ServeHTTP() There is a getValue function, and its return value has two important members: the processing logic of the current route and the url parameter list, so when the route is registered, we need to put params is passed in as input parameters.

Like this:

router.Handle(http.MethodGet, "/user/query/:name", 
// Anonymous function, the ps parameter here is extracted for you when ServeHttp()
func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
	fmt.Println(params.ByName("name"))
})

getValue()When judging the type of the current node n, if it is of the ‘param’ type (it has been classified according to the url when addingRoute), the parameter Params to be returned is filled.

//-----------go
//...github.com/julienschmidt/httprouter@v1.3.0/tree.go:367
switch n.nType {
case param:
	// find param end (either '/' or path end)
	end := 0
	for end < len(path) && path[end] != '/' {
		end++
	}
	
    // node parameter encountered
	if p == nil {
		// lazy allocation
		p = make(Params, 0, n.maxParams)
	}
	i := len(p)
	p = p[:i+1] // expand slice within preallocated capacity
	p[i].Key = n.path[1:]
	p[i].Value = path[:end]
//...

Process Step
So far,Let’s summarize the routing process of httprouter again:

  • Step 1 Initialize the creation of the router router
  • Step 2 Register a function whose signature is: type Handle func(http.ResponseWriter, *http.Request, Params) to the router
  • Step 3 Call the HTTP common interface ServeHTTP() to extract the expected parameters of the current url and use it for the business layer

The above is my understanding of httprouter in the past two days. By the way, I have a further understanding of Go’s Http. I will try to enter gin later to see what Gin has done based on httprouter and familiar with common usage.

julienschmidt/httprouter
https://github.com/julienschmidt/httprouter