Before talking about the Gin web framework, let's take a look at httprouter
The etymology of Gin a strong alcohol, comes from the Netherlands.
In Go, there is a frequently mentioned web framework, which is Gin, which has the characteristics of high performance and flexible customization. Since it is so promising, before understanding it in depth, you might as well Let’s take a look at what he is based on.
Consider the origin
According to the description of the Git author, the high performance of Gin is generated by a framework called httprouter. Looking at the source, we first target httprouter and start from HTTP routing , as an introduction to the Gin framework.
Route
First sort out the concept of routing:
Probably means to achieve interconnection by forwarding data packets, such as the common physical router in life, which refers to the distribution of information flow between the internal network and the external network.
Similarly, the software layer also has the concept of routing, which is generally exposed on the business layer and used to forward requests to the appropriate logical processor.
The router matches incoming requests by the request method and the path.
In the application of the program, it is common for the external server to send the external request to the Nginx gateway, and then routing (forwarding) to the internal service or the “control layer” of the internal service, such as Java’s springMVC, Go’s native router, etc. forward different requests to different business layers.
Or to be more specific, calling a method with the same name with different parameters, such as Java overloading, can also be understood as the program is routed to different methods according to different parameters.
httprouter
method usage:
On the README document of Git, httprouter shows one of its common functions straight to the point,
Start an HTTP server, listen on port 8080, and perform parameter parsing on the request. It is only a few lines of code. When I first saw this implementation, I really felt that the implementation of go is quite elegant.
router.GET("/", Index)
router.GET("/hello/:name", Hello)
func Hello(w http.ResponseWriter, r *http.Request) {
// get the parameters by the context of the http.Request structure
params := httprouter.ParamsFromContext(r.Context())
fmt.Fprintf(w, "hello, %s!\n", params.ByName("name"))
}
http.ListenAndServe(":8080", router)
interface implement
After observing how to create a listener, before digging into how this elegance is encapsulated and implemented, we must first understand that in native Go, each Router``` routing structure implements
`http.Handlerinterface,
Handlerhas only one method body, which is
ServerHTTP```, which has only one function, that is, to process requests and respond.
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
Go is more inclined to KISS Pattern or single responsibility, the function of each interface is singled, and if necessary we can combine them again, use combination instead of inheritance, and it will be used as a code specificationsin the future.
net\http\server
// The HandlerFunc type is an adapter to allow the use of
// ordinary functions as HTTP handlers. If f is a function
// with the appropriate signature, HandlerFunc(f) is a
// Handler that calls f.
type HandlerFunc func(ResponseWriter, *Request)
// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
f(w, r)
}
It can be seen that in the Go
native library, the ServeHTTP()
implementation body HandlerFunc
is a func function type, and the specific implementation is to directly apply HandlerFunc
Is there a kind of “I realize myself”.
All in all, Let’s put aside the encapsulation of third-party libraries and review the standard library. If we want to use the native http\server package to build an HTTP processing logic, we can generally use:
Way 1:
- define a method with signature as(ResponseWriter, *Request)
- register it to
http.Server
as a member callHandler
- call ListenAndServer to listen on http.Server
Way 2:
- define a struct and implement it with
ServeHTTP(w http.ResponseWriter, req *http.Request)
- register it to
http.Server
as a member callHandler
- call ListenAndServer to listen on http.Server
Code Example:
// Way 1
func SelfHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "<h1>Hello!</h1>")
}
// Way 2
type HelloHandler struct {
}
// write your implement logic
func (* HelloHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
fmt.Fprint(w, "<h1>Hello!</h1>")
}
s := &http.Server{
Addr: ":8080",
// Way 1
Handler: SelfHandler,
// Way 2
// Handler: &HelloHandler{},
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
s.ListenAndServe()
Take a shot
The above is the common usage of the Go standard library to implement http services. Now let’s expand. If we need to get parameters through the url, such as Get request, localhost:8080/abc/1
Q: How do we get abc or 1?
A:In fact, there is a relatively crude method
- Use the
Parse()
function ofnet/url
to extract the paragraph after8080
- Use
stings.split(ul, "/")
- Parameter ranges using subscripts
Code Example
func TestStartHelloWithHttp(t *testing.T) {
//fmt.Println(path.Base(ul))
ul := `https://localhost:8080/pixeldin/123`
parse, e := url.Parse(ul)
if e != nil {
log.Fatalf("%v", e)
}
//fmt.Println(parse.Path) // "/pixeldin/123"
name := GetParamFromUrl(parse.Path, 1)
id := GetParamFromUrl(parse.Path, 2)
fmt.Println("name: " + name + ", id: " + id)
}
// specify the subscript to return the value of the relative url
func GetParamFromUrl(base string, index int) (ps string) {
kv := strings.Split(base, "/")
assert(index < len(kv), errors.New("index out of range."))
return kv[index]
}
func assert(ok bool, err error) {
if !ok {
panic(err)
}
}
Output:
name: pixeldin, id: 123
This method feels quite violent, and it needs to remember the location and corresponding value of each parameter, and multiple URLs cannot be managed uniformly, and each addressing is traversal. Although the Go standard library also provides some general functions.
Such as the following example:
GET request url: https://localhost:8080/?key=hello
we can use *http.Request
to obtain, this request method is to declare the key-value pair in the url, and then the background extracts according to the request key.
// Referenced from:https://golangcode.com/get-a-url-parameter-from-a-request/
func handler(w http.ResponseWriter, r *http.Request) {
keys, ok := r.URL.Query()["key"]
if !ok || len(keys[0]) < 1 {
log.Println("Url Param 'key' is missing")
return
}
// Query()["key"] will return an array of items,
// we only want the single item.
key := keys[0]
log.Println("Url Param 'key' is: " + string(key))
}
I believe that we all prefers the example listed at the beginning, including several mainstream frameworks that we are used to now, all tend to use the location of the url to search for parameters. Of course, the advantages of httprouter
are definitely not only here, but here only as a An entry point to understand httprouter
.
router.GET("/hello/:name", Hello)
router.GET("/hello/*name", HelloWorld)
Later, we will track the underlying implementation after they are encapsulated and how to plan url parameters.
How httprouter implement the ServerHTTP()
As mentioned earlier, all routing structures implement the ServeHTTP()
method of the http.Handler
interface. Let’s take a look at the implementation method of httprouter
based on it.
julienschmidt\httprouter
In httprouter, the implementation structure of ServeHTTP() is called *Router
, which encapsulates the tree structure used to retrieve url, several commonly used Boolean options, and several default handlers are also implemented based on http.Handler
, and its implementation is as follows:
// ServeHTTP makes the router implement the http.Handler interface.
func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
if r.PanicHandler != nil {
defer r.recv(w, req)
}
path := req.URL.Path
if root := r.trees[req.Method]; root != nil {
// getValue() returns the processing method and parameter list
if handle, ps, tsr := root.getValue(path); handle != nil {
handle(w, req, ps)
return
} else if req.Method != http.MethodConnect && path != "/" {
//...
}
}
if req.Method == http.MethodOptions && r.HandleOPTIONS {
// Handle OPTIONS requests
} else if r.HandleMethodNotAllowed { // Handle 405
// execute default handler...
}
// Handle 404
if r.NotFound != nil {
r.NotFound.ServeHTTP(w, req)
} else {
http.NotFound(w, req)
}
}
At this point, it can be roughly guessed that it injects the processing method into the internal trees structure, uses the incoming url to perform matching searches in the trees, and executes the execution chain accordingly.
It can be guessed that this Router.trees
contains handle and corresponding parameters, and then we enter its routing index function to see how it implements url matching and parameter parsing.
Sneaking in
This trees
exists in the tree.go source file, which is actually a map key-value pair,
The key is HTTP methods (such as GET/HEAD/POST/PUT, etc.), and the method is the node on which the current method is bound to the method
I added some comments to the source code, believe it will more easy for you to understand.
// Handle registers a new request handle with the given path and method.
//
// For GET, POST, PUT, PATCH and DELETE requests the respective shortcut
// functions can be used.
// ...
func (r *Router) Handle(method, path string, handle Handle) {
if len(path) < 1 || path[0] != '/' {
panic("path must begin with '/' in path '" + path + "'")
}
if r.trees == nil {
r.trees = make(map[string]*node)
}
// bind http methods root node, method can be GET/POST/PUT, etc.
root := r.trees[method]
if root == nil {
root = new(node)
r.trees[method] = root
r.globalAllowed = r.allowed("*", "")
}
root.addRoute(path, handle)
}
The value of the ``Router.treesmap key-value pair is a
node``` structure, each HTTP METHOD is a root node, and the most important path allocation is in these nodes the addRoute() function, eventually those paths with the same prefix will be bound to the same branch direction of the tree, which directly improves the efficiency of the index.
Below I will list a few more important members of node
:
type node struct {
path string
// identifies whether the path is followed by ':', which is used for parameter judgment
wildChild bool
/* type of the current node, the default is 0,
(root/param/catchAll) Identify separately
(root/with parameters/full path) */
nType nodeType
maxParams uint8
// the priority of the current node, the more child nodes hanging on it, the higher the priority
priority uint32
indices string
// child nodes that satisfy the prefix can be extended
children []*node
// the processing logic block bound to the current node
handle Handle
}
The more child nodes, or the root nodes with more handle methods bound, the higher the priority is. The author consciously prioritizes each registration completion. Quoting the author’s note:
This helps in two ways:
- Nodes which are part of the most routing paths are evaluated first. This helps to make as much routes as possible to be reachable as fast as possible.
- It is some sort of cost compensation. The longest reachable path (highest cost) can always be evaluated first. The following scheme visualizes the tree structure. Nodes are evaluated from top to bottom and from left to right.
Nodes with high priority are conducive to the rapid positioning of handles. I believe it is easier to understand. In reality, densely populated areas are often crossroads, similar to transportation hubs. Matching based on the prefix tree allows addressing to start at a dense location, which helps to improve efficiency.
Step by step
Let’s first register the GET processing logic provided by several authors with the router, and then start debugging to see what changes to the tree members as the url is added.
router.Handle("GET", "/user/ab/", func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
//do nothing, just add path+handler
})
router.Handle("GET", "/user/abc/", func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
//do nothing, just add path+handler
})
router.Handle(http.MethodGet, "/user/query/:name", func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
routed = true
want := httprouter.Params{httprouter.Param{"name", "gopher"}}
if !reflect.DeepEqual(ps, want) {
t.Fatalf("wrong wildcard values: want %v, got %v", want, ps)
}
})
The above operation takes [RESTful GET method, url path, anonymous function handler] as the parameters of router.Handler()
, router.Handler( )
operation has been briefly analyzed above. The main nodes are divided into the addRoute()
function. Let’s briefly go over its logic.
// put the current url and processing logic on the current node
func (n *node) addRoute(path string, handle Handle) {
fullPath := path
n.priority++
// extract the number of current url parameters
numParams := countParams(path)
// if the current node already has a registered link
if len(n.path) > 0 || len(n.children) > 0 {
walk:
for {
if numParams > n.maxParams {
n.maxParams = numParams
}
// This also implies that the common prefix contains no ':' or '*'
// since the existing key can't contain those chars.
i := 0
max := min(len(path), len(n.path))
for i < max && path[i] == n.path[i] {
i++
}
/*
If the matching length of the incoming url is greater than the existing url of the current node, create a child node
Like a /user/ab/ with func1,
then receive a /user/abc/ with func2, then need to create child nodes of /user/ab/ and c/
the tree will look like this:
|-/user/ab
|--------|-/ func1
|--------|-c/ func2
and now, a new map comming, /user/a/ with func3,
then the final tree will be adjusted to:
Priority 3 |-/user/a
Priority 2 |--------|-b
Priority 1 |----------|-/ func1
Priority 1 |----------|-c/ func2
Priority 1 |--------|-/ func3
*/
if i < len(n.path) {
child := node{
path: n.path[i:],
wildChild: n.wildChild,
nType: static,
indices: n.indices,
children: n.children,
handle: n.handle,
priority: n.priority - 1,
}
// traverse the child nodes and take the highest priority as the priority of the parent node
for i := range child.children {
if child.children[i].maxParams > child.maxParams {
child.maxParams = child.children[i].maxParams
}
}
n.children = []*node{&child}
// []byte for proper unicode char conversion, see #65
n.indices = string([]byte{n.path[i]})
n.path = path[:i]
n.handle = nil
n.wildChild = false
}
// Make new node a child of this node
if i < len(path) {
path = path[i:]
if n.wildChild {
n = n.children[0]
n.priority++
// Update maxParams of the child node
if numParams > n.maxParams {
n.maxParams = numParams
}
numParams--
// Check if the wildcard matches
if len(path) >= len(n.path) && n.path == path[:len(n.path)] &&
// Adding a child to a catchAll is not possible
n.nType != catchAll &&
// Check for longer wildcard, e.g. :name and :names
(len(n.path) >= len(path) || path[len(n.path)] == '/') {
continue walk
} else {
// Wildcard conflict
var pathSeg string
if n.nType == catchAll {
pathSeg = path
} else {
pathSeg = strings.SplitN(path, "/", 2)[0]
}
prefix := fullPath[:strings.Index(fullPath, pathSeg)] + n.path
panic("'" + pathSeg +
"' in new path '" + fullPath +
"' conflicts with existing wildcard '" + n.path +
"' in existing prefix '" + prefix +
"'")
}
}
c := path[0]
// slash after param
if n.nType == param && c == '/' && len(n.children) == 1 {
n = n.children[0]
n.priority++
continue walk
}
// Check if a child with the next path byte exists
for i := 0; i < len(n.indices); i++ {
if c == n.indices[i] {
// increase the priority of the current node and make a position adjustment
i = n.incrementChildPrio(i)
n = n.children[i]
continue walk
}
}
// Otherwise insert it
if c != ':' && c != '*' {
// []byte for proper unicode char conversion, see #65
n.indices += string([]byte{c})
child := &node{
maxParams: numParams,
}
n.children = append(n.children, child)
n.incrementChildPrio(len(n.indices) - 1)
n = child
}
n.insertChild(numParams, path, fullPath, handle)
return
} else if i == len(path) { // Make node a (in-path) leaf
if n.handle != nil {
panic("a handle is already registered for path '" + fullPath + "'")
}
n.handle = handle
}
return
}
} else { // Empty tree
n.insertChild(numParams, path, fullPath, handle)
n.nType = root
}
}
The general idea above is to register each handle func() on a url prefix tree, and branch according to the same matching degree of url prefixes to improve routing efficiency.
Parameter Lookup
Next, let’s see how httprouter encapsulates the parameter param in the context:
It is not difficult to guess that when the branch is divided, the pre-received parameters are extracted by judging the keyword “:”. These parameters are stored in string (dictionary) key-value pairs, and the bottom layer is stored in a Param
structure:
type Param struct {
Key string
Value string
}
The concept of context is also very common in other languages, such as application-context
in the Java Spring framework, which is used to manage some global properties throughout the program life cycle.
The context of Go also has multiple implementations in different frameworks. Here we first understand that the top-level context of the Go program is background()
, which is the source of all subcontexts, similar to the Linux system’s init() process.
Let’s take an example first, show the usage of context parameters in Go:
func TestContext(t *testing.T) {
ctx := context.Background()
valueCtx := context.WithValue(ctx, "hello", "pixel")
value := valueCtx.Value("hello")
if value != nil {
fmt.Printf("Params type: %v, value: %v.\n", reflect.TypeOf(value), value)
}
}
Output:
Params type: string, value: pixel.
In httprouter, the context encapsulated in http.request
is actually a valueCtx, the type is the same as valueCtx in our chestnut above, the framework provides A method to get the Params key-value pair from the context.
func ParamsFromContext(ctx context.Context) Params {
p, _ := ctx.Value(ParamsKey).(Params)
return p
}
Using the returned Params, we can get our target value according to the key
params.ByName(key)
After tracing, Params comes from a function called getValue(path string) (handle Handle, p Params, tsr bool)
, remember the ServeHTTP()
implemented by the *Router route listed above?
// part of the ServeHTTP() function
if root := r.trees[req.Method]; root != nil {
if handle, ps, tsr := root.getValue(path); handle != nil {
// matching execution, the handle here is the anonymous function above
handle(w, req, ps)
return
} else if req.Method != http.MethodConnect && path != "/" {
// ...
}
}
ServeHTTP()
There is a getValue
function, and its return value has two important members: the processing logic of the current route and the url parameter list, so when the route is registered, we need to put params is passed in as input parameters.
Like this:
router.Handle(http.MethodGet, "/user/query/:name",
// Anonymous function, the ps parameter here is extracted for you when ServeHttp()
func(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Println(params.ByName("name"))
})
getValue()
When judging the type of the current node n, if it is of the ‘param’ type (it has been classified according to the url when addingRoute), the parameter Params to be returned is filled.
//-----------go
//...github.com/julienschmidt/httprouter@v1.3.0/tree.go:367
switch n.nType {
case param:
// find param end (either '/' or path end)
end := 0
for end < len(path) && path[end] != '/' {
end++
}
// node parameter encountered
if p == nil {
// lazy allocation
p = make(Params, 0, n.maxParams)
}
i := len(p)
p = p[:i+1] // expand slice within preallocated capacity
p[i].Key = n.path[1:]
p[i].Value = path[:end]
//...
Process Step
So far,Let’s summarize the routing process of httprouter
again:
- Step 1 Initialize the creation of the router router
- Step 2 Register a function whose signature is:
type Handle func(http.ResponseWriter, *http.Request, Params)
to the router - Step 3 Call the HTTP common interface
ServeHTTP()
to extract the expected parameters of the current url and use it for the business layer
The above is my understanding of httprouter in the past two days. By the way, I have a further understanding of Go’s Http. I will try to enter gin later to see what Gin
has done based on httprouter and familiar with common usage.
Reference Link
julienschmidt/httprouter
https://github.com/julienschmidt/httprouter