Golang httptrace Example

The httptrace package is a new package included in the Go (Golang) standard library from Go1.7. The package is extremely useful if you want to monitor the performance of your http requests or if you want to collect and monitor statistics about your http clients. The main idea is the ability to track a set of events that occur within a lifecycle of a http request. For example, dns resolution, tcp connection creation, data written to the tcp connection, data received from the tcp connection and so on.

In this artice we are going to have a look at how to use the httptrace package to monitor the performance of our Go (Golang) http requests.

1. Initialising httptrace.ClientTrace

The main exported type from the httptrace package is the ClientTrace struct type. It is used to define the events we want to listen to. Those events are defined as functions and also known as “hooks”. Let’s have a look at some of them.

This is a stripped-down version of the httptrace.ClientTrace struct

ClientTrace is a set of hooks to run at various stages of an outgoing HTTP request. Any particular hook may be nil

type ClientTrace struct {
    // GetConn is called before a connection is created or
    // retrieved from an idle pool. The hostPort is the
    // "host:port" of the target or proxy. GetConn is called even
    // if there's already an idle cached connection available.
    GetConn func(hostPort string)

    // GotConn is called after a successful connection is
    // obtained. There is no hook for failure to obtain a
    // connection; instead, use the error from
    // Transport.RoundTrip.
    GotConn func(GotConnInfo)

    [...]

    // DNSStart is called when a DNS lookup begins.
    DNSStart func(DNSStartInfo)

    // DNSDone is called when a DNS lookup ends.
    DNSDone func(DNSDoneInfo)

    // ConnectStart is called when a new connection's Dial begins.
    // If net.Dialer.DualStack (IPv6 "Happy Eyeballs") support is
    // enabled, this may be called multiple times.
    ConnectStart func(network, addr string)

    // ConnectDone is called when a new connection's Dial
    // completes. The provided err indicates whether the
    // connection completedly successfully.
    // If net.Dialer.DualStack ("Happy Eyeballs") support is
    // enabled, this may be called multiple times.
    ConnectDone func(network, addr string, err error)

    [...]
}

Each of the functions you see defined in the ClientTrace are the events you want to listen to when creating and sending an http request in Go. Let’s use some of those to track events in our http request example.

package main

import (
    "fmt"
    "log"
    "net/http"
    "net/http/httptrace"
)

func main() {
    req, _ := http.NewRequest(http.MethodGet, "http://example.com", nil)
    clientTrace := &httptrace.ClientTrace{
        GetConn:      func(hostPort string) { fmt.Println("starting to create conn ", hostPort) },
        DNSStart:     func(info httptrace.DNSStartInfo) { fmt.Println("starting to look up dns", info) },
        DNSDone:      func(info httptrace.DNSDoneInfo) { fmt.Println("done looking up dns", info) },
        ConnectStart: func(network, addr string) { fmt.Println("starting tcp connection", network, addr) },
        ConnectDone:  func(network, addr string, err error) { fmt.Println("tcp connection created", network, addr, err) },
        GotConn:      func(info httptrace.GotConnInfo) { fmt.Println("connection established", info) },
    }
}

2. Create a context using httptrace.WithClientTrace

In order to use our client trace instructions (function hooks) we need to attach the struct into a context that is later on passed into the request context and used by the http client. We can do that by using the httptrace.WithClientTrace function

func WithClientTrace(ctx context.Context, trace *ClientTrace) context.Context

WithClientTrace returns a new context based on the provided parent ctx. HTTP client requests made with the returned context will use the provided trace hooks, in addition to any previous hooks registered with ctx. Any hooks defined in the provided trace will be called first.

Here’s what we will need to add to our example

clientTraceCtx := httptrace.WithClientTrace(req.Context(), clientTrace)

3. Use the httptrace.ClientTrace with our HTTP Client

Our final result is the following. Define the client trace hooks, create a context with the client trace and pass that into our request object.

package main

import (
    "fmt"
    "log"
    "net/http"
    "net/http/httptrace"
)

func main() {
    req, _ := http.NewRequest(http.MethodGet, "http://example.com", nil)
    clientTrace := &httptrace.ClientTrace{
        GetConn:      func(hostPort string) { fmt.Println("starting to create conn ", hostPort) },
        DNSStart:     func(info httptrace.DNSStartInfo) { fmt.Println("starting to look up dns", info) },
        DNSDone:      func(info httptrace.DNSDoneInfo) { fmt.Println("done looking up dns", info) },
        ConnectStart: func(network, addr string) { fmt.Println("starting tcp connection", network, addr) },
        ConnectDone:  func(network, addr string, err error) { fmt.Println("tcp connection created", network, addr, err) },
        GotConn:      func(info httptrace.GotConnInfo) { fmt.Println("connection established", info) },
    }
    clientTraceCtx := httptrace.WithClientTrace(req.Context(), clientTrace)
    req = req.WithContext(clientTraceCtx)
    _, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
}

Source code on Go playground https://play.golang.org/p/iJ9HDabEWMu

Expected result

starting to create conn  example.com:80
starting to look up dns {example.com}
done looking up dns {[{93.184.216.34 } {2606:2800:220:1:248:1893:25c8:1946 }] <nil> false}
starting tcp connection tcp 93.184.216.34:80
tcp connection created tcp 93.184.216.34:80 <nil>
connection established {0xc000010010 false false 0s}

These are just a subset of the possible events you can use to track your outgoing HTTP requests in Go. But as you can see from this example this is already extremely useful to track down what’s happening and troubleshooting your http clients

Go httptrace YouTube Tutorial