How To Reuse Http Connections In Go

The HTTP 1.1 protocol supports HTTP Persistent connections, or also known as HTTP Keep-Alive. This allows a client and a server to re-use the same underlying TCP connection when sending multiple HTTP Request/Responses. So instead of establishing a connection for each HTTP Request, the client re-uses the TCP connection previously created more than once. This is particularly useful for performance reasons and when sending multiple requests to the same host.

HTTP connection not reused

Here’s a test example of a series of HTTP requests to the same host. We are going to use the httptrace package to inspect the status of the underlying http connection info. Check more about the httptrace usage in this blogpost https://golang.cafe/blog/golang-httptrace-example.html

package main

import (
    "context"
    "log"
    "net/http"
    "net/http/httptrace"
)

func main() {
    // client trace to log whether the request's underlying tcp connection was re-used
    clientTrace := &httptrace.ClientTrace{
        GotConn: func(info httptrace.GotConnInfo) { log.Printf("conn was reused: %t", info.Reused) },
    }
    traceCtx := httptrace.WithClientTrace(context.Background(), clientTrace)

    // 1st request
    req, err := http.NewRequestWithContext(traceCtx, http.MethodGet, "http://example.com", nil)
    if err != nil {
        log.Fatal(err)
    }
    _, err = http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    // 2nd request
    req, err = http.NewRequestWithContext(traceCtx, http.MethodGet, "http://example.com", nil)
    if err != nil {
        log.Fatal(err)
    }
    _, err = http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
}

The expected output is as follows. You can also view the source code in the Go playground https://play.golang.org/p/0ABI_PYj7MQ

2021/03/29 20:30:18 conn was reused: false
2021/03/29 20:30:18 conn was reused: false

The HTTP underlying tcp connection is not reused even though the requests are made to the same host. This is due to the following reason, specifically with the way we use the response body in our Go HTTP response.

$ go doc http.Response.Body
...
// The default HTTP client's Transport
// may not reuse HTTP/1.x "keep-alive" TCP connections 
// if the Body is not read to completion and closed.
...
Body io.ReadCloser

HTTP connection reused

In order to let the client reuse the underlying connection we just need to read the body entirely and close it before we issue a new HTTP request. Let’s see a practical example.

package main

import (
    "context"
    "io"
    "io/ioutil"
    "log"
    "net/http"
    "net/http/httptrace"
)

func main() {
    // client trace to log whether the request's underlying tcp connection was re-used
    clientTrace := &httptrace.ClientTrace{
        GotConn: func(info httptrace.GotConnInfo) { log.Printf("conn was reused: %t", info.Reused) },
    }
    traceCtx := httptrace.WithClientTrace(context.Background(), clientTrace)

    // 1st request
    req, err := http.NewRequestWithContext(traceCtx, http.MethodGet, "http://example.com", nil)
    if err != nil {
        log.Fatal(err)
    }
    res, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    if _, err := io.Copy(ioutil.Discard, res.Body); err != nil {
        log.Fatal(err)
    }
    res.Body.Close()
    // 2nd request
    req, err = http.NewRequestWithContext(traceCtx, http.MethodGet, "http://example.com", nil)
    if err != nil {
        log.Fatal(err)
    }
    _, err = http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
}

This now correctly reuses the tcp connection as expected. You can also view the source code in the Go playground https://play.golang.org/p/Ee3Cc77IQvT

2021/03/29 20:31:05 conn was reused: false
2021/03/29 20:31:05 conn was reused: true