Go gRPC Client Connection Scope and Pooling

Question

Considering the example from the Go gRPC code base:

func main() {
    // Set up a connection to the server.
    conn, err := grpc.Dial(address, grpc.WithInsecure())
    if err != nil {
        log.Fatalf("did not connect: %v", err)
    }
    defer conn.Close()
    c := pb.NewGreeterClient(conn)

    // Contact the server and print out its response.
    name := defaultName
    if len(os.Args) > 1 {
        name = os.Args[1]
    }
    r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: name})
    if err != nil {
        log.Fatalf("could not greet: %v", err)
    }
    log.Printf("Greeting: %s", r.Message)
}

When consuming a gRPC service from another service what should the scope of the connection (conn) be? I assume it should have affinity with the scope of the request being handled be the consumer service, but I have yet to find any documentation around this. Should I be using a connection pool here?

E.G.

1. gRPC consumer service receives request
2. establish connection to gRPC service (either directly or via pool)
3. make n requests to gRPC service
4. close gRPC connection (or release back to the pool)

Solution

From experience, gRPC client connections should be re-used for the lifetime of the client application as they are safe for concurrent use. Furthermore, one of the key features of gRPC is rapid response from remote procedural calls, which would not be achieved if you have to reconnect on every request received.

Nonetheless, it is highly recommended to use some kind of gRPC load balancing along with these persistent connections. Otherwise, a lot of the load may end up on a few long-lived grpc client-server connections. Load Balancing options include:

1. A gRPC connection pool on client side combined with a server side TCP (Layer 4) load balancer. This will create a pool of client connections initially, and re-use this pool of connections for subsequent gRPC requests. This is the easier route to implement in my opinion. See Pooling gRPC Connections for an example of grpc connection pooling on grpc client side which uses the grpc-go-pool library.
2. HTTP/2(Layer 7) load balancer with gRPC support for load balancing requests. See gRPC Load Balancing which gives an overview of different grpc load balancing options. nginx recently added support for gRPC load balancing.

Comments

[Myles McDonnell]

Sure, the example is just taken from the grpc repo to make it crystal clear what I mean by a connection. There must be some documentation somewhere around how to use a connection...for example is it safe for concurrent use?

[Marc]

They are safe for concurrent use. You have a point though, that could use some documentation. It seems you're not the only one to think so.