Go gRPC simple service Asynchronous and Synchronous explanation

Question

I am trying to understand GoLang "Go" together with gRPC and to make a simple service scalable.

Lets say I have a client1 that calls a service1(adds numbers) that calls service2(determines if the result is prime), and service2 returns the result to service1 that returns the result to client1 all via gRPC.

When I use protocol buffers "proto3" and generate the Go code via protoc. I get generated methods that call the service in one particular way. I see no distinction to call the methods asynchronously "Go".

And the underlying call seems to be "Invoke" which I believe is synchronous,the call returns once a result is received.

How do I make service1 "performant", I know I can run this in a cluster and have copies, but that would mean I can only serve clients as per the amount of instances within the cluster.

I want a "single" service to be able to serve multiple clients(e.g. 1000) .

Here is a simple server and I am not sure if this is performant or not: I do know that the getprime function does dial every time, and this could probably be moved to make this dial persist and be re-used; But more importantly I want to make a simple performant scaleable service and get a good understanding.

(A) Perhaps the whole design is incorrect and the service1 should just return as soon as the instruction is received "ack", do the addition and send the next request to sercice2 which determines if the answer is prime or not; again service2 just responds with an acknowledgement of the request being received. Once prime is determined by the service2 a call is made to the client with an answer.

If (A) above is the better approach, then still please explain the bottlenecks below; what happens when multiple clients are processed? The call to "Listen" does what, "blocks, or does not block", etc.

package main

import (
    pb "demo/internal/pkg/proto_gen/calc"
    "fmt"
    "golang.org/x/net/context"
    "google.golang.org/grpc"
    "google.golang.org/grpc/reflection"
    "log"
    "net"
)

const (
    port = ":8080"
)

type service struct {
}

func (s *service) Calculate(ctx context.Context, req *pb.Instruction) (*pb.Response, error) {

    var answer float64
    answer = req.Number1 + req.Number2

    // call service prime
    p := getprime(int(answer))
    pa := pb.PrimeAnswer{Prime: p}
    return &pb.Response{Answer: answer, Prime: &pa}, nil
}

const (
    primeAddress = "127.0.0.1:8089"
)

func getprime(number int) bool {
    conn, err := grpc.Dial(primeAddress, grpc.WithInsecure())
    if err != nil {
        log.Fatalf("Did not connect to prime service: %v", err)
    }
    defer conn.Close()

    client := pb.NewPrimeServiceClient(conn)
    p := pb.PrimeMessage{"", float64(number)}

    r, err := client.Prime(context.Background(), &p)
    if err != nil {
        log.Fatalf("Call to prime service failed: %v", err)
    }
    return r.Prime
}

func main() {
    lis, err := net.Listen("tcp", port)
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    s := grpc.NewServer()
    pb.RegisterCalculatorServer(s, &service{})
    reflection.Register(s)
    if err := s.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}

Solution 1

Thanks for your question. It is true that gRPC-Go is sync only; that is your Unary RPC(the one in your example) will return only when the RPC has finished (got a response from the server).

About performance:

1. The Dial operation establishes an underlying connection which may be expensive. So it not wise to do it every time getprime is called. A better way is to create a client, keep it around and make calls to the prime server on it. This way only first RPC incurs the cost of connection.
2. For each RPC request a server gets we launch a goroutine to process that request. So in general, this should scale fairly well.

About (A): It is not uncommon for a service handler to make an RPC call to yet another server and wait for its response before returning back. Note that there's no way for a server to make call to the client.

Solution 2

To phrase what JimB said as an answer: "Synchronous" means that the function that makes the remote call waits for a reply before continuing, not that the whole server or client does. The server is normally multithreaded, even when processing synchronous calls; it can accept and work on a second call while it's responding to the first.

And similarly, if a client has multiple concurrent tasks that each have a gRPC call running, that won't block the process. Clients like that could include net/http servers serving end users, or gRPC servers handling multiple RPCs.

Where you might add explicit go statements is if you want to do something else from the specific function making the RPC call. For example, if you want to issue several RPC calls at once then wait for all their results to come in, you could write code following the examples of fan-out calls.

Comments

[Wayne]

Thanks for the explanation: It is getting much clearer. Please verify my understanding: (1) method "func (s *service) Calculate(ctx context.Context, req *pb.Instruction)" = If we add a deliberate sleep of 10 minutes if the number is prime, clients that do get results that are not prime are not delayed by previous prime requests not processed yet? (2) "the function that makes the remote call waits for a reply before continuing" = the client side call to the server (gRPC service). (3) "The server is normally multithreaded" = the server does not block while requests are being processed.

[Wayne]

Can you please elaborate on point (2). Please correct my understanding ... Does this mean that the "net/network server" creates a go routine for each client that connects that calls the gRPC stub implementation ["func (s *service) Calculate(ctx contex...."]. These "threaded" calls do not block each other, also there is no synchronous blocking required. The gRPC stub methods have their own "stack of local variables". These gRPC method calls are isolated and do not effect each other: not blocking each other and no locally shared state to worry about.

[twotwotwo]

1) Yes--easy for you to write a test for this. 2) Yes, the wait's in the client. 3) Yes, one pending request does not block the server from processing others. If the terminology here remains confusing, you might want to look at materials about threads, like these lecture notes, the wiki page, the Tour of Go's bits on goroutines, or Go by Example starting at Goroutines.

[twotwotwo]

Looking at others' code (e.g. on GitHub) and writing more test programs may also help, especially when the question is more "what is library XYZ's concurrency model?" than "how do the concurrency primitives work in general?"

[twotwotwo]

Your comment is basically right, although you will likely have "shared state to worry about" if you make your server do something more complex, e.g. if you add global in-memory caches anywhere. Folks use tools from the sync module to coordinate access to shared data.

[twotwotwo]

I want to emphasize you'll need to learn to run your own tests to get far with this kind of thing. Writing a test program to see "what happens if I add time.Sleep(10*time.Second) here?" can get an answer in minutes, versus waiting overnight for someone to answer here--having to wait overnight sounds like what people had to do in the old mainframe days!

[Mahak Mukhi]

Adding to @twotwotwo: Yes that's correct. However, keep in mind connecting to a server and making an RPC are two different things and should be thought of as differently. Not every call to the gRCP stub will result in establishing a connection. But other than that your understanding is right, each call to the service handler on server-side happens from a different goroutine.