Part-1: Building a basic microservice with gRPC using Golang

Introduction

Being easy to use, REST API is the current most popular framework among developers for web application development. REST has been used to expose the services to the outer world, and also for internal communication among internal microservices. However, ease and flexibility come with some pitfalls. REST requires very strict Human Agreement, and relies on Documentation. Also, it has been found not so very performant in the case of internal communication and real-time applications. In 2015, gRPC kicked in. gRPC initially developed at Google is now disrupting the industry. gRPC is a modern open-source high-performance RPC framework, which comes with a simple language-agnostic Interface Definition Language (IDL) system, leveraging Protocol Buffers.

Objective

The purpose of this blog is to get you started with gRPC in Go with a simple working example. The blog covers basic information like What, Why, When/Where, and How about the gRPC. We'll majorly focus on the How section, to implement the client and server, to write unittests to test the client and server code separately, and will run the code to establish a client-server communication.

What is gRPC?

gRPC — Remote Procedure Call

  • It enables the server and client applications to communicate transparently and build connected systems
  • gRPC is developed and open-sourced by Google (but no, the g doesn’t stand for Google)

Why use gRPC?

Better Design

  • Ability to auto-generate and publish SDKs as opposed to publishing the APIs for services
  • Advantages of improved features of HTTP/2
    - Multiplexing: This forces service-client to utilize a single TCP connection to simultaneously handle multiple requests.
    - Binary Framing and Compression
  • Server-side streaming RPC
  • Client-side streaming RPC
  • Bidirectional streaming RPC

Where to use gRPC?

The “where” is pretty easy: we can leverage gRPC almost anywhere we have two computers communicating over a network:

  • Client-Server Applications
  • Integrations and APIs
  • Browser-based Web Applications

How to use gRPC?

Our example is a simple “Stack Machine” as a service that lets clients perform operations like, PUSH, ADD, SUB, MUL, DIV, FIBB, AP, GP.

Prerequisites

Go

  • Version 1.6 or higher.
  • For installation instructions, see Go’s Getting Started guide.

gRPC

Use the following command to install gRPC.

~/disk/E/workspace/grpc-eg-go
$ go get -u google.golang.org/grpc

Protocol Buffers v3

~/disk/E/workspace/grpc-eg-go
$ go get -u github.com/golang/protobuf/proto
  • Install the protoc plugin for Go
~/disk/E/workspace/grpc-eg-go
$ go get -u github.com/golang/protobuf/protoc-gen-go

Setting Project Structure

~/disk/E/workspace/grpc-eg-go
$ go mod init github.com/toransahu/grpc-eg-go
$ mkdir machine
$ mkdir server
$ mkdir client
$ tree
.
├── client/
├── go.mod
├── machine/
└── server/

Defining the service

Our first step is to define the gRPC service and the method request and response types using protocol buffers.

service Machine {
...
}
// Execute accepts a set of Instructions from the client and returns a Result.
rpc Execute(InstructionSet) returns (Result) {}
// Result represents the output of execution of the instruction(s).
message Result {
float output = 1;
}

Generating client and server code

We need to generate the gRPC client and server interfaces from the machine/machine.proto service definition.

~/disk/E/workspace/grpc-eg-go
$ SRC_DIR=./
$ DST_DIR=$SRC_DIR
$ protoc \
-I=$SRC_DIR \
--go_out=plugins=grpc:$DST_DIR \
$SRC_DIR/machine/machine.proto
~/disk/E/workspace/grpc-eg-go
$ tree machine/
.
├── machine/
│ ├── machine.pb.go
│ └── machine.proto

Server

Let’s create the server.

  • Running the Machine gRPC server: Run the server to listen for requests from clients and dispatch them to the right service implementation.
type MachineServer struct{}// Execute runs the set of instructions given.
func (s *MachineServer) Execute(ctx context.Context, instructions *machine.InstructionSet) (*machine.Result, error) {
return nil, status.Error(codes.Unimplemented, "Execute() not implemented yet")
}

Implementing Simple RPC

MachineServer implements only Execute() service method as of now - as per Part-1 of this blog series.
Execute(), just gets a InstructionSet from the client and returns the value in a Result by executing every Instruction in the InstructionSet into our Stack Machine.

type Stack []float32func (s *Stack) IsEmpty() bool {
return len(*s) == 0
}
func (s *Stack) Push(input float32) {
*s = append(*s, input)
}
func (s *Stack) Pop() (float32, bool) {
if s.IsEmpty() {
return -1.0, false
}
item := (*s)[len(*s)-1]
*s = (*s)[:len(*s)-1]
return item, true
}
type OperatorType stringconst (
PUSH OperatorType = "PUSH"
POP = "POP"
ADD = "ADD"
SUB = "SUB"
MUL = "MUL"
DIV = "DIV"
)
type MachineServer struct{}// Execute runs the set of instructions given.
func (s *MachineServer) Execute(ctx context.Context, instructions *machine.InstructionSet) (*machine.Result, error) {
if len(instructions.GetInstructions()) == 0 {
return nil, status.Error(codes.InvalidArgument, "No valid instructions received")
}
var stack stack.Stack for _, instruction := range instructions.GetInstructions() {
operand := instruction.GetOperand()
operator := instruction.GetOperator()
op_type := OperatorType(operator)
fmt.Printf("Operand: %v, Operator: %v", operand, operator) switch op_type {
case PUSH:
stack.Push(float32(operand))
case POP:
stack.Pop()
case ADD, SUB, MUL, DIV:
item2, popped := stack.Pop()
item1, popped := stack.Pop()
if !popped {
return &machine.Result{}, status.Error(codes.Aborted, "Invalide sets of instructions. Execution aborted")
}
if op_type == ADD {
stack.Push(item1 + item2)
} else if op_type == SUB {
stack.Push(item1 - item2)
} else if op_type == MUL {
stack.Push(item1 * item2)
} else if op_type == DIV {
stack.Push(item1 / item2)
}
default:
return nil, status.Errorf(codes.Unimplemented, "Operation '%s' not implemented yet", operator)
}
} item, popped := stack.Pop()
if !popped {
return &machine.Result{}, status.Error(codes.Aborted, "Invalide sets of instructions. Execution aborted")
}
return &machine.Result{Output: item}, nil
}

Code to run the gRPC server

To run the gRPC server we need to:

  • To serve our StackMachine service over the gRPC server, we need to register the service with the newly created gRPC server.
var (
port = flag.Int("port", 9111, "Port on which gRPC server should listen TCP conn.")
)
func main() {
flag.Parse()
lis, err := net.Listen("tcp", fmt.Sprintf(":%d", *port))
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
grpcServer := grpc.NewServer()
machine.RegisterMachineServer(grpcServer, &server.MachineServer{})
grpcServer.Serve(lis)
log.Printf("Initializing gRPC server on port %d", *port)
}

Client

As we already know that the same machine/machine.proto file, which is our IDL (Interface Definition Language) is capable of generating interfaces for clients as well. One has to just implement those interfaces to communicate with the gRPC server.

var (
serverAddr = flag.String("server_addr", "localhost:9111", "The server address in the format of host:port")
)
func runExecute(client machine.MachineClient, instructions *machine.InstructionSet) {
log.Printf("Executing %v", instructions)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
result, err := client.Execute(ctx, instructions)
if err != nil {
log.Fatalf("%v.Execute(_) = _, %v: ", client, err)
}
log.Println(result)
}
func main() {
flag.Parse()
var opts []grpc.DialOption
opts = append(opts, grpc.WithInsecure())
opts = append(opts, grpc.WithBlock())
conn, err := grpc.Dial(*serverAddr, opts...)
if err != nil {
log.Fatalf("fail to dial: %v", err)
}
defer conn.Close()
client := machine.NewMachineClient(conn)
// try Execute()
instructions := []*machine.Instruction{}
instructions = append(instructions, &machine.Instruction{Operand: 5, Operator: "PUSH"})
instructions = append(instructions, &machine.Instruction{Operand: 6, Operator: "PUSH"})
instructions = append(instructions, &machine.Instruction{Operator: "MUL"})
runExecute(client, &machine.InstructionSet{Instructions: instructions})
}

Test

Server

Let’s write a unit test to validate our business logic of Execute() method.

  • Write the unit test, it should look like this.
~/disk/E/workspace/grpc-eg-go
$ go test server/machine.go server/machine_test.go -v
=== RUN TestExecute
--- PASS: TestExecute (0.00s)
PASS
ok command-line-arguments 0.004s

Client

To test client-side code without the overhead of connecting to a real server, we’ll use Mock. Mocking enables users to write light-weight unit tests to check functionalities on the client-side without invoking RPC calls to a server.

  • Generate mock for MachineClient
  • Create a test file mock/machine_mock_test.go
  • Write the unit test
~/disk/E/workspace/grpc-eg-go
$ go get github.com/golang/mock/mockgen@latest
~/disk/E/workspace/grpc-eg-go
$ mkdir mock_machine && cd mock_machine
$ mockgen github.com/toransahu/grpc-eg-go/machine MachineClient > machine_mock.go
~/disk/E/workspace/grpc-eg-go
$ go test mock_machine/machine_mock.go mock_machine/machine_mock_test.go -v
=== RUN TestExecute
output:30
--- PASS: TestExecute (0.00s)
PASS
ok command-line-arguments 0.004s

Run

Now we are assured through unit tests that the business logic of the server & client codes is working as expected, let’s try running the server and communicating to it via our client code.

Server

To turn on the server we need to run the previously created cmd/run_machine_server.go file.

~/disk/E/workspace/grpc-eg-go
$ go run cmd/run_machine_server.go

Client

Now, let’s run the client code client/machine.go.

~/disk/E/workspace/grpc-eg-go
$ go run client/machine.go
Executing instructions:<operator:"PUSH" operand:5 > instructions:<operator:"PUSH" operand:6 > instructions:<operator:"MUL" >
output:30
  • How to install all the prerequisites
  • How to define an interface using protobuf
  • How to write gRPC server & client logic for Simple RPC
  • How to write and run the unit test for server & client logic
  • How to run the gRPC server and a client can communicate to it

An OSS Enthusiast