74. Integrating gRPC with Envoy Proxy

74. Integrating gRPC with Envoy Proxy

gRPC has become one of the top choices for building fast, efficient, and easily maintainable microservices. However, at production scale, requirements such as load balancing, service discovery, observability, and mTLS often call for a sophisticated proxy—and this is exactly where Envoy Proxy comes in as the solution. In this article, I’ll walk through, in a hands-on way, how to integrate gRPC with Envoy Proxy, covering everything from the concepts and benefits to an implementation example you can try out right away.

Why Should You Use Envoy for gRPC?

Before we dive into the code and simulation, let’s examine the reasoning behind placing Envoy in front of a gRPC server:

• gRPC uses an HTTP/2-based protocol that isn’t always compatible with conventional HTTP proxies.
• Smart load balancing: Envoy supports application-level load balancing and is aware of HTTP/2 connection state.
• Observability: Integrating logging, metrics (Prometheus), and tracing (Jaeger, Zipkin) becomes much easier.
• Security: Envoy can enforce mTLS without requiring any changes to the application.
• Network resilience: It can manage retries, timeouts, circuit breaking, and more.

Components We’ll Build

The simple scenario we’ll simulate:

• Service A: The application client (gRPC client)
• Envoy Proxy: Acting as a reverse proxy and load balancer
• Service B: A gRPC server (hello world)
• All services run locally, either with Docker or by running the binaries directly.

High-Level Architecture

flowchart LR
    subgraph Client Side
        A[gRPC Client]
    end
    subgraph Proxy
        B[Envoy Proxy]
    end
    subgraph Backend
        C[gRPC Server]
    end
    A -- gRPC/HTTP2 --> B
    B -- gRPC/HTTP2 --> C

flowchart LR
    subgraph Client Side
        A[gRPC Client]
    end
    subgraph Proxy
        B[Envoy Proxy]
    end
    subgraph Backend
        C[gRPC Server]
    end
    A -- gRPC/HTTP2 --> B
    B -- gRPC/HTTP2 --> C

1. Creating a Simple gRPC Service

We’ll start by defining a simple service with Protocol Buffers:

helloworld.proto

 1syntax = "proto3";
 2
 3package helloworld;
 4
 5service Greeter {
 6  rpc SayHello (HelloRequest) returns (HelloReply);
 7}
 8
 9message HelloRequest {
10  string name = 1;
11}
12
13message HelloReply {
14  string message = 1;
15}

Generate the stub code using protoc. For Go, for example:

1protoc --go_out=. --go-grpc_out=. helloworld.proto

A simple server implementation (Go):

 1// server.go
 2package main
 3
 4import (
 5    "context"
 6    "log"
 7    "net"
 8
 9    pb "yourmod/helloworld"
10    "google.golang.org/grpc"
11)
12
13type server struct {
14    pb.UnimplementedGreeterServer
15}
16
17func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
18    return &pb.HelloReply{Message: "Hello, " + in.Name}, nil
19}
20
21func main() {
22    lis, err := net.Listen("tcp", ":50051")
23    if err != nil {
24        log.Fatalf("failed to listen: %v", err)
25    }
26    s := grpc.NewServer()
27    pb.RegisterGreeterServer(s, &server{})
28    log.Println("gRPC server listening on :50051")
29    s.Serve(lis)
30}

2. Configuring Envoy Proxy for gRPC

Envoy must be configured to understand the HTTP/2 protocol. We’ll set it up as a reverse proxy from port 9901 to the backend gRPC server on 50051.

envoy.yaml:

 1static_resources:
 2  listeners:
 3    - name: listener_0
 4      address:
 5        socket_address: { address: 0.0.0.0, port_value: 9901 }
 6      filter_chains:
 7        - filters:
 8            - name: envoy.filters.network.http_connection_manager
 9              typed_config:
10                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
11                codec_type: AUTO
12                stat_prefix: ingress_http
13                route_config:
14                  name: local_route
15                  virtual_hosts:
16                    - name: backend
17                      domains: ["*"]
18                      routes:
19                        - match: { prefix: "/" }
20                          route: { cluster: greeter_service }
21                http_filters:
22                  - name: envoy.filters.http.router
23    # Envoy will use HTTP/2 to proxy the gRPC calls.
24  clusters:
25    - name: greeter_service
26      connect_timeout: 0.25s
27      type: logical_dns
28      lb_policy: round_robin
29      http2_protocol_options: {} # Enables HTTP2 for this cluster
30      load_assignment:
31        cluster_name: greeter_service
32        endpoints:
33          - lb_endpoints:
34              - endpoint:
35                  address:
36                    socket_address:
37                      address: 127.0.0.1
38                      port_value: 50051
39admin:
40  access_log_path: "/tmp/admin_access.log"
41  address:
42    socket_address:
43      address: 0.0.0.0
44      port_value: 9900

Key Points:

• http2_protocol_options: {} instructs Envoy to use HTTP/2 when communicating with the backend cluster.
• Make sure the cluster points to the port where the gRPC server is running.
• The Envoy proxy exposes port 9901 (frontend).

3. Creating a Client to Connect to Envoy

The Go client implementation, specifically connecting to Envoy, opens a channel to localhost:9901:

 1// client.go
 2package main
 3
 4import (
 5    "context"
 6    "log"
 7    "time"
 8
 9    pb "yourmod/helloworld"
10    "google.golang.org/grpc"
11)
12
13func main() {
14    conn, err := grpc.Dial("localhost:9901", grpc.WithInsecure())
15    if err != nil {
16        log.Fatalf("did not connect: %v", err)
17    }
18    defer conn.Close()
19    c := pb.NewGreeterClient(conn)
20
21    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
22    defer cancel()
23    r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "Envoy"})
24    if err != nil {
25        log.Fatalf("could not greet: %v", err)
26    }
27    log.Printf("Greeting: %s", r.Message)
28}

Run the server, then Envoy, then the client:

1go run server.go
2envoy -c envoy.yaml
3go run client.go

4. Simulating the Request Flow

Let’s visualize the request flow:

sequenceDiagram
    participant Client
    participant Envoy
    participant Server

    Client->>Envoy: gRPC Request (SayHello)
    Envoy->>Server: gRPC Request (forward)
    Server-->>Envoy: gRPC Response
    Envoy-->>Client: gRPC Response (relay)

sequenceDiagram
    participant Client
    participant Envoy
    participant Server

    Client->>Envoy: gRPC Request (SayHello)
    Envoy->>Server: gRPC Request (forward)
    Server-->>Envoy: gRPC Response
    Envoy-->>Client: gRPC Response (relay)

The benefits:

• The client doesn’t need to know the IP/host of the backend gRPC server
• You can scale the gRPC servers and update the Envoy cluster configuration dynamically
• Observability, circuit breaking, and more can be leveraged right away

5. Case Study: Load Balancing and Health Checking

Imagine we add two server nodes behind Envoy (for example, on ports 50051 and 50052). All you need to do is modify the load_assignment section:

 1load_assignment:
 2  cluster_name: greeter_service
 3  endpoints:
 4    - lb_endpoints:
 5        - endpoint:
 6            address:
 7              socket_address:
 8                address: 127.0.0.1
 9                port_value: 50051
10        - endpoint:
11            address:
12              socket_address:
13                address: 127.0.0.1
14                port_value: 50052

We can also add a health check:

1health_checks:
2  - timeout: 1s
3    interval: 2s
4    unhealthy_threshold: 2
5    healthy_threshold: 2
6    grpc_health_check: {} # Follows the standard gRPC health check

A quick summary table of Envoy’s key features for gRPC integration:

6. Conclusion, Challenges, and Best Practices

By integrating gRPC with Envoy Proxy, your microservices application gains the bonus of scalability, observability, and security without requiring many changes at the application level. That said, there are a few important things to keep in mind:

• Always use health checks on backend clusters
• Monitor Envoy metrics to understand traffic and latency
• Configure rate limiting and circuit breaking appropriately
• For production, make sure Envoy runs with mTLS
• Deployment is best done via container orchestration (Kubernetes, Nomad, etc.) for optimal results

This integration isn’t a “one size fits all” solution, but it offers a solid foundation for a service mesh and a scalable microservices platform. Try out this simple simulation in your local environment, and explore the full potential of Envoy in production!

References

• gRPC Official Docs
• Envoy Proxy gRPC

Happy experimenting! Don’t hesitate to share your insights and questions in the comments section.