77. gRPC on Kubernetes with LoadBalancer

These days, gRPC has become one of the go-to choices for service-to-service communication in modern microservices architectures. Some of its key strengths are efficiency, a structured API contract, and support for a wide range of programming languages. Running gRPC in a Kubernetes environment, however, brings its own unique challenges—especially when we want to expose a gRPC service to the outside world. This is exactly where the LoadBalancer plays a vital role.

In this article, I’ll walk through the best practices for running a gRPC service on Kubernetes using a LoadBalancer service type, complete with deployment examples, traffic simulation, diagrams, and engineering insights—all framed from the perspective of a professional engineer.

Why Use gRPC on Kubernetes?

At its core, Kubernetes provides built-in mechanisms for service discovery and internal load balancing. But gRPC has some particular characteristics that set it apart from plain HTTP/REST:

• gRPC uses HTTP/2 as its transport, which features multiplexed streaming, binary framing, and persistent connections.
• Many default Ingress Controllers do not support the end-to-end HTTP/2 that gRPC requires.
• Exposing a service externally usually calls for an L4 (TCP) LoadBalancer rather than an L7 (HTTP) one, so that gRPC communication can run optimally.

This scenario is common when you need to expose a gRPC service publicly (for example, to a third party or a mobile client), or when building a hybrid cloud setup where the clients live outside the cluster.

gRPC Service Flow Diagram on Kubernetes

To make things clearer, take a look at the following flow:

flowchart LR
    A[gRPC Client] -- koneksi secure HTTP/2 --> B(LoadBalancer Service)
    B -- forward TCP --> C(gRPC Server Pod)
    C -- response binary/grpc --> A

flowchart LR
    A[gRPC Client] -- koneksi secure HTTP/2 --> B(LoadBalancer Service)
    B -- forward TCP --> C(gRPC Server Pod)
    C -- response binary/grpc --> A

The diagram above shows that traffic from the client is forwarded directly to the gRPC pod through the LoadBalancer service. The benefit? The full HTTP/2 feature set is preserved, with no termination or interception by an L7 proxy.

Simulation: Deploying a gRPC Service with LoadBalancer

To demonstrate, we’ll deploy a sample gRPC server on Kubernetes and expose it using a Service of type LoadBalancer.

1. A Simple gRPC Application

Let’s take a simple gRPC server written in Node.js for the demonstration. For example, the file greeter_server.js:

 1const grpc = require('@grpc/grpc-js');
 2const protoLoader = require('@grpc/proto-loader');
 3const packageDefinition = protoLoader.loadSync('greeter.proto');
 4const proto = grpc.loadPackageDefinition(packageDefinition);
 5
 6function sayHello(call, callback) {
 7  callback(null, { message: 'Hello ' + call.request.name });
 8}
 9
10const server = new grpc.Server();
11server.addService(proto.Greeter.service, { sayHello: sayHello });
12server.bindAsync('0.0.0.0:50051', grpc.ServerCredentials.createInsecure(), () => {
13  server.start();
14});

greeter.proto:

 1syntax = "proto3";
 2service Greeter {
 3  rpc SayHello (HelloRequest) returns (HelloReply) {}
 4}
 5message HelloRequest {
 6  string name = 1;
 7}
 8message HelloReply {
 9  string message = 1;
10}

2. Dockerfile for Containerization

1FROM node:18
2WORKDIR /app
3COPY . .
4RUN npm install
5EXPOSE 50051
6CMD ["node", "greeter_server.js"]

Build and push the image to a registry (for example: gcr.io, Docker Hub).

3. Kubernetes Deployment Manifest

grpc-deployment.yaml:

 1apiVersion: apps/v1
 2kind: Deployment
 3metadata:
 4  name: grpc-greeter
 5spec:
 6  replicas: 2
 7  selector:
 8    matchLabels:
 9      app: grpc-greeter
10  template:
11    metadata:
12      labels:
13        app: grpc-greeter
14    spec:
15      containers:
16        - name: greeter
17          image: <IMAGE_URL> # Replace with your image
18          ports:
19            - containerPort: 50051

4. Service of Type LoadBalancer

grpc-service.yaml:

 1apiVersion: v1
 2kind: Service
 3metadata:
 4  name: grpc-greeter-service
 5spec:
 6  type: LoadBalancer
 7  selector:
 8    app: grpc-greeter
 9  ports:
10    - protocol: TCP
11      port: 50051
12      targetPort: 50051

Notes:

• The Service uses the LoadBalancer type so that it can obtain an external/public IP.
• The port must match exactly the one used by gRPC in the pod (50051/TCP).

Apply both resources:

1kubectl apply -f grpc-deployment.yaml
2kubectl apply -f grpc-service.yaml

Explaining the Traffic Flow

Once the service is deployed, the cloud provider (such as GKE, EKS, or AKS) will provision an external IP. Traffic from the gRPC client then flows as follows:

1. The client opens a gRPC connection to <External-IP>:50051
2. The LoadBalancer forwards the raw TCP connection to one of the backend pods (round-robin)
3. The gRPC pod receives the request, processes it, and sends back a response
4. The response is returned along the same path

Request Routing Simulation Table

The illustration above shows how the LoadBalancer balances TCP connections across the pods.

Challenges and Best Practices for gRPC LoadBalancer on Kubernetes

1. Ensure End-to-End HTTP/2

• A cloud LoadBalancer (for example, AWS Classic ELB or a Layer 4 TCP load balancer) forwards TCP packets in a connection-aware manner.
• Do not use an L7 Ingress (Nginx, ALB, GCLB HTTP) if it doesn’t support HTTP/2 passthrough! Most of them only support HTTP/1.1. So the simplest path is to publish an L4 LoadBalancer.
• Alternative: use an NGINX TCP LoadBalancer or the Kubernetes Gateway API, which support TCP passthrough.

2. Pod Health Checks

Use a gRPC-based readinessProbe so that a pod only receives traffic when gRPC is truly ready. Since Kubernetes 1.23+, the readinessProbe has supported gRPC:

1readinessProbe:
2  grpc:
3    port: 50051

3. Network Policy & Security

• Enable a NetworkPolicy so that only trusted clients can access the LoadBalancer.
• For public access, use TLS with gRPC. Don’t use ServerCredentials.createInsecure() in production.
• Include a TLS cert (self-signed or from a CA) and enable a secure channel.

4. Autoscaling & Service Capacity

With the LoadBalancer type, scaling pod replicas is easy. Even so, keep the following in mind:

• Attach a horizontal pod autoscaler (HPA) based on resource usage (CPU/RAM/connections).
• Adjust Session Affinity if your gRPC client needs sticky sessions.
• Monitor LoadBalancer metrics (concurrent connections, errors, latency).

FAQ and Practical Insights

Q: Can I use a regular Ingress Controller?
A: Not always. HTTP-based Ingress typically terminates HTTP at L7, which only supports HTTP/1.1. gRPC requires end-to-end HTTP/2. Look for an Ingress Controller that supports it (for example, recent versions of NGINX, Kong, or Istio with special configuration).

Q: What’s the connection limit on a LoadBalancer?
A: It varies by cloud provider. For example, an AWS NLB supports 1 million+ connections, while a GCP LB depends on the backend quota.

Q: What’s the risk of exposing a gRPC service directly?
A: Raw exposure means any client can attempt to connect. Minimize the attack surface using mTLS and a NetworkPolicy.

Conclusion

Running gRPC on Kubernetes with a LoadBalancer is the recommended pattern when exposing a gRPC service from the cluster to the outside world. The main focus is ensuring the HTTP/2 connection stays intact, keeping performance stable, and maintaining security. Understand the limitations of L4 vs L7 routing, keep your health checks in place, and scale pods correctly.

Whatever your backend language, this deployment pattern is highly reusable—from Go and Java to Node.js and Python. Just make sure you choose a LoadBalancer configured for L4, combined with a NetworkPolicy, TLS, and monitoring, for production-ready gRPC on Kubernetes.

Further Reading:

• Kubernetes Docs - gRPC Health Checks
• Kubernetes Services
• gRPC.io

Happy Scaling! 🚀

A Challenge for You:
Already using gRPC on Kubernetes? Try sharing your own unique experience in the comments: how do you handle scaling, have you ever run into trouble with Ingress, or have you tried a service mesh like Istio—let’s discuss!