63. The gRPC Health Checking Standard

63. The gRPC Health Checking Standard: A Modern Approach for Microservices Reliability

When building distributed systems based on microservices, making sure every service is running properly is crucial. The failure of a single service can ripple out and affect the entire system. That’s why health checks are such an important foundation. But how do you perform health checks on a gRPC-based service, the modern RPC protocol favored by engineers?

In this article, I’ll walk through the gRPC Health Checking standard, covering the concept, implementation, simulation, code examples, and best practices that have been widely adopted across the industry.

Why Health Checks Matter in gRPC

In traditional HTTP/REST applications, health checks are usually fairly simple: you expose an endpoint like GET /healthz that returns the service status (200 OK, 503 Service Unavailable, and so on). A load balancer or orchestrator such as Kubernetes then uses that endpoint to monitor and redeploy services that have failed.

With a gRPC architecture, however, the service only exposes a binary port rather than an HTTP endpoint. There’s no universal standard for health checks the way there is with REST APIs. Without a health check, the orchestrator has no way of knowing when to replace or recover a failing service instance.

This is exactly where the gRPC Health Checking Protocol (or Health Checking Standard) comes into play.

The gRPC Health Checking Standard

The gRPC Health Checking Protocol is an open standard for checking the status of gRPC services. The standard is defined as a service named grpc.health.v1.Health. This service exposes a few simple methods that make it easy for clients (load balancers, sidecars, orchestrators) to check a service’s status.

Protobuf Definition

Here is the health-check service definition according to the standard protocol:

 1syntax = "proto3";
 2
 3package grpc.health.v1;
 4
 5service Health {
 6    // Check the status of a single service
 7    rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
 8
 9    // Stream the status continuously
10    rpc Watch(HealthCheckRequest) returns (stream HealthCheckResponse);
11}
12
13message HealthCheckRequest {
14    string service = 1;
15}
16
17message HealthCheckResponse {
18    enum ServingStatus {
19        UNKNOWN = 0;
20        SERVING = 1;
21        NOT_SERVING = 2;
22        SERVICE_UNKNOWN = 3;  // status for an unknown service, used only by Watch.
23    }
24    ServingStatus status = 1;
25}

1. Check: The client sends a service name and receives a SERVING/NOT_SERVING status.
2. Watch: The client can subscribe to status changes in real time (for example, for a load balancer or sidecar).

Status table:

Health Checking Flow Diagram

Let’s visualize the health check process with the following mermaid diagram:

sequenceDiagram
    participant Orchestrator as Orchestrator / Load Balancer
    participant Service as gRPC Service
    Orchestrator->>Service: Health.Check(serviceName)
    Service-->>Orchestrator: HealthCheckResponse(SERVING / NOT_SERVING)
    Note right of Orchestrator: Decision 
 Based on response

sequenceDiagram
    participant Orchestrator as Orchestrator / Load Balancer
    participant Service as gRPC Service
    Orchestrator->>Service: Health.Check(serviceName)
    Service-->>Orchestrator: HealthCheckResponse(SERVING / NOT_SERVING)
    Note right of Orchestrator: Decision <br/> Based on response

Implementing Health Checks on the Client and Server

Implementation in a Go Server

The gRPC ecosystem provides a standard implementation for health checking, for example in Go (google.golang.org/grpc/health).

1. Server Setup

Suppose you have a service like the one below:

 1package main
 2
 3import (
 4    "context"
 5    "google.golang.org/grpc"
 6    "google.golang.org/grpc/health"
 7    healthpb "google.golang.org/grpc/health/grpc_health_v1"
 8    "net"
 9    "log"
10)
11
12func main() {
13    lis, err := net.Listen("tcp", ":50051")
14    if err != nil {
15        log.Fatalf("failed to listen: %v", err)
16    }
17    s := grpc.NewServer()
18
19    // Create a gRPC health check instance
20    healthServer := health.NewServer()
21
22    // Mark this server as SERVING for its initial status
23    healthServer.SetServingStatus("", healthpb.HealthCheckResponse_SERVING)
24    healthpb.RegisterHealthServer(s, healthServer)
25
26    log.Println("gRPC server with healthcheck running at :50051")
27    if err := s.Serve(lis); err != nil {
28        log.Fatalf("failed to serve: %v", err)
29    }
30}

2. Simulation: Changing the Health Check Status

Suppose that during maintenance you want to declare the service unhealthy (NOT_SERVING). You simply update the status using the SetServingStatus method:

1// Notify the orchestrator and load balancer that the service is under maintenance
2healthServer.SetServingStatus("", healthpb.HealthCheckResponse_NOT_SERVING)

Client: Simulating a Health Check with Go

As a client (for example a load balancer, a Kubernetes probe, etc.), you only need to call the following method:

 1import (
 2    "context"
 3    "google.golang.org/grpc"
 4    healthpb "google.golang.org/grpc/health/grpc_health_v1"
 5    "log"
 6    "time"
 7)
 8
 9func main() {
10    conn, err := grpc.Dial(":50051", grpc.WithInsecure())
11    if err != nil {
12        log.Fatal("fail connect:", err)
13    }
14    defer conn.Close()
15    hc := healthpb.NewHealthClient(conn)
16    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
17    defer cancel()
18
19    res, err := hc.Check(ctx, &healthpb.HealthCheckRequest{Service: ""}) // Empty = the entire server
20    if err != nil {
21        log.Fatalf("could not check health: %v", err)
22    }
23    log.Printf("Health status: %s", res.Status)
24}

Integration with Kubernetes

In Kubernetes, an HTTP probe isn’t an ideal solution for gRPC. With the gRPC health standard, however, Kubernetes >=v1.23 already supports grpc.health/v1 :

1livenessProbe:
2  grpc:
3    port: 50051
4readinessProbe:
5  grpc:
6    port: 50051

The result: Kubernetes queries the service directly via the health protocol, with no need for an extra HTTP endpoint.

Best Practices and Tips

1. Always Expose a Health Service
   Every gRPC service is encouraged to run the standard health check. Many frameworks already automate this for you.

2. Granular Health Checking
   You can use the service name in the request to get more granular status information (for example: database, cache).

3. Dynamic Status
   Update the health status dynamically and periodically, for instance when a connection to a dependency fails (a DB going down, etc.).

4. Use the Watch Stream
   If your environment supports it, take advantage of Watch for real-time notifications of service status.

5. Security Concerns
   For security, expose the health service only on an internal network, or use mTLS between the orchestrator and the service.

Case Study: Health Checks for Multiple Dependencies

Suppose your gRPC service depends on several databases. You can build a checker process like the following (pseudo-code):

 1go func() {
 2    for {
 3        if db.Ping() == nil && redis.Ping() == nil {
 4            healthServer.SetServingStatus("", healthpb.HealthCheckResponse_SERVING)
 5        } else {
 6            healthServer.SetServingStatus("", healthpb.HealthCheckResponse_NOT_SERVING)
 7        }
 8        time.Sleep(time.Second)
 9    }
10}()

Conclusion

The gRPC health checking standard is a modern solution for microservices application reliability. It helps load balancers, orchestrators, and monitoring systems detect failures early without having to add a new HTTP endpoint. With this standard, you can deliver an architecture that is robust, maintainable, and future-proof.

And don’t forget: your application’s reliability is the result of the discipline of consistently applying standards like this one!

References:

• gRPC Health Checking Protocol
• Kubernetes gRPC Probes
• Official Go Implementation

Happy coding, and show the world your healthy microservices! 🚀