89. Performance Benchmarking with ghz — A Complete Guide to Measuring gRPC

Performance benchmarking might sound like something only the QA or SRE team cares about, but for me—as a developer—understanding and applying benchmarks is a vital part of building scalable systems. This is especially true when working with gRPC, the modern RPC framework that many companies have adopted for their microservices. One of my favorite tools is ghz , an open-source benchmarker built specifically for gRPC. In this 89th article of the series, I’ll cover benchmarking with ghz in depth, walk through the use cases, and show real examples that you can adapt right away.

Why Do You Need to Benchmark gRPC?

Before getting into the technical details, let’s agree on the motivation:

• gRPC relies on high-performance binary communication (Protocol Buffers), which is different from REST/JSON.
• SLA/SLI requirements such as latency and throughput are crucial in microservices, especially real-time systems.
• There are often bottlenecks in certain services—whether in the sidecar, the network, or in data processing.
• Without measurable benchmarks, scaling and tuning become a matter of “going by feel”.

With a tool like ghz, you can simulate traffic, measure response times, evaluate the RPS a service can handle, and generate benchmark reports that are easy to analyze.

What Is ghz?

First of all, ghz is not part of the gRPC core, nor is it official Google tooling. It’s an open-source project written in Go, and it’s currently one of the de-facto tools for gRPC benchmarking.

Key features:

• Can benchmark unary and streaming gRPC endpoints (client-side and bidirectional).
• Supports custom metadata, TLS, headers, and authentication.
• Multiple output formats: CLI, JSON, HTML, and even Grafana dashboards.
• Easy to automate via CI/CD scripts.

Installation and Setup

1go install github.com/bojand/ghz/cmd/ghz@latest

Make sure you’ve also installed the service/protobuf client for the endpoint you’re going to test. For example, suppose we have the following proto:

 1// helloworld.proto
 2syntax = "proto3";
 3package helloworld;
 4
 5service Greeter {
 6  rpc SayHello (HelloRequest) returns (HelloReply);
 7}
 8
 9message HelloRequest {
10  string name = 1;
11}
12
13message HelloReply {
14  string message = 1;
15}

A Basic Benchmark Example

Let’s benchmark the SayHello endpoint!

1ghz --insecure \
2    --proto ./helloworld.proto \
3    --call helloworld.Greeter.SayHello \
4    --data '{"name":"Medium Reader"}' \
5    -c 50 -n 10000 \
6    localhost:50051

Explanation:

• –insecure: Don’t use TLS.
• –proto: Location of the proto file.
• –call: Fully qualified method.
• –data: Request payload.
• -c 50: 50 concurrent connections.
• -n 10000: 10,000 total requests.

Benchmark Results — Initial Analysis

The output is quite comprehensive. Example (abbreviated):

 1Summary:
 2  Count:     10000
 3  Total:     2.33 secs
 4  Slowest:   0.0121 secs
 5  Fastest:   0.0003 secs
 6  Average:   0.0052 secs
 7  Requests/sec: 4296.75
 8
 9Status code distribution:
10  [OK]   10000 responses (100.00%)

Insights you can draw from this:

Simulation Methods: An Advanced Example

Suppose we want to test with different user scenarios, random data, and a custom header (for example, a JWT):

1. Create an input file with multiple data entries:

1[
2  {"name": "UserA"},
3  {"name": "UserB"},
4  {"name": "UserC"}
5]

2. Use a custom header & token
   bash

   Copy

   1ghz --insecure \
   2    --proto ./helloworld.proto \
   3    --call helloworld.Greeter.SayHello \
   4    --data-file ./data.json \
   5    -c 100 -n 50000 \
   6    --metadata "authorization=Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9" \
   7    localhost:50051

ghz Workflow Diagram (Mermaid)

flowchart TD
    A[Start ghz CLI] --> B[Load proto + Data]
    B --> C[Initiate N concurrent client(s)]
    C --> D[Generate and Send Requests]
    D --> E[Measure Response & Gather Metrics]
    E --> F[Aggregate Results]
    F --> G[Print / Export Report]

flowchart TD
    A[Start ghz CLI] --> B[Load proto + Data]
    B --> C[Initiate N concurrent client(s)]
    C --> D[Generate and Send Requests]
    D --> E[Measure Response & Gather Metrics]
    E --> F[Aggregate Results]
    F --> G[Print / Export Report]

Reading the Report & What’s Next?

Besides the command-line summary, you can also generate an HTML report:

1ghz --insecure \
2    ...[other params as above] \
3    -O report.html

Here’s an example of an important summary table to pay attention to:

Analysis:

• P99 latency is extremely important in real-time/high-concurrency systems.
• If P99 balloons far beyond the average (mean), there may be a “long tail”/outlier problem.
• Use this data to:
  • Tune the thread/connection pool.
  • Scale instances.
  • Fix problematic IO paths or downstream databases.

CI/CD Integration

ghz is a great fit for integrating into a CI/CD pipeline:

• Make sure the gRPC service is already running in the background (using Docker Compose/k8s job).
• Run the benchmark in a metrics or regression stage (it can serve as a smoke test).
• Parse the JSON output and trigger an alert in the pipeline if P99 latency exceeds a threshold.
• Store the report in object storage for historical comparison.

Useful Tips

• Test different endpoints, not just the “happy path”.
• Simulate spikes (stress tests, using a thread/concurrency level much higher than usual).
• Check the impact on server resources (CPU/mem), and correlate it with the benchmark results.
• Tweak the gRPC server/client parameters (maxConcurrentStreams, connection pooling, etc.).

Conclusion

In the era of microservices and cloud-native, benchmarking is a non-negotiable investment. With ghz, we have a lightweight, powerful tool that’s easy to integrate for testing the real-world performance of gRPC services.

Remember: What you don’t measure, you can’t improve.

Happy experimenting, and see you in the next article!