80. Monitoring gRPC with Jaeger + OpenTelemetry
Monitoring gRPC with Jaeger + OpenTelemetry
Monitoring gRPC-based microservices is becoming increasingly important as the complexity of modern application architectures grows — especially when tracing requests across microservices becomes a core requirement for observability and troubleshooting. One powerful combination you can adopt is Jaeger as the distributed tracing backend, paired with OpenTelemetry (OTel) as a cross-language SDK and telemetry collection framework.
In this article, I’ll walk through the practical implementation of monitoring in gRPC applications using OpenTelemetry and Jaeger. We’ll break down the distributed tracing architecture, integrate tracing into the application, simulate tracing for a simple request, and compare the overhead against the benefits of the implementation.
A Quick Look at Jaeger and OpenTelemetry
Jaeger
Jaeger is an open-source distributed tracing platform originally designed by Uber Technologies. It is used to monitor and troubleshoot performance issues in microservices, with key features such as:
- Distributed context propagation
- Tracing
- Node monitoring
- Visualization of traces across services
OpenTelemetry (OTel)
OpenTelemetry is a CNCF project that defines a modern observability standard across platforms and programming languages. OTel supports the collection, transformation, and exporting of metrics, traces, and log data, and can be integrated with several backends such as Jaeger, Prometheus, Grafana, and Elasticsearch.
Architecture: The Tracing Data Flow
Before getting hands-on, let’s look at a simple diagram of the tracing flow with Jaeger + OpenTelemetry.
flowchart TD
subgraph A["Aplikasi (gRPC Client/Server)"]
G1[gRPC Client (OTel SDK)] -->|TraceContext| G2[gRPC Server (OTel SDK)]
end
G2 -->|Batch Trace Data| C[Collector]
C -->|Batch| J[Jaeger Backend]
J --> D[Jaeger UI]
Explanation:
- The application (both client and server) instruments traces through the OpenTelemetry SDK.
- The trace context is automatically injected and propagated within the gRPC request metadata.
- The collector (which can be the otel-collector, or the Jaeger Agent directly) receives batches of traces from the application.
- The trace data flows into the Jaeger backend and is then visualized in the Jaeger UI for root-cause analysis.
Installing Jaeger and the OpenTelemetry Collector
For local demonstration purposes, we’ll run Jaeger via Docker.
1docker run -d --name jaeger \
2 -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
3 -p 6831:6831/udp -p 6832:6832/udp \
4 -p 5778:5778 -p 16686:16686 -p 14268:14268 -p 14250:14250 \
5 jaegertracing/all-in-one:1.39The Jaeger UI can now be accessed at http://localhost:16686.
Hands-on: gRPC Server and Client with OpenTelemetry
A Simple Scheme
We’ll create two services:
user-serviceas the gRPC serverclient-appas the gRPC client
Both will use the OpenTelemetry SDK and export traces to Jaeger.
Dependencies
Installing the packages (example using Go):
1go get go.opentelemetry.io/otel
2go get go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
3go get go.opentelemetry.io/otel/exporters/jaeger
4go get google.golang.org/grpcSetting Up OTel & Jaeger Tracing
A function to initialize the Jaeger exporter and the tracer provider:
1// tracing.go
2package main
3
4import (
5 "go.opentelemetry.io/otel"
6 "go.opentelemetry.io/otel/exporters/jaeger"
7 "go.opentelemetry.io/otel/sdk/trace"
8 "log"
9)
10
11func initTracer() func() {
12 exporter, err := jaeger.New(jaeger.WithCollectorEndpoint(jaeger.WithEndpoint("http://localhost:14268/api/traces")))
13 if err != nil {
14 log.Fatal(err)
15 }
16
17 tp := trace.NewTracerProvider(
18 trace.WithBatcher(exporter),
19 )
20 otel.SetTracerProvider(tp)
21 return func() {
22 _ = tp.Shutdown(nil)
23 }
24}Server: Integrating the OTel gRPC Interceptor
1import (
2 "google.golang.org/grpc"
3 "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
4)
5
6func main() {
7 shutdown := initTracer()
8 defer shutdown()
9
10 s := grpc.NewServer(
11 grpc.UnaryInterceptor(otelgrpc.UnaryServerInterceptor()),
12 )
13 // ... register services & start server
14}Client: Propagating the Trace Context
1import (
2 "google.golang.org/grpc"
3 "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
4)
5
6func main() {
7 shutdown := initTracer()
8 defer shutdown()
9
10 conn, _ := grpc.Dial(
11 "localhost:50051",
12 grpc.WithInsecure(),
13 grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()),
14 )
15 // ... create the stub client and call the user-service endpoint
16}Simulating Tracing
To make the tracing more interesting, add a few manual spans inside the server’s method handler.
1import (
2 "context"
3 "go.opentelemetry.io/otel"
4)
5
6func (s *server) GetUser(ctx context.Context, req *pb.UserRequest) (*pb.UserResponse, error) {
7 tracer := otel.Tracer("user-service")
8 ctx, span := tracer.Start(ctx, "GetUser")
9 defer span.End()
10
11 // Simulate the DB query process
12 time.Sleep(80 * time.Millisecond)
13 // Set a trace attribute
14 span.SetAttributes(attribute.String("user.name", "alvin"))
15
16 return &pb.UserResponse{Id: req.Id, Name: "alvin"}, nil
17}The Result in the Jaeger UI
After the client sends a request to the server, open the Jaeger UI (http://localhost:16686):
- Select the
user-serviceservice. - Look for the most recent trace and click it.
- You’ll see the complete end-to-end trace, from the client call all the way to the handler, complete with a span time breakdown.
Analysis: Monitoring & Overhead
The table below shows a simulation of the overhead that tracing adds to request latency (illustrative figures):
| Configuration | Average Latency (ms) | Notes |
|---|---|---|
| Without Tracing | 100 | gRPC application only |
| With OTel+Jaeger | 110 | OTel exporter active |
| With 5% Sampling | 104 | Low span sampling |
Production Checklist
| Checklist | Practical Tips |
|---|---|
| Sampling Rate | Use probabilistic sampling (e.g., 5%) |
| Context Propagation | Make sure the context propagates to all services |
| Trace Attribute | Add relevant business info (userID, method) |
| Data Retention | Configure Jaeger trace retention (default 7 days) |
| Resource Tag | Use resource tags so traces are easy to filter |
| Security | Keep the Jaeger endpoint from being wide open |
Conclusion
With Jaeger and OpenTelemetry, engineering teams can build deep observability into request flows across services in a gRPC ecosystem. Distributed tracing makes root-cause analysis, bottleneck analysis, and resource profiling possible in real time.
Implementing tracing with OTel is relatively easy for gRPC — it only takes a few extra lines of code — yet it delivers tremendous value for system reliability and monitoring.
Don’t forget to use sampling wisely, and make sure to propagate the trace context across the entire service call chain so that your tracing stays complete and informative.
Happy instrumentation! 🚀