30 Case Study: Streaming Progress Updates
Streaming Progress Updates Using Go and gRPC
In modern backend development, a smooth user experience is a top priority. One of the challenges we frequently run into is how to provide progress updates to the user in real time, especially for long-running backend processes such as parsing, conversion, or importing large datasets.
In this article, we will walk through a case study on implementing Streaming Progress Updates using gRPC Stream in Go (Golang). We will cover the architectural concept, the server and client implementation code, and a simulation example complete with a table and a process-flow diagram.
The Problem
Imagine a desktop or web application that uploads a large file to the backend for processing. The backend takes quite a while, but the user is never told about its progress. This hurts the UX because the user has no idea whether the process is still running or has failed.
The Solution: Streaming Progress via gRPC
gRPC supports server-side streaming, which allows the server to send a continuous stream of data to the client over a single gRPC connection. This is a perfect fit for delivering progress updates.
Architecture Diagram
flowchart LR
Client -->|StartJob RPC| Server
Server -->|Stream Progress| Client
Server -->|Simulasi Proses| Processor
Project Structure
1progress-grpc/
2├── proto/
3│ └── progress.proto
4├── server/
5│ └── main.go
6├── client/
7│ └── main.go1. Protobuf Definition
proto/progress.proto
1syntax = "proto3";
2
3package progress;
4
5service ProgressService {
6 rpc StartJob(JobRequest) returns (stream ProgressResponse);
7}
8
9message JobRequest {
10 string job_id = 1;
11}
12
13message ProgressResponse {
14 int32 percent = 1;
15 string message = 2;
16}Generate it with:
1protoc --go_out=. --go-grpc_out=. proto/progress.proto2. Server Implementation
server/main.go
1package main
2
3import (
4 "context"
5 "fmt"
6 "log"
7 "math/rand"
8 "net"
9 "time"
10
11 pb "progress-grpc/proto"
12 "google.golang.org/grpc"
13)
14
15type server struct {
16 pb.UnimplementedProgressServiceServer
17}
18
19func (s *server) StartJob(req *pb.JobRequest, stream pb.ProgressService_StartJobServer) error {
20 progress := 0
21 for progress < 100 {
22 time.Sleep(1 * time.Second)
23 step := rand.Intn(10) + 5
24 progress += step
25 if progress > 100 {
26 progress = 100
27 }
28 msg := fmt.Sprintf("Progress %d%%", progress)
29 stream.Send(&pb.ProgressResponse{
30 Percent: int32(progress),
31 Message: msg,
32 })
33 }
34 return nil
35}
36
37func main() {
38 lis, err := net.Listen("tcp", ":50051")
39 if err != nil {
40 log.Fatalf("failed to listen: %v", err)
41 }
42 s := grpc.NewServer()
43 pb.RegisterProgressServiceServer(s, &server{})
44 log.Println("gRPC server running on :50051")
45 if err := s.Serve(lis); err != nil {
46 log.Fatalf("failed to serve: %v", err)
47 }
48}3. Client Implementation
client/main.go
1package main
2
3import (
4 "context"
5 "log"
6 "time"
7
8 pb "progress-grpc/proto"
9 "google.golang.org/grpc"
10)
11
12func main() {
13 conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure())
14 if err != nil {
15 log.Fatalf("failed to connect: %v", err)
16 }
17 defer conn.Close()
18
19 client := pb.NewProgressServiceClient(conn)
20 ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
21 defer cancel()
22
23 stream, err := client.StartJob(ctx, &pb.JobRequest{JobId: "abc123"})
24 if err != nil {
25 log.Fatalf("error calling StartJob: %v", err)
26 }
27
28 for {
29 msg, err := stream.Recv()
30 if err != nil {
31 log.Printf("stream ended: %v", err)
32 break
33 }
34 log.Printf("[%d%%] %s", msg.Percent, msg.Message)
35 }
36}Progress Output Simulation
Here is a simulated example of the progress output in the client console:
1[6%] Progress 6%
2[13%] Progress 13%
3[25%] Progress 25%
4[39%] Progress 39%
5[52%] Progress 52%
6[66%] Progress 66%
7[81%] Progress 81%
8[95%] Progress 95%
9[100%] Progress 100%Comparison Table of Approaches
| Approach | Compatibility | Overhead | Responsiveness | Best for |
|---|---|---|---|---|
| REST Polling | High | High | Slow | Lightweight tasks |
| Websocket | Medium | Medium | Fast | Two-way interactivity |
| SSE (EventStream) | High (HTTP) | Low | Medium | Browser, one-way |
| gRPC Stream | gRPC only | Low | Fast | Internal RPC/Streaming |
Conclusion
By leveraging server-side gRPC streaming, we can deliver progress updates efficiently and in real time. This approach is ideal for internal microservice-based applications, CLIs, or desktop clients connected to a Go-based backend.
The case study above can be developed even further:
- Integrating with Redis to track job status across multiple processes.
- Adding a gRPC authentication middleware.
- Using a UUID or tracing span for the jobID.
We hope this case study helps you understand how to implement scalable and effective streaming updates. 🚀