gRPC Connection Pool in Practice: Using grpc-go-pool to Improve Service Performance and Stability

Introduction

gRPC, with its high performance, cross-language support, and bidirectional streaming capabilities, has become one of the preferred protocols for microservice communication. However, while enjoying its convenience, an easily overlooked performance trap is lurking in the code—Connection Management.

Creating a new connection for every gRPC call means repeatedly incurring overhead such as the TCP three-way handshake and TLS negotiation. In high-concurrency scenarios, this leads to:

• Significant increase in latency(tens of milliseconds to hundreds of milliseconds)

• Resource waste(File descriptor, memory)

• Server overload(Handling a large number of handshake requests)

This article will provide an in-depth introduction to github.com/processout/grpc-go-pool This popular gRPC connection pool library in the Go ecosystem helps you elegantly solve the above problems.

Why do we need a connection pool?

What is the overhead of establishing a new connection?

We illustrate this through a simple comparison:

Operation	Time (approx.)	Main expenses
Reuse an existing connection to initiate RPC	1-5ms	Data serialization/deserialization + network transmission
Create a new connection and initiate RPC	20-100ms+	TCP handshake + TLS handshake (if enabled) + the above overhead

In a system with thousands of requests per second, the accumulated latency and resource consumption caused by creating new connections are unacceptable.

The Hidden Dangers of Unrestricted Connections

// Dangerous practice: creating a new connection on every call
func callService(msg string) error {
    conn, _ := grpc.Dial("server:50051", grpc.WithInsecure())
    defer conn.Close() // Close every time, but closing itself also has overhead.
    client := pb.NewServiceClient(conn)
    // ... call
}

The above code under concurrency will cause:

• Port exhaustion: Each connection occupies one local port

• File descriptor exhaustion: The default Linux limit is usually 1024

• Server-side SYN Flood: A large number of handshake requests may trigger the server's protection mechanism.

grpc-go-pool Core Features

Main capabilities provided by this library:

• ✅Connection reuse: Multiple requests share a set of persistent connections

• ✅Connection limit: Prevent unbounded resource growth

• ✅Idle reclamation: Idle connections that exceed the configured time are automatically closed.

• ✅Health check: Detect connection validity via gRPC Keepalive or Ping

• ✅Thread safety Supports concurrent connection acquisition and release.

Quick Start

1. Installation

go get github.com/processout/grpc-go-pool

2. Create a connection pool

package main

import (
    "context"
    "time"

    "github.com/processout/grpc-go-pool"
    "google.golang.org/grpc"
    "google.golang.org/grpc/keepalive"
)

func main() {
    // Define a factory function that creates a new connection
    factory := func() (*grpc.ClientConn, error) {
        return grpc.Dial(
            "localhost:50051",
            grpc.WithInsecure(),
            grpc.WithKeepaliveParams(keepalive.ClientParameters{
                Time:    10 * time.Second, // Send ping every 10 seconds
                Timeout: 3 * time.Second,  // ping timeout duration
            }),
        )
    }

    // Create a connection pool.
    pool, err := grpcpool.New(
        factory,
        2,                // Initial connection count (warm-up)
        10,               // Maximum number of connections
        30 * time.Second, // Idle connection timeout reclamation time
    )
    if err != nil {
        panic(err)
    }
    defer pool.Close() // Release all connections when the program exits.

    // ... use pool
}

3. Borrowing and returning connections

Core mode:Get Use →Close(Return)

func callService(pool *grpcpool.Pool, msg string) error {
    // Get a connection from the pool (blocks until a connection is available)
    connWrapper, err := pool.Get(context.Background())
    if err != nil {
        return err
    }
    // Key: defer to return the connection, not close it
    defer connWrapper.Close()

    // Create a gRPC client
    client := pb.NewYourServiceClient(connWrapper.ClientConn)

    // Initiate an RPC call
    resp, err := client.SayHello(context.Background(), &pb.HelloRequest{Name: msg})
    if err != nil {
        return err
    }

    log.Printf("Response: %s", resp.Message)
    return nil
}

⚠️ Common mistakes:

// ❌ Error: Close ClientConn directly
defer connWrapper.ClientConn.Close()

// ✅ Correct: Return to the pool
defer connWrapper.Close()

4. Connection acquisition with timeout

// Set the timeout for obtaining a connection
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()

connWrapper, err := pool.Get(ctx)
if err != nil {
    if err == context.DeadlineExceeded {
        log.Warn("Timed out obtaining connection, system is busy")
        // Consider a degradation or retry strategy
    }
    return err
}
defer connWrapper.Close()

Configuration Tuning Guide

parameter	Recommended value	Description
Initial connection count	Core concurrency count / 2	Avoid cold start latency, but do not over-preallocate.
Maximum connections	Core concurrency count * 1.5	Leave a buffer, but do not exceed the server-side limit.
idle timeout	30s - 5min	Adjust based on connection keep-alive cost
Get timeout	50ms - 500ms	Set based on business tolerance for latency

Calculation reference example

Assuming service QPS = 1000 and average response time = 10ms, then theoretically:

• Required number of connections ≈ QPS × average response time = 1000 × 0.01 = 10 connections

• Considering peak fluctuations, set 最大连接数 = 20 relatively prudent

Server-side Keepalive Capability Detection (Essential for Team Collaboration)

In actual team collaboration, we often cannot control the server-side configuration. If the client enables Keepalive Ping but the server is not properly configured, it will cause problems such as frequent connection closures and connection pool failure. This section provides three verification methods to help you quickly confirm server capabilities without relying on the operations team.

Method 1: Use grpcurl for quick verification (fastest)

grpcurl It is the most commonly used debugging tool in the gRPC ecosystem, allowing intuitive testing of server-side behavior.

# Test whether the server supports streamless Ping
grpcurl -plaintext localhost:50051 list

Judgment criteria:

• ✅Successfully returned the service list. Server supports Keepalive (connection persistence)

• ❌Return GOAWAY or the connection is closed Server not enabled PermitWithoutStream: true

If the server implements the standard gRPC Health Checking Protocol, it can be verified more precisely:

# Call the health check endpoint
grpcurl -plaintext \
  -d '{"service": ""}' \
  localhost:50051 \
  grpc.health.v1.Health/Check

Expected output:

{
  "status": "SERVING"
}

If returned Unimplemented or the connection is interrupted, indicating that the server may not support the Keepalive or Health interface.

Method 2: Go automated test script (can be integrated into CI)

package main

import (
    "context"
    "fmt"
    "time"

    "google.golang.org/grpc"
    "google.golang.org/grpc/connectivity"
    "google.golang.org/grpc/keepalive"
    "google.golang.org/grpc/health/grpc_health_v1"
)

func testServerKeepalive(addr string) (bool, error) {
    // 1. Create a client and enable Keepalive
    conn, err := grpc.Dial(
        addr,
        grpc.WithInsecure(),
        grpc.WithKeepaliveParams(keepalive.ClientParameters{
            Time:                5 * time.Second,  // Start sending Ping in 5 seconds
            Timeout:             2 * time.Second,
            PermitWithoutStream: true,            // Key: Send Ping even when there is no stream
        }),
    )
    if err != nil {
        return false, fmt.Errorf("connection failed: %w", err)
    }
    defer conn.Close()

    // 2. Wait for Keepalive to take effect
    time.Sleep(6 * time.Second)

    // 3. Check the connection status
    state := conn.GetState()
    if state == connectivity.Ready {
        fmt.Printf("✅ Connection status: %s (Keepalive working normally)\n", state)
        return true, nil
    }

    // 4. Further verification: try calling the Health check
    healthClient := grpc_health_v1.NewHealthClient(conn)
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()

    resp, err := healthClient.Check(ctx, &grpc_health_v1.HealthCheckRequest{})
    if err == nil && resp.GetStatus() == grpc_health_v1.HealthCheckResponse_SERVING {
        fmt.Println("✅ Server supports Health Check + Keepalive")
        return true, nil
    }

    // 5. If the connection is still Ready but Health is not implemented, Keepalive is also considered supported.
    if state == connectivity.Ready {
        fmt.Println("⚠️  Connection is normal, but the server does not implement the Health interface")
        fmt.Println("💡 Suggestion: Keep Keepalive enabled, but add application-layer heartbeat")
        return true, nil
    }

    return false, fmt.Errorf("❌ Keepalive test failed, connection state: %s", state)
}

func main() {
    addr := "localhost:50051"
    supported, err := testServerKeepalive(addr)
    if err != nil {
        fmt.Printf("Detection failed: %v\n", err)
        return
    }

    if supported {
        fmt.Println("🎉 Server supports PermitWithoutStream: true")
    } else {
        fmt.Println("⚠️  The server does not support Keepalive; please contact the operations team to enable it")
    }
}

Method 3: Dynamic Degradation Strategy (Recommended for Production Environments)

The most elegant way is to let the client Automatic detection Server-side capabilities and adaptively adjust the configuration:

type ServerCapability struct {
    SupportsKeepalive bool
    SupportsHealth    bool
}

func detectServerCapabilities(addr string) (*ServerCapability, error) {
    cap := &ServerCapability{}

    // 1. First try a connection without Keepalive
    conn, err := grpc.Dial(addr, grpc.WithInsecure())
    if err != nil {
        return nil, err
    }
    defer conn.Close()

    // 2. Check the Health endpoint
    healthClient := grpc_health_v1.NewHealthClient(conn)
    ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
    defer cancel()

    _, err = healthClient.Check(ctx, &grpc_health_v1.HealthCheckRequest{})
    if err == nil {
        cap.SupportsHealth = true
        cap.SupportsKeepalive = true // Health usually means supporting Keepalive
        return cap, nil
    }

    // 3. Actively test Keepalive (using a short timeout)
    testConn, err := grpc.Dial(
        addr,
        grpc.WithInsecure(),
        grpc.WithKeepaliveParams(keepalive.ClientParameters{
            Time:                2 * time.Second,
            Timeout:             1 * time.Second,
            PermitWithoutStream: true,
        }),
    )
    if err != nil {
        return cap, nil
    }
    defer testConn.Close()

    // Waiting for Ping interaction
    time.Sleep(3 * time.Second)

    if testConn.GetState() == connectivity.Ready {
        cap.SupportsKeepalive = true
    }

    return cap, nil
}

// Use in actual business code
func createClientPool(addr string) (*grpcpool.Pool, error) {
    cap, err := detectServerCapabilities(addr)
    if err != nil {
        log.Warn("Capability detection failed, using default configuration")
        return createDefaultPool(addr)
    }

    if cap.SupportsKeepalive {
        log.Info("✅ Server supports Keepalive, enabling Ping")
        return createPoolWithKeepalive(addr)
    } else {
        log.Warn("⚠️  Server does not support Keepalive, falling back to short connection management")
        return createPoolWithoutKeepalive(addr)
    }
}

Team collaboration best practices

Method	Applicable scenarios	difficulty	Recommendation rating
grpcurl	Quick verification and local debugging	⭐	⭐⭐⭐⭐⭐
Go test	CI/CD integration, automated detection	⭐⭐	⭐⭐⭐⭐
Dynamic degradation	Production environment, multi-environment adaptation	⭐⭐⭐	⭐⭐⭐⭐⭐

Suggestion:

• For local development grpcurl Quick validation

• Integrating Go test scripts into CI/CD for automated detection

• The production environment uses a dynamic degradation strategy for automatic adaptation.

Performance Comparison

Test results before and after using a connection pool under 100 concurrent requests (single server):

indicator	No connection pool	Use a connection pool	Improve
Average latency	45ms	8ms	-82%
P99 latency	120ms	25ms	-79%
Server CPU	65%	30%	-54%
Peak number of connections	100+	15	-85%

Measured data

After using a connection pool, system throughput increased from 200 req/s to 800 req/s (a 4x improvement), while resource consumption dropped significantly.

Advanced usage

Custom connection health check

// Check whether the connection is valid via ping
func isHealthy(conn *grpc.ClientConn) bool {
    ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
    defer cancel()
    return conn.Invoke(ctx, "/grpc.health.v1.Health/Check", &healthpb.HealthCheckRequest{}, nil) == nil
}

Monitor connection pool status

type PoolStats struct {
    Available int // Available connections
    InUse     int // Number of connections in use
    Total     int // Total number of connections (including those being created)
}

func getStats(pool *grpcpool.Pool) PoolStats {
    // Obtained through reflection or extension methods (the specific implementation depends on the library version)
    // It is recommended to expose metrics to monitoring systems such as Prometheus.
}

Pitfall avoidance guide

1. Forgetting to return the connection

Use defer connWrapper.Close() is the safest approach, ensuring it can be returned even if a panic occurs.

2. Improper connection pool size configuration

• Too small: Request blocked, timeout increased

• Too big: Resource waste and high server pressure

It is recommended to make adjustments gradually through stress testing.

3. Ignore server-side connection limits

The server usually also has a maximum connection limit (such as grpc.MaxConcurrentStreams), the client pool size should not exceed the server limit.

4. Do not use Keepalive

Enabling Keepalive on idle connections can detect and remove failed connections early, avoiding the use of "dead connections".

grpc.WithKeepaliveParams(keepalive.ClientParameters{
    Time:                10 * time.Second,
    Timeout:             3 * time.Second,
    PermitWithoutStream: true,
})

5. Keepalive is configured, but it is unknown whether the server supports it.

Question: Client enabled PermitWithoutStream: true, but the server was not configured, causing connections to be closed frequently.

Detection method: Refer to the "Server-side Keepalive Capability Detection" section above.

Solution: Use a dynamic degradation strategy to automatically detect server capabilities and adaptively adjust the configuration.

6. Keepalive conflicts with the server side

If the client and server Keepalive configurations do not match, it may cause:

• The server considers the client's Ping too frequent and actively disconnects.

• The client considers the server response too slow and actively disconnects.

Best practices: Unify Keepalive configuration standards within the team and maintain them in the service registry or configuration center.

Summary

grpc-go-pool is a small and elegant connection management library that can significantly improve the performance and stability of gRPC clients. When used correctly, it can:

• Reduce latency by more than 80%

• Reduce resource consumption by more than 50%

• Increase system throughput several times

Key points:

• ✅ Use defer connWrapper.Close() Return the connection

• ✅ Set the pool size reasonably based on business concurrency.

• ✅ Enable Keepalive and health checks

• ✅ Verify whether the server supports Keepalive

• ✅ Monitor connection pool status and adjust configuration promptly

Related Resources:

• grpc-go-pool GitHub

• gRPC Go Official Documentation

• Connection Pool Design Pattern

• gRPC Keepalive Official Documentation