A Powerful Tool for gRPC Calls: The Complete Guide to Protoset Files

Introduction

When developing and debugging gRPC services in daily work,grpcurl is one of our most commonly used tools. It is like the ... of the HTTP world curl Just as convenient. However, we often encounter such an awkward scenario:

$ grpcurl -plaintext localhost:50051 list
Failed to list services: server does not support the reflection API

The service has not enabled the reflection API. What should I do?

This article will introduce in detail protoset file concepts, use cases, and best practices, helping you easily handle such problems.

What is a Protoset file?

Protoset file (.protoset) is Protocol Buffers'descriptor set(Descriptor Set), which is a binary file containing complete proto definition metadata.

Core features

• platform-independent: Compatible across macOS/Linux/Windows

• lightweight: Contains only metadata, no executable code

• self-contained: Includes all dependent proto definitions

• portable: Generate once, use everywhere

Working principle

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│  .proto file │ ──→ │   protoc     │ ──→ │ .protoset   │
│             │      │  --descriptor │      │  (binary)    │
│ getentry    │      │  _set_out    │      │             │
│ response    │      │              │      │             │
└─────────────┘      └──────────────┘      └─────────────┘
                                                    │
                                                    ↓
                                            ┌─────────────┐
                                            │  grpcurl    │
                                            │  -protoset  │
                                            └─────────────┘

Why are Protoset files needed?

Scenario 1: Reflection is not enabled for the service

The most common use cases For security or performance reasons, many production gRPC services disable the reflection API.

# In this case, it cannot be called directly.
grpcurl -plaintext localhost:8502 list

# Use protoset files to bypass reflection restrictions
grpcurl -plaintext -protoset ./api.protoset \
  -d '{"name":"test"}' localhost:8502 MyService/GetData

Scenario 2: Cross-Platform Development

Developers write proto on macOS, while the test environment is on Linux:

# Generate protoset on macOS
protoc --descriptor_set_out=./api.protoset --include_imports ./api.proto

# Copy directly to a Linux server for use.
scp api.protoset user@linux-server:/tmp/

Scenario 3: CI/CD Integration

In automated testing and deployment processes, protoset can serve as part of the interface contract:

# GitLab CI example
test:
  script:
    - grpcurl -protoset ./api.protoset -d '{}' grpc-service:50051 Health/Check

Scenario 4: Offline Environment

In isolated environments without external network access, protoset provides a local cache of proto definitions.

Protoset vs Other Solutions

Plan	Advantages	shortcoming	Applicable scenarios
Reflection API	Dynamic discovery, no additional files required	Need service support, security risks	Development/Testing Environment
Proto file	Readable source code, version control	Need to manage dependency paths	There is a complete proto project.
Protoset file	Cross-platform, self-contained, efficient	Requires additional compilation steps	Cross-team collaboration, CI/CD
Stub code	Type safety, IDE support	Requires compilation, language bindings	production business code

How to generate and use Protoset files

1. Generate Protoset file

# Basic usage
protoc --proto_path=. \
  --descriptor_set_out=./service.protoset \
  --include_imports \
  ./your_service.proto \
  ./dependency.proto

# Include all dependencies (recommended)
protoc --proto_path=. \
  --proto_path=./third_party \
  --descriptor_set_out=./service.protoset \
  --include_imports \
  $(find . -name "*.proto")

2. Call Services Using Protoset

# Call a unary RPC
grpcurl -plaintext -protoset ./service.protoset \
  -d '{"field":"value"}' \
  localhost:8502 PackageName.Service/Method

# View service definition
grpcurl -plaintext -protoset ./service.protoset \
  describe PackageName.Service

# View message type
grpcurl -plaintext -protoset ./service.protoset \
  describe PackageName.RequestMessage

# List all services
grpcurl -plaintext -protoset ./service.protoset list

3. Practical Example

Based on real project scenarios:

# Generate protoset
protoc --proto_path=. \
  --descriptor_set_out=./getentry.protoset \
  --include_imports \
  ./getentry.proto ./response.proto

# 2. Verify protoset
grpcurl -plaintext -protoset ./getentry.protoset list

# 3. Call the interface
grpcurl -plaintext -protoset ./getentry.protoset \
  -d '{
    "use_absolute_path": true,
    "path": "/mnt/finalfs/kubernetes/pfs/pvc-xxx"
  }' \
  localhost:8502 Agent.GetEntry/GetEntry

Best practices

Recommended practice

• Version management: Include protoset files in version control (or manage them together with proto)

• CI automation: Automatically generate protoset in the build process

• Naming conventions: Use clear naming, such as <service>_<version>.protoset

• Verify integrity: Use after generation grpcurl list Verification

Things to avoid

• Do not include sensitive information.: protoset contains complete field names and comments

• Pay attention to file size: For large projects, the protoset may be very large; pay attention to optimization.

• Version consistency: Ensure the protoset matches the running service version

Frequently Asked Questions

Q: How are dependencies handled across multiple proto files?

# Use --proto_path to specify the search path
protoc --proto_path=. \
  --proto_path=./vendor \
  --descriptor_set_out=./api.protoset \
  --include_imports \
  ./service.proto ./common.proto

Q: What should I do if the protoset file is too large?

# Contains only the necessary proto files.
protoc --descriptor_set_out=./api.protoset \
  --include_imports \
  ./api.proto  # Do not include all .proto

# Or use compression
protoc --descriptor_set_out=./api.protoset \
  --include_imports \
  --include_source_info=false \  # Exclude source code information
  ./api.proto

Q: How to debug the contents of a protoset?

# View the services contained in the protoset
grpcurl -protoset ./api.protoset list

# View detailed definition
grpcurl -protoset ./api.protoset describe

# Use protoc to decode
protoc --decode_raw < ./api.protoset

Summary

Protoset files are a powerful but often overlooked tool in gRPC development. It solves:

• Invocation issue when reflection is not enabled for the service

• Proto definition distribution for cross-platform development

• API testing automation in CI/CD

• Service invocation in offline environments

Master the use of protoset to make your gRPC debugging more effortless.

Reference materials

• grpcurl GitHub

• Protocol Buffers documentation

• gRPC Reflection Specification