Handling gRPC-to-REST Translation at Scale

Problem Framing

When REST clients need to reach internal gRPC services without being re-written to speak HTTP/2 and Protocol Buffers, the API gateway becomes the mandatory translation boundary. Placed at the data plane, it must deserialize JSON payloads into binary protobuf messages, invoke the gRPC upstream over an HTTP/2 connection, then marshal the binary response back to JSON before the REST client’s timeout fires. At modest traffic this is straightforward. Under sustained load the same gateway must manage HTTP/2 stream multiplexing limits, upstream connection pool sizing, and serialization buffer pressure — all simultaneously. Getting any one of those knobs wrong produces a class of failures that looks identical from the outside (cascading 503s) but has completely different root causes and remediation paths.

Prerequisite Concepts

This page assumes familiarity with Protocol Translation Patterns, which covers the gateway’s general role as a format bridge, and with API Gateway Fundamentals & Architecture for data-plane mechanics and request lifecycle. If you are choosing between gateway products for this workload, Kong vs Tyk vs Envoy for Microservices provides a capability matrix covering transcoding support across the major options.

Translation Architecture

The diagram below shows the full transcoding path from REST client to gRPC upstream. Every stage that can fail under load is labelled.

Step ② is where payload size mismatches and unknown field names cause immediate 400 errors. Step ③ is where gRPC status codes must be mapped correctly or downstream clients receive misleading HTTP status codes. Step ④ and the HTTP/2 stream channel are where pool exhaustion and GOAWAY frames originate under sustained concurrency.

Header Normalization and Status Code Mapping

Every unary gRPC call that crosses the transcoder must have its headers normalized in both directions. Incoming REST requests carry Authorization, Content-Type: application/json, and optional Accept-Encoding. The gateway rewrites these before dispatch: Content-Type becomes application/grpc, and grpc-timeout must be derived from the remaining time left in the gateway’s own upstream request timeout — not passed verbatim, since REST clients rarely set it.

On the response path, the gRPC status in the grpc-status trailer must be mapped to an HTTP status code before the REST client sees it. The scaling limits and capacity planning guidance covers how to set alert thresholds on trailer-missing response counts.

If the gateway drops the grpc-status trailer — which can happen with premature connection resets — the REST client receives a 200 with an empty body, a silent failure that is very hard to debug.

Step-by-Step Configuration: Envoy grpc_json_transcoder (Envoy 1.32+)

Envoy’s built-in grpc_json_transcoder filter is the most capable open-source option. It requires a compiled protobuf descriptor file generated from your .proto sources at build time.

Generate the descriptor:

protoc \
  --proto_path=./proto \
  --descriptor_set_out=/etc/envoy/proto/api.pb \
  --include_imports \
  api/v1/user.proto

Envoy static configuration:

# Envoy 1.32+ — static bootstrap
static_resources:
  listeners:
    - name: rest_listener
      address:
        socket_address: { address: 0.0.0.0, port_value: 8080 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_grpc_transcode
                common_http_protocol_options:
                  idle_timeout: 30s
                  max_request_headers_bytes: 8192
                http_filters:
                  - name: envoy.filters.http.grpc_json_transcoder
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder
                      proto_descriptor: "/etc/envoy/proto/api.pb"
                      services:
                        - "com.example.v1.UserService"
                      print_options:
                        add_whitespace: false
                        # Emit zero-value fields so clients can distinguish unset from zero
                        always_print_primitive_fields: true
                        always_print_enums_as_ints: false
                        preserve_proto_field_names: false
                      request_validation_options:
                        reject_unknown_method: true
                        reject_unknown_query_parameters: true
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  clusters:
    - name: grpc_upstream
      connect_timeout: 2s
      # Cap per-connection buffer to prevent memory exhaustion under backpressure
      per_connection_buffer_limit_bytes: 32768
      http2_protocol_options:
        max_concurrent_streams: 100
        initial_stream_window_size: 65536
        initial_connection_window_size: 1048576
        # Keep connections alive so pool reuse is effective
        connection_keepalive:
          interval: 20s
          timeout: 5s
      circuit_breakers:
        thresholds:
          - priority: DEFAULT
            max_connections: 50
            max_pending_requests: 200
            max_requests: 500
      load_assignment:
        cluster_name: grpc_upstream
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address: { address: 10.0.1.10, port_value: 50051 }

Key decisions in this configuration:

• always_print_primitive_fields: true prevents fields like count: 0 or active: false from being silently omitted from the JSON response, which would break REST clients that distinguish missing from zero.
• reject_unknown_query_parameters: true returns 400 for stale query params rather than silently ignoring them after a proto schema migration.
• per_connection_buffer_limit_bytes: 32768 caps the in-memory buffer per connection. Raise this only if you have large streaming responses and have confirmed available heap headroom.
• Circuit breaker max_pending_requests: 200 triggers 503 before the upstream queue grows unbounded and latency balloons.

Step-by-Step Configuration: Kong Gateway 3.x (grpc-gateway plugin)

Kong’s grpc-gateway plugin (bundled since Kong Gateway 2.0) reads a .proto file annotated with google.api.http bindings and handles transcoding at the plugin layer. For teams already running Kong for authentication proxying and token validation, adding gRPC transcoding requires no additional infrastructure — it reuses existing routes, rate-limiting, and key-auth plugins.

Annotate your proto with HTTP bindings:

syntax = "proto3";
package com.example.v1;

import "google/api/annotations.proto";

service UserService {
  rpc GetUser (GetUserRequest) returns (User) {
    option (google.api.http) = {
      get: "/api/v1/users/{id}"
    };
  }
  rpc CreateUser (CreateUserRequest) returns (User) {
    option (google.api.http) = {
      post: "/api/v1/users"
      body: "*"
    };
  }
}

Kong declarative configuration (deck / Kong 3.x):

# Kong Gateway 3.x — deck format
_format_version: "3.0"

services:
  - name: grpc_user_service
    protocol: grpc
    host: 10.0.1.10
    port: 50051
    connect_timeout: 2000   # ms
    write_timeout: 30000    # ms
    read_timeout: 30000     # ms
    routes:
      - name: rest_to_grpc_users
        protocols: ["http", "https"]
        paths: ["/api/v1/users"]
        strip_path: false
        methods: ["GET", "POST"]

plugins:
  - name: grpc-gateway
    service: grpc_user_service
    config:
      proto: /usr/local/kong/proto/api.proto

  # Rate-limit REST consumers before they saturate the gRPC pool
  - name: rate-limiting
    service: grpc_user_service
    config:
      minute: 1000
      policy: redis
      redis_host: redis.internal
      redis_port: 6379

Kong’s grpc-gateway plugin does not expose buffer or stream concurrency settings directly. Control request body size via the client_max_body_size Nginx directive in kong.conf (client_max_body_size 4m). For upstream connection pool limits, set upstream_keepalive_max_requests and upstream_keepalive_pool_size in kong.conf rather than in the declarative config.

Decision Matrix

Gotchas and Failure Signals

HTTP/2 stream exhaustion is the most common production failure. When the count of in-flight REST requests exceeds max_concurrent_streams on the upstream connection, new streams queue. Envoy emits envoy_cluster_upstream_rq_pending_overflow when max_pending_requests is breached. The observable signal is a sudden spike in 503 responses with no corresponding upstream error in the gRPC service logs — the gateway is shedding requests before they reach the upstream.

GOAWAY frames from the gRPC upstream signal that it is closing the HTTP/2 connection gracefully (e.g., after a rolling restart). If the gateway does not drain in-flight streams before reusing the connection, those requests fail silently. Configure upstream keepalive_timeout to be shorter than the upstream service’s own graceful shutdown window.

Trailer-dropped 200s occur when the gateway receives a reset stream from the upstream before the gRPC status trailer is sent. The transcoder has no status to map, so it emits 200 with an empty body. These show up as mysteriously empty REST responses and can be identified by cross-correlating grpc_server_handled_total{grpc_code!="OK"} with gateway access logs showing 200 for the same request IDs.

Proto descriptor staleness causes 404 or 400 for newly added RPCs that the gateway has not yet learned about. The gap exists between when the upstream service is deployed with new proto methods and when the gateway’s descriptor is reloaded. Always roll out the gateway configuration first (new descriptor, existing RPCs still present), then roll out the upstream service.

always_print_primitive_fields false (Envoy’s default) causes REST clients that check for false or 0 values to miss them, since protobuf omits zero-value scalar fields in binary encoding and the transcoder follows the same rule by default. Enable this option from day one — retrofitting it later is a breaking change for consumers who coded around the missing-field behavior.

grpc-accept-encoding vs Accept-Encoding: gRPC uses grpc-encoding for message-level compression (gzip, snappy). HTTP REST uses Accept-Encoding for transfer-level compression. The transcoder must not conflate these. Passing a REST client’s Accept-Encoding: gzip header verbatim to the gRPC upstream triggers message-level gzip on the protobuf frame, which the transcoder then has to decompress before JSON marshalling. This doubles the CPU cost. Strip Accept-Encoding from the upstream request and let the gRPC service use its own encoding negotiation.

Validation Checklist

• Protobuf descriptor compiled with --include_imports and committed to the gateway config repo alongside the upstream service version it matches
• always_print_primitive_fields: true set in Envoy’s transcoder, or equivalent option verified in Kong plugin
• reject_unknown_query_parameters: true and reject_unknown_method: true active to surface schema drift early
• Circuit breaker max_pending_requests set at or below the gRPC upstream’s own max_concurrent_streams limit
• per_connection_buffer_limit_bytes set and verified against peak payload size in load tests
• gRPC status → HTTP status mapping validated for all non-OK codes the upstream can return (especially RESOURCE_EXHAUSTED → 429)
• grpc-trace-bin → W3C traceparent mapping active and verified end-to-end in distributed trace UI
• Load test run at 80% of max_concurrent_streams; p99 latency overhead confirmed below 30% of SLA budget
• Blue/green rollout procedure documented: gateway descriptor updated before upstream service, not after
• Kong: client_max_body_size set in kong.conf; Envoy: max_request_headers_bytes set at listener level

FAQ

Does gRPC-to-REST transcoding add significant latency?

Expect 15–30% overhead on small payloads from protobuf serialization and JSON marshalling. This shrinks proportionally on larger payloads where HTTP framing overhead dominates. Tune per_connection_buffer_limit_bytes and enable keep-alive on upstream connections to recover most of that budget.

Can Envoy’s grpc_json_transcoder handle server-streaming RPCs?

Yes. Each streaming frame is transcoded to a separate JSON object and flushed as chunked transfer encoding or HTTP/2 DATA frames. Clients must read the response as a newline-delimited JSON stream rather than a single response body.

What happens when the .proto descriptor changes while the gateway is live?

Envoy requires a full reload or control-plane xDS push to pick up a new proto_descriptor. During the gap between deployment and reload, requests that hit changed or removed RPCs return 404 or 503. Use a blue/green deployment of the transcoder configuration alongside the upstream service rollout.

Up: Protocol Translation Patterns

Related:

• Gateway Selection Criteria — evaluate which gateway fits your transcoding requirements before committing to Envoy or Kong
• Scaling Limits & Capacity Planning — size connection pools and circuit breaker thresholds for gRPC upstream clusters
• Request & Response Transformation — header rewriting and payload mutation patterns that complement protocol transcoding