protocol buffers - How to create GRPC client directly from protobuf without compiling it into java code

Question

When working with GRPC, we need to generate the gRPC client and server interfaces from our .proto service definition via protocol buffer compiler (protoc) or using Gradle or Maven protoc build plugin.

Flow now: protobuf file -> java code -> gRPC client.

So, is there any way to skip this step?

How to create a generic GRPC client that can call the server directly from the protobuf file without compile into java code? Or, is there a way to Generated Code at runtime?

Flow expect: protobuf file -> gRPC client.

I want to build a generic gRPC client system with the input are protobuf files along with description of method, package, message request ... without having to compile again for each protobuf.

Thank you very much.

Answer 1

Protobuf systems really need protoc to be run. However, the generated code could be skipped. Instead of passing something like --java_out and --grpc_java_out to protoc you can pass --descriptor_set_out=FILE which will parse the .proto file into a descriptor file. A descriptor file is a proto-encoded FileDescriptorSet. This is the same basic format as used with the reflection service.

Once you have a descriptor, you can load it a FileDescriptor at a time and create a DynamicMessage.

Then for the gRPC piece, you need to create a gRPC MethodDescriptor.

MethodDescriptor.<DynamicMessage, DynamicMessage>newBuilder()
    // UNKNOWN is fine, but the "correct" value can be computed from
    // methodDesc.toProto().getClientStreaming()/getServerStreaming()
    .setType(MethodDescriptor.MethodType.UNKNOWN)
    .setFullMethodName(MethodDescriptor.generateFullMethodName(
        serviceDesc.getFullName(), methodDesc.getName()))
    .setRequestMarshaller(ProtoUtils.marshaller(
        DynamicMessage.newBuilder(methodDesc.getInputType()).buildPartial()))
    .setResponseMarshaller(ProtoUtils.marshaller(
        DynamicMessage.newBuilder(methodDesc.getOutputType()).buildPartial()))
    .build();

At that point you have everything you need and can call Channel.newCall(method, CallOptions.DEFAULT) in gRPC. You're also free to use ClientCalls to use something more similar to the stub APIs.

So dynamic calls are definitely possible, and is used for things like grpcurl. But it also is not easy and so is generally only done when necessary.