> Sadly, capnproto's awesome RPC system doesn't appear to support streaming
Sure it does. You can implement "streaming" in Cap'n Proto by introducing a callback object, and making one RPC call for each item / chunk in the stream. In Cap'n Proto, "streaming" is just a design pattern, not something that needs to be explicitly built in, because Cap'n Proto is inherently far more expressive than gRPC.
That is, you can define a type like:
interface Stream(T) {
write @0 (item :T);
end @1 (); # signal successful end of stream
}
Admittedly, this technique has the problem that the application has to do its own flow control -- it has to keep multiple calls in-flight to saturate the connection, but needs to place a cap on the number of calls in order to avoid excess buffering. This is doable, but somewhat inconvenient.
So I am actually in the process of extending the implementation to make this logic built-in:
Interesting. I do tend to favor protocols that are a bit lower level and more flexible for composing higher-level functionality. Though this does sound pretty complicated to implement using only what capnproto offers right now. Would there be a way to jury rig "request(n)" backpressure as described in reactive streams[0] (also implemented by RSocket[1]) on top of capnproto? That's what I'm using for omnistreams and it's proven very simple to reason implement and reason about.
Sure it does. You can implement "streaming" in Cap'n Proto by introducing a callback object, and making one RPC call for each item / chunk in the stream. In Cap'n Proto, "streaming" is just a design pattern, not something that needs to be explicitly built in, because Cap'n Proto is inherently far more expressive than gRPC.
That is, you can define a type like:
Then you can define streaming methods like: Admittedly, this technique has the problem that the application has to do its own flow control -- it has to keep multiple calls in-flight to saturate the connection, but needs to place a cap on the number of calls in order to avoid excess buffering. This is doable, but somewhat inconvenient.So I am actually in the process of extending the implementation to make this logic built-in:
https://github.com/capnproto/capnproto/pull/825
Note that PR doesn't add anything new to the RPC protocol; it just provides helpers to tell the app how many concurrent calls to make.