1.1 KiB
1.1 KiB
C++ Performance Notes
Streaming write buffering
Generally, each write operation (Write(), WritesDone()) implies a syscall. gRPC will try to batch together separate write operations from different threads, but currently cannot automatically infer batching in a single stream.
If message k+1 in a stream does not rely on responses from message k, it's possible to enable write batching by passing a WriteOptions argument to Write with the buffer_hint set:
stream_writer->Write(message, WriteOptions().set_buffer_hint());
The write will be buffered until one of the following is true:
- the per-stream buffer is filled (controllable with the channel argument GRPC_ARG_HTTP2_WRITE_BUFFER_SIZE) - this prevents infinite buffering leading to OOM
- a subsequent Write without buffer_hint set is posted
- the call is finished for writing (WritesDone() called on the client, or Finish() called on an async server stream, or the service handler returns for a sync server stream)
Completion Queues and Threading in the Async API
Right now, the best performance trade-off is having numcpu's threads and one completion queue per thread.