Transfering files with gRPC | Kreya

Skip to main contentKreyaDownloadPricingDocsResourcesBlogRecent posts2026Transfering files with gRPCCatching API regressions with snapshot testing20255 Best API Testing ToolsKreya 1.18 - What's NewComparing the privacy of popular API clientsDemystifying the protobuf wire format - Part 2Import HAR Files in Kreya: Debug APIs & gRPC-WebKreya 1.17 - What's NewKreya 1.16 - What's NewLooking back on 20242024Kreya 1.15 - What's NewDemystifying the protobuf wire formatProtobuf Editions explainedKreya 1.14 - What's NewLooking back on 2023Kreya 1.13 - What's New2023Solving Advent of Code with KreyaKreya 1.12 - What's NewBring your own storageKreya 1.11 - What's NewTesting APIs with KreyaA detailed comparison of REST and gRPCKreya 1.10 - What's NewLooking back on 2022BloomRPC just got deprecatedTransfering files with gRPCJanuary 26, 2026ManuelIs transfering files with gRPC a good idea? Or should that be handled by a separate REST API endpoint?
In this post, we will implement a file transfer service in both, use Kreya to test those APIs, and finally compare the performance to see which one is better.
Challenges when doing file transfers​
When handling large files, it is important to stream the file from one place to another.
This might sound obvious, but many developers (accidentally) buffer the whole file in memory, potentially leading to out-of-memory errors.
For a web server that provides files for download, a correct implementation would stream the files directly from the file system into the HTTP response.
Another problem with very large files are network failures.
Imagine you are downloading a 10 GB file on a slow connection, but your connection is interrupted for a second after downloading 90% of it.
With REST, this could be solved by sending a HTTP Range header, requesting the rest of the file content without download the first 9 GB again.
For the simplicity of the blogpost and since something similar is possible with gRPC, we are going to ignore this problem.
Transfering files with REST​
Handling file transfers with REST (more correctly plain HTTP) is pretty straight forward in most languages and frameworks.
In C# or rather ASP.NET Core, an example endpoint offering a file for downloading could look like this:
[HttpGet("api/files/pdf")]public PhysicalFileResult GetFile(){ return new PhysicalFileResult("/files/test-file.pdf", "application/pdf");}
We are effectively telling the framework to stream the file /files/test-file.pdf as the response.
Internally, the framework repeatedly reads a small chunk (usually a few KB) from the file and writes it to the response.

The whole response body will consist of the file content and Kreya, our API client, automatically renders it as a PDF.
Other information about the file, such as content type or file name, will have to be sent via HTTP headers.
This is important. If you have a JSON REST API and try to send additional information in the response body like this:
{ "name": "my-file.pdf", "created": "2026-01-28", "content": "a3JleWE..."}
This is bad! The whole file content will be Base64-encoded and takes up 30% more space than the file size itself.
In most languages/frameworks (without additional workarounds), this would also buffer the whole file in memory since
the Base64-encoding process is usually not streamed while creating the JSON response.
If the file itself is also buffered in memory, you could see memory usage over twice the size of the file.
This may be fine if your files are only a few KB in size.
But even then, if multiple requests are concurrently hitting this endpoint, you may notice quite a lot of memory usage.
Transfering files with gRPC​
While the REST implementation was straight forward, this is not the case with gRPC.
The design of gRPC is based on protobuf messages.
There is no concept of "streaming" the content of a message. Instead, gRPC is designed to buffer a message fully in memory.
This is the reason why individual gRPC messages should be kept small. The default maximum size is set at 4 MB.
So how do we send large files bigger than that?
While gRPC cannot stream the content of a message, it allows streaming multiple messages.
The solution is to break up the file into small chunks (usually around 32 KB) and then send these chunks until the file is transferred completely.
The protobuf definition for a file download service could look like this:
edition = "2023";package filetransfer;import "google/protobuf/empty.proto";service FileService { rpc DownloadFile(google.protobuf.Empty) returns (stream FileDownloadResponse);}message FileDownloadResponse { oneof data { FileMetadata metadata = 1; bytes chunk = 2; }}message FileMetadata { string file_name = 1; string content_type = 2; int64 size = 3;}
This defines the FileService.DownloadFile server-streaming method, which means that the method accepts a single (empty) request and returns multiple responses.
While we could send the file metadata via gRPC metadata (=HTTP headers or trailers), I think it's nicer to define it explicitly via a message.
The server should send the metadata first, as it contains important information, such as the file size.
A naive server implementation in C# could look like this:
private const int ChunkSize = 32 * 1024;public override async Task DownloadFile(Empty request, IServerStreamWriter<FileDownloadResponse> responseStream, ServerCallContext context){ await using var fileStream = File.OpenRead("/files/test-file.pdf"); // Send the metadata first await responseStream.WriteAsync(new FileDownloadResponse { Metadata = new FileMetadata { ContentType = "application/pdf", FileName = "test-file.pdf", Size = fileStream.Length, } }); // Then, chunk the file and send each chunk until everything has been sent var buffer = new byte[ChunkSize]; int read; while ((read = await fileStream.ReadAsync(buffer)) > 0) { await responseStream.WriteAsync(new FileDownloadResponse { Chunk = ByteString.CopyFrom(buffer, 0, read), }); }}
This works, but has some issues:

A new byte array buffer is created for each request
The buffer is copied each time to create a ByteString

The first point is easily solved by using a buffer from the shared pool, potentially re-using the same buffer for subsequent requests.
The second point happens due to the gRPC implementation in practically all languages.
Since the implementation wants to guarantee that the bytes are not modified while sending them, it performs a copy first.
This is a design decision which favors stability over performance.
Luckily, there is a workaround by using a "unsafe" method, which is perfectly safe in our scenario and improves performance:
private const int ChunkSize = 32 * 1024;public override async Task DownloadFile(Empty request, IServerStreamWriter<FileDownloadResponse> responseStream, ServerCallContext context){ await using var fileStream = File.OpenRead("/files/test-file.pdf"); // Send the metadata first await responseStream.WriteAsync(new FileDownloadResponse { Metadata = new FileMetadata { ContentType = "application/pdf", FileName = "test-file.pdf", Size = fileStream.Length, } }); // Then, chunk the file and send each chunk until everything has been sent using var rentedBuffer = MemoryPool<byte>.Shared.Rent(ChunkSize); var buffer = rentedBuffer.Memory; int read; while ((read = await fileStream.ReadAsync(buffer)) > 0) { await responseStream.WriteAsync(new FileDownloadResponse { Chunk = UnsafeByteOperations.UnsafeWrap(buffer[..read]), }); }}
Let's try this out. After importing the protobuf definition, we call the gRPC method with Kreya:

This works, but where is our PDF?
Since we are sending individual chunks, we need to put them back together manually.
To achieve this, we simply need to append each chunk to a file.
In Kreya, this is done via Scripting:
import { writeFile, appendFile } from 'fs/promises';const path = './preview-pdf.pdf';// Initialize an empty fileawait writeFile(path, '');// Hook to handle each individual gRPC messagekreya.grpc.onResponse(async msg => { if (msg.content.metadata) { // Ignore the metadata for now return; } // Note: The data is only in Base64 here because Kreya encodes them as such for Scripting purposes. // On the network, gRPC transfers the chunk as length-delimeted bytes await appendFile(path, msg.content.chunk, 'base64');});// When we received everything, show the PDFkreya.grpc.onCallCompleted(async () => await kreya.preview.file(path, 'PDF'));
This allows us to view the PDF:

Comparison​
Great! So transfering files with gRPC is definitely possible.
But how do these two technologies compare against each other?
Which one is faster and has less overhead?
Total bytes transferred​
The total amount of bytes transferred on the wire is actually a pretty difficult topic.
It depends on a lot of factors, such as the HTTP protocol (HTTP/1.1, HTTP/2 or HTTP/3),
the package size of TCP/IP, whether TLS is being used etc.
We are going to take a look how this applies to REST and gRPC.
Streaming files over a gRPC connection generates overhead, although not much.
Since gRPC uses HTTP/2 under the hood, each individual chunk message has a few bytes overhead due to the HTTP/2
DATA frame information needed.
Additionally, each chunk needs a few bytes to describe the content of the gRPC message.
You are looking at roughly 15 bytes per message chunk if it fits into one HTTP/2 DATA frame.
Transfering 4 GB of data with a chunk size of 16 KB would need around 250,000 messages to transfer the file completely,
incurring an overhead of ~3.7 MB.
This may or may not be negilible depending on the use case.
Up- or downloading huge files with REST over HTTP/1.1 has less overhead.
Since the bytes of the file are sent as the response/request body, there is not much else that takes up space.
In case of uploads to a server, HTTP multipart requests incur a small overhead cost to define the multipart boundary.
Additionally, HTTP headers and everything else that is needed to send the request over the wire take up space,
but this is the case for all HTTP-based protocols.
Downloading files, whether small or large, have roughly the same amount of bytes overhead with HTTP/1.1.
Depending on the count and size of HTTP headers, this is around a few hundred bytes.
Funnily enough, transfering files with REST over HTTP/2 incurs a bigger overhead.
HTTP/2 splits the payload into individual DATA frames, very similar to our custom gRPC solution.
Each frame, often with a maximum size of 16 KB, has an overhead of 9 bytes.
For a file 4 GB in size, this amounts to ~2.2 MB overhead.
While HTTP/2 has many performance advantages, transfering a single, large file over HTTP/1.1 has less overhead.
In these examples, we omitted the overhead created by TCP/IP, TLS and the lower network layers, which both HTTP/1.1 and HTTP/2 share.
Comparing it with HTTP/3, which uses UDP, would make everything even more complicated, so we leave this as exercise for the reader :)
Performance and memory usage​
I spun up the local server plus client and took a look at the CPU and memory usage.
Please note that this is not an accurate benchmark, which would require a more complex setup.
Nevertheless, it provides some insight into the differences between the approaches.
The example file used was 4 GB in size.
gRPC (naive)gRPC (optimized)REST (HTTP/1.1)REST (HTTP/2)Duration24s22s20s28sMax memory usage36 MB35 MB32 MB35 MBTotal memory allocation4465 MB165 MB38 MB137 MB
If done right, memory usage is no problem for streaming very large files.
And as expected, the naive gRPC implementation which copies a lot of ByteStrings around allocates a lot of memory!
It needs constant garbage collections to clean up the mess.
The maximum memory usage however stays low for all approaches.
What really surprised me was the bad performance of HTTP/2 in comparison to HTTP/1.1.
It was even slower than gRPC, which builds on top of it!
I cannot really explain this huge difference, especially since the code is exactly the same,
both on the client and the server.
I was running these tests on .NET 10 on Windows 11.
The optimized gRPC version performs pretty well, but is still slower than REST via HTTP/1.1.
Since it has to do more work, it takes longer and uses more memory (and CPU).
Optimizing the gRPC code was very important, as the naive implementation allocates so much memory!
I also tested HTTP/1.1 with TLS disabled, but it did not really make a difference.
Conclusion​
A REST endpoint for up- and downloading files is still the best option.
If you are forced to use gRPC or simply too lazy to add REST endpoints in addition to your gRPC services,
transfering files via gRPC is not too bad!
If you use some kind of S3 compatible storage backend, the best options is to generate a presigned URL.
Then, download your files directly from the S3 storage instead of piping it through your backend.
There are lots of points to consider when implementing file transfer APIs.
For example, if your users may have slow networks, it may be useful to compress the data before sending it over the wire.
Should you need resumable up- or downloads, instead of rolling your own, you could use https://tus.io/.
This open-source protocol has implementations in various languages.Older postCatching API regressions with snapshot testingChallenges when doing file transfersTransfering files with RESTTransfering files with gRPCComparisonTotal bytes transferredPerformance and memory usageConclusionCommunityBlogDocumentationGitHubTwitterComparisonsvs Postmanvs Insomniavs BloomRPCvs Brunovs SoapUIvs SwaggerUISupportPricingManage subscriptionFAQDocumentationAboutPrivacy policyTermsriok GmbHMade with ❤️ in Switzerland🇨🇭Copyright © 2026 riok GmbH

The implementation of file transfer services using gRPC and REST offers distinct advantages and disadvantages, demanding careful consideration based on specific requirements. Kreya, an API testing client, facilitates this exploration by offering a framework to test both approaches. This summary details the core concepts, challenges, and comparative performance characteristics of file transfer using gRPC and REST, highlighting the trade-offs involved.

When handling large files, streaming is crucial to avoid overwhelming memory resources. Traditional approaches, characterized by buffering the entire file into memory, can lead to out-of-memory errors. Correct implementations should stream files directly from the file system into the HTTP response. Network failures during large file transfers pose a significant challenge. For instance, a 10 GB file download interrupted after 90% completion necessitates retransmission of the remaining data. RESTful APIs can mitigate this issue using HTTP Range headers, allowing partial downloads without redundant transfers.

However, the simplicity of RESTful file transfer often conflicts with the design of gRPC. gRPC's protocol, built around protobuf messages, lacks inherent streaming capabilities. To address this, file transfers are broken into smaller chunks, typically 32KB, and transmitted sequentially until the entire file is transferred. This approach introduces considerable overhead and complexity compared to REST. The protobuf definition showcases this: a `FileDownloadResponse` message can contain either metadata or a chunk of data. The server first sends the metadata, crucial for managing the download process. This metadata includes details like the file name, content type, and file size.

The naive gRPC implementation suffers from excessive memory allocation due to numerous ByteString copies. This design decision, while ensuring data integrity, severely impacts performance. The optimization strategy leverages "unsafe" methods to mitigate these copying operations. The `MemoryPool<byte>.Shared.Rent()` approach reuses a pre-allocated buffer reducing memory allocation by orders of magnitude. The final gRPC implementation leverages Kreya to seamlessly append each chunk to a file, demonstrating the power of data streaming when combined with robust API testing tools.

Comparing the two approaches reveals significant differences in performance and resource utilization. The sheer volume of bytes transferred via gRPC, coupled with the overhead introduced by protobuf message structure and chunking, creates a noticeable data transfer tax. While gRPC theoretically supports HTTP/2, performance comparisons indicate that HTTP/1.1 often exhibits superior transfer speeds. Ultimately, RESTful file transfer via standard HTTP remains the more efficient option, particularly for large files. This difference is partly due to reduced protocol overhead and simpler implementation.

Memory usage during file transfer also reveals notable contrasts. The naive gRPC implementation consumes a disproportionately large amount of memory, largely due to wasteful copy operations. The optimized version, employing pre-allocated buffers, significantly decreases memory allocation. However, performance limitations still exist, illustrating the importance of efficient design choices.

In conclusion, while gRPC offers a viable solution for file transfer, particularly within gRPC-centric architectures, RESTful APIs remain the preferred option for handling large files due to their inherent performance and efficiency. The technology comparison highlights that the best solution hinges on the context of the broader system and the specific constraints of the application. When incorporating streaming, gRPC can be a powerful option, but careful attention to memory management and protocol limitations is necessary. Tools like Kreya are invaluable in evaluating and testing these approaches to ensure optimal performance and resilience.