Optimizing dd block size?
For a long time, I have always believed that the parameters for bs
and count
for dd
were just for the convenience of people; as dd just multiplies them and uses the byte value. A month ago, when installing Ubuntu for my mother, I shrunk the partition on the right (never do this, it takes ages ) and saw the gparted
calculation of the "optimal block size".
My question is, does the same idea of optimal "block size" apply to dd
? If so, are there any specific situations in which it is used most strongly? How can I use the block size to my advantage?
a source to share
Is the same idea used regarding the optimal block size for dd?
I think so. Consider a 4kb buffer. It can be filled with a simple mmap operation if the filesystem also has 4k blocks (common to new flash drives). If one stream fills a buffer and then the buffer needs to be written, the input stream blocks until the buffer is flushed. This is the reason for using multiple buffers. On the other hand, my guess is that one buffer needs to be filled first before it is written to the output. Therefore, even if there is already some data, the buffer must be kept in memory. If you are using a 1Gig buffer (or block size), that won't be optimistic either.
I just looked at the buffer size in gparted and it seems like they are just switching between 128kb and 256kb (maybe more complex) and it looks like they want to make most of the caches found on most systems. Given a 2MB disk cache, it might be wise to transfer data in chunks of this size, and those chunks might even go into the processor cache if not using mmapped io.
If so, are there any specific situations in which it is used most strongly?
All operations that transfer a lot of data if the data can be read and written block by block.
How can I use the block size to my advantage?
Calculate it by checking which one is fasting for you. It's that simple and with the above explanations, you can start with something around 256k as a block. Or add autobufsize option to dd :).
a source to share