Ruby zlib dumping massive data
I am trying to use Zlib :: Deflate.deflate on a massive file (4 gigs). There are obvious problems with this, the first of which is that I cannot load the entire file into memory at once. Zlib :: GzipWriter will work as it works with streams, but it is not zlib compression. Any ideas?
+2
source to share
1 answer
You can try creating an instance of the Zlib :: Deflate stream and passing its data from your large file piece by piece. Zlib :: Deflate :: deflate aims to do this behind the scenes.
It will look something like this:
z = Zlib::Deflate.new
File.open "big_uncompressed_file" do |f|
File.open "big_compressed_file", "w" do |w|
f.each do |str|
w << z.deflate str, Zlib::SYNC_FLUSH
end
end
end
z.finish
z.close
+3
source to share