Zlib for Mono / .Net without known checksum
I need to implement a custom ZLib implementation that should work under .Net and Mono. Data / string messages are received over the socket and therefore there is no checksum. It's about raw string data, not files.
unsigned char zlib_header[]={
// custom additional Zlib Id
'Z', // Our own ID
// The normal GZIP header
0x1f,
0x8b, // GZIP ID
0x08, // Deflated
0x00, // Flags
0, 0, 0, 0, // Timestamp,
0x00, // Extra flags
0x00, // OS identifier
// afterwards compressed data without a checksum
};
I tried to decompress the data using GZipStream and DeflateStream, but I think the GZStream is failing due to a missing checksum. I have also tried various offsets, but I have no luck. The checksum is not used as data is received over the socket anyway, so the ZLib checksum will be an additional overhead. Am I missing something, or could you explain to me how to add the checksum and call the library I want, or should I look at a third party library that supports Mono and .Net? Edit: Performance is very important as this runs at least once a second. Would you advise me at the end to use C-Lib via Interop? I am always getting Invalid Data Exception at the moment and I am guessing it is due to an incorrect checksum.This is the actual code I tried to use without success:
const int HeaderSize = 1;
System.IO.MemoryStream ms = new System.IO.MemoryStream(compressedBuffer, HeaderSize, compressedBuffer.Length-HeaderSize);//remove the additional Z from the header
GZipStream zipStream = new GZipStream(ms, CompressionMode.Decompress,true);
byte[] deCompressedBytes = new byte[actualBufferLength* 10];
int resultSize=zipStream.Read(deCompressedBytes, 0, actualBufferLength);//get rid of the header
UTF8Encoding enc = new UTF8Encoding();
string result = enc.GetString(deCompressedBytes, 0, resultSize);
a source to share
Are you sure it has something to do with the checksum?
The 32-bit checksum is optional in GZIP format. I don't understand what you mean by "data received over socket, so there is no checksum". It doesn't matter if you receive the data via a carrier pigeon; if it is a valid GZIP stream, it must have a 32-bit CRC. Who or what created the raw data?
There is an optional part in the GZIP specification - a 16-bit checksum. (Its inclusion is also not based on how the GZIP stream was created.) The System.IO.GZipStream class will gladly accept a GZIP stream that lacks this CRC16, as well as one that includes it.
You have other problems in your code. The actual BufferLength in the code - what is it? This is, of course, not the length of the buffer to store the decompressed data. This is 10 times. But 10x seems pretty arbitrary. For highly compressible data, you can go over 10x. I suggest you take a streaming approach to decompression.
As far as if you can handle 1 decompression per second, yes, System.IO.GZipStream will be fast enough for small enough chunks of data. There is probably no need to switch to your own C / C ++ library.
ps: the DotNetZip library includes GZipStream, which is open source; you can use it out of the box, or if you like, you can just grab the GZip material if that's all you need.
a source to share