Deflate64, or “expanded deflate”, or “enhanced deflate”, is an improved version of the Deflate compression algorithm. The original Deflate algorithm was introduced by PKWare almost 30 years ago, with the release of PKZip 2.04g. This algorithm was quite groundbreaking – it was fast and still achieved moderately good compression. It was a lossless algorithm, and was patented in the USA under patent number 5051745.
Deflate was the most widely used compression algorithm for many years, but the compression it offered was insufficient for very large files. In 2001, PKWare released PKZip 4.5, and with it they introduced Deflate64. This compression algorithm is almost as fast as Deflate, but offers much better compression.
How Does Deflate64 Work?
The main difference between Deflate and Deflate64 is that Deflate53 uses a 64K sliding window, rather than a 32K sliding window, when performing Lempel-Ziv compression. This means that it is not backwards compatible with Deflate, however on supported machines it offers much better compression.
The final length code has been expanded by 16 extra bits, and the distance codes have also been expanded to address a 64K range. These codes were unused in Standard Deflate. Compression is achieved in a two-step process of duplicate string elimination followed by bit reduction.
Availability in Programming Libraries
Implementations of Deflate are available in several languages through the zlib and gzip libraries. Most C programs use zlib, while Pascal developers can use paszlib. In Java, developers (I know Magento programmers who know that too) can make use of the implementation found in Java.Lib.Zip.
PKWare has trademarked Deflate64, and it is considered to be a proprietary implementation. Deflate64 is not supported in zlib because of its proprietary nature, and the marginal performance difference when compared to Deflate. Deflate is also used in several hardware encoders. Comtech has produced a PCI-X card which can compress streams at a rate of 3Gbit/Sec, however this uses standard Deflate rather than Deflate64.