Compression library
The idea of this target is to provide a single interface that will allow one to use various compression libraries according to their needs. Currently, the goal was to use compression libraries for raw data decoding, where huge files are obtained. Therefore, optimizations that might be applicable for small files are not used and we consider: 1 file -> 1 compressed file, i.e., no tars, and similar. If smaller files are needed, one might approach problems differently, or consider using libarchive instead - it already provides everything one might need
Supported compression libraries
Currently, this target is only able to read (decompress) data. Currently supported libraries:
- zstd - A relatively new library developed by Facebook that compresses data very very well while still keeping excellent compression/decompression speeds.
- zlib - An old compression standard based on the Deflate algorithm. It provides good compression ratios (still lower than zstd) and decent compression/decompression speeds.
- zlib-ng - A newer compression library similar to zlib (it's called zlib new generation) that also uses the Deflate algorithm, reaches similar compression ratios while still maintaining extremely fast compression/decompression (for higher compression levels even faster than zstd)
- lzma2 - A LZMA2 algorithm used by xz that's developed based on the original LZMA algorithm of 7Zip. This algorithm can achieve maximal compression ratios, but compresses data very slowly. Decompression speeds are decent, but still much slower than either of the previous libraries.
For actual testing results, check "ArchiveTest.txt" that can be found in the same directory as this file
How to use
There's only a few main decompression headers that users should be aware of:
- BmnCompressionGlobalFunctions.h exposes some basic functionality for opening an appropriate file reader and obtaining whether specific readers are available in the first place. Later, we will be adding functionality for writing as well.
- BmnFileProcessingFlags.h provides various flags that can be used by the API of this target. For instance, one can notice BmnFileType which tells us what kind of library is used in the background, or BmnBinaryFileProcessingOptions to instruct how to process files. For more info, see the guidance in the header
- BmnIFileReader.h provides an API for an interface class that should be used for any reading. That is, with BmnCompressionCreateNewSuitableReaderForAFile one can generate a file, and later use the interface class for obtaining any necessary data. For instance:
// This will generate an appropriate reader based on the provided file (usually through the magic byte)
BmnIFileReader* readingInstance = BmnCompressionCreateNewSuitableReaderForAFile("someFilePath");
// This will actually open the file for reading (always reads in the binary mode)
readingInstance->Open();
if (readingInstance->IsOpen())
//return/break/handle;
// Then read and process the file as
while (true /*or some other condition*/) {
if (readingInstance->Read(someData, numberOfBytesToRead) != numberOfBytesToRead)
// handle error or EOF here
// process the read data here
}
// Later don't forget to free the memory
delete readingInstance;
// And set to nullptr for safety if smart pointers are not used
readingInstance = nullptr;
Structure
| Directory / file | Description |
| include | Public headers: interfaces, file readers, flags and global functions |
| sources | Implementation of the file readers and global functions |
| ArchiveTest.txt | Results of the compression and decompression tests of the supported libraries |
Library
Built into the BmnCompression library (no ROOT dictionary). The zstd, liblzma, zlib and zlib-ng readers are enabled only if the corresponding library is found by CMake (the BMNCOMPRESSION_WITH_* definitions); the raw reader is always available.
1.17.0