Blosc is a high performance compressor optimized for binary data. It has been
designed to transmit data to the processor cache faster than the traditional,
non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc is
meant not only to reduce the size of large datasets on-disk or in-memory, but
also to accelerate memory-bound computations.
It uses the blocking technique so as to reduce activity in the memory bus as
much as possible. In short, this technique works by dividing datasets in blocks
that are small enough to fit in caches of modern processors and perform
compression/decompression there. It also leverages, if available, SIMD
instructions (SSE2, AVX2) and multi-threading capabilities of CPUs, in order to
accelerate the compression/decompression process to a maximum.
From martin@; input and OK from bcallah@, feinerer@, and sthen@
