data compression
================

borg create --compression N repo::archive data

Currently, borg only supports zlib compression. There are plans to expand this
to other, faster or better compression algorithms in the future.

N == 0 -> zlib level 0 == very quick, no compression
N == 1 -> zlib level 1 == quick, low compression
...
N == 9 -> zlib level 9 == slow, high compression

Measurements made on a Haswell Ultrabook, SSD storage, Linux.


Example 1: lots of relatively small text files (linux kernel src)
-----------------------------------------------------------------

N == 1 does a good job here, it saves the additional time needed for
compression because it needs to store less into storage (see N == 0).

N == 6 is also quite ok, a little slower, a little less repo size.
6 was the old default of borg.

High compression levels only give a little more compression, but take a lot
of cpu time.

$ borg create --stats --compression 0
------------------------------------------------------------------------------ 
Duration: 50.40 seconds
Number of files: 72890

                       Original size      Compressed size    Deduplicated size
This archive:                1.17 GB              1.18 GB              1.01 GB

                       Unique chunks         Total chunks
Chunk index:                   70263                82309
------------------------------------------------------------------------------ 

$ borg create --stats --compression 1
------------------------------------------------------------------------------ 
Duration: 49.29 seconds
Number of files: 72890

                       Original size      Compressed size    Deduplicated size
This archive:                1.17 GB            368.62 MB            295.22 MB

                       Unique chunks         Total chunks
Chunk index:                   70280                82326
------------------------------------------------------------------------------

$ borg create --stats --compression 5
------------------------------------------------------------------------------ 
Duration: 59.99 seconds
Number of files: 72890

                       Original size      Compressed size    Deduplicated size
This archive:                1.17 GB            331.70 MB            262.20 MB

                       Unique chunks         Total chunks
Chunk index:                   70290                82336
------------------------------------------------------------------------------

$ borg create --stats --compression 6
------------------------------------------------------------------------------ 
Duration: 1 minutes 13.64 seconds
Number of files: 72890

                       Original size      Compressed size    Deduplicated size
This archive:                1.17 GB            328.79 MB            259.56 MB

                       Unique chunks         Total chunks
Chunk index:                   70279                82325
------------------------------------------------------------------------------

$ borg create --stats --compression 9
------------------------------------------------------------------------------
Duration: 3 minutes 1.58 seconds
Number of files: 72890

                       Original size      Compressed size    Deduplicated size
This archive:                1.17 GB            326.57 MB            257.57 MB

                       Unique chunks         Total chunks
Chunk index:                   70292                82338
------------------------------------------------------------------------------


Example 2: large VM disk file (sparse file)
-------------------------------------------

The file's directory size is 80GB, but a lot of it is sparse (and reads as
zeros).

$ borg create --stats --compression 0
------------------------------------------------------------------------------
Duration: 13 minutes 48.47 seconds
Number of files: 1

                       Original size      Compressed size    Deduplicated size
This archive:               80.54 GB             80.55 GB             10.87 GB

                       Unique chunks         Total chunks
Chunk index:                  147307               177109
------------------------------------------------------------------------------

$ borg create --stats --compression 1
------------------------------------------------------------------------------
Duration: 15 minutes 31.34 seconds
Number of files: 1

                       Original size      Compressed size    Deduplicated size
This archive:               80.54 GB              6.68 GB              5.67 GB

                       Unique chunks         Total chunks
Chunk index:                  147309               177111
------------------------------------------------------------------------------

$ borg create --stats --compression 6
------------------------------------------------------------------------------
Duration: 18 minutes 57.54 seconds
Number of files: 1

                       Original size      Compressed size    Deduplicated size
This archive:               80.54 GB              6.19 GB              5.44 GB

                       Unique chunks         Total chunks
Chunk index:                  147307               177109
------------------------------------------------------------------------------
