[mdlug] Compression and Archiving

[Adam Tauno Williams]

On Mon, 2010-01-11 at 05:11 -0500, Drew wrote:
> I find myself with a lot of data to archive and back up - 120 GB of 
> it, with possibly more to come at a
> later date - and am considering DVDs and squashfs. Current DVD cost 
> weighs in at 18 cents a disk
> (Office Max), which is under 4 cents a gigabyte, which is less than 
> half the cost of current terabyte
> range hard drives as far as I know.

If you care about your data either (a) don't backup to DVD or (b) make 
multiple copies of each disc.

<quote source="WMOGAG">
The longevity of a burned CD is strongly influenced by environmental
factors. CDs used for backup should always be stored in clean jewel
cases [not sleeves!] at a reasonably constant 10°C - 15°C and 20% - 50%
relative humidity; even relatively brief periods of higher temperatures
can accelerate the aging of the media. Exposure to UV light also rapidly
decomposes the CD media; media should be stored in a dark place away
from sunlight. In general, optical media is a very poor choice as a
backup solution; claims that burned CDs should last for decades are
highly suspect.
</quote>

Tape is cheap, tape is reliable... buy a tape drive.

> Squashfs seems to give me compression down to from 66% to 77% of 
> original size with most of the
> files being PDFs. I had been hoping for something a bit better. Is 
> there a way to specify better compression
> methods? Is there a better filesystem to use? Or would I be better 
> off using something like gzip or bz2?
> I'd like to be able to mount these as loop devices.

PDFs are usually pre-compressed (internally).  I doubt anything is going
to give you much better than ~50% (you are lucky to be getting the rates
you are) except at very high CPU cost.

> Also, can an entire DVD be a squashfs filesystem? Or does it have to 
> be wrapped in an iso?

To be readable my anything 'normal' it must be an ISO.

-- 
OpenGroupware developer: awilliam at whitemice.org
<http://whitemiceconsulting.blogspot.com/>
OpenGroupare & Cyrus IMAPd documenation @
<http://docs.opengroupware.org/Members/whitemice/wmogag/file_view>