To use multi-volume, do something like:
cd to directory with files
tar cvf /dir/out1.tar -M -L 52428800 *
(where /dir/ is a place to hold the 50GB tar files, and 52428800 is
50*1024*1024, the number of 1k blocks in 50GB)
It will write the first 50GB to /dir/out1.tar, then prompt with:
Prepare volume #2 for '/dir/out1.tar' and hit return:
Type in
n /dir/out2.tar
(that's 'n space /dir' etc) and press return. The next 50GB goes to
/dir/out2.tar and it prompts with
Prepare volume #3 for '/dir/out2.tar' and hit return:
Type in
n /dir/out3.tar
and press return. Continue for 37 more files!
(When I started writing this, I didn't realise how much effort it was!).
Another solution could be:
tar cvf - * | split -b 52428800 -d - /dir/out
This will write /dir/out00, /dir/out01, /dir/out02 and so on to
/dir/out40. The tar writes to stdout and pipes to split. The '-b' splits
the file into that number of bytes per file, and the '-d' uses numeric
suffices on the filenames (otherwise it would be aa, ab, ac etc to b0).
The '-' reads from stdin, and the '/dir/out' is the prefix for the
output filenames.
This splits the files at a given number of bytes, so all parts need to
be connected together to extract data. The benefit of the 'tar -M'
version is that each part is a valid tar file (though some files may be
split across two tar files).
Simon
On 29/05/2019 16:12, James Courtier-Dutton via Hampshire wrote:
> Hi,
> Tar already has such a feature. A long time ago, it would have needed
> to spread data across backup tapes.
> look at the man page for tag for something like --multi-volume.
>
> Kind Regards
>
> James
>
>
> On Wed, 29 May 2019 at 12:12, Rob Malpass via Hampshire
> <hampshire@??? <mailto:hampshire@mailman.lug.org.uk>>
> wrote:
>
> Hi all
>
> I’ve never been a bash script expert (to say the least!) but I
> need a clever script (I assume that’s the easiest way but please
> say if not) that can turn 120 files (amounting to 2Tb of data)
> into 40 files of <=50Gb for an archiving project I’m about to
> embark on. Can anyone help / give me some pointers?
>
> I can guarantee none of the files are over 50Gb in size. So it’s a
> question of something like:
>
> Create an empty archive (call it n.tar.gz)
>
> Repeat
>
> Find next (uncompressed) file
>
> Compress it to tmp.tar.gz
>
> If size of tmp.tar.gz < (50Gb – current size of n.tar.gz) then
>
> Append tmp.tar.gz to n.tar.gz
>
> Else
>
> Close n.tar.gz
>
> Alert me so I can write n.tar.gz to MD
>
> Create next empty archive ((n+1).tar.gz)
>
> Add tmp.tar.gz to (n+1).tar.gz
>
> End If
>
> Until all files have been compressed
>
> The end product should be 40 files called n.tar.gz (where 0<n<41).
>
> Any (constructive!) ideas very welcome.
>
> Cheers
>
> --
> Please post to: Hampshire@???
> <mailto:Hampshire@mailman.lug.org.uk>
> Web Interface: https://mailman.lug.org.uk/mailman/listinfo/hampshire
> LUG URL: http://www.hantslug.org.uk
> --------------------------------------------------------------
>
>
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
> Virus-free. www.avg.com
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
>
>
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
--
Please post to: Hampshire@???
Web Interface:
https://mailman.lug.org.uk/mailman/listinfo/hampshire
LUG URL:
http://www.hantslug.org.uk
--------------------------------------------------------------
