# dump -- I just don't get it



## kr651129 (Aug 21, 2012)

I read the handbook on using dump and I guess I just don't understand.  Can I use dump to take an image of my entire disk incase of failure and restore my hdd to exactly how my pc is functioning from the day of the image?


----------



## zhoopin (Aug 21, 2012)

Cloning or duplicating a running system using dump/restore


----------



## wblock@ (Aug 21, 2012)

Backup Options For FreeBSD shows how to do that.

Realize that dump(8) only backs up filesystems, not partition tables or bootblocks.  It's not a bad idea to at least print out the output of gpart show.


----------



## Anonymous (Aug 21, 2012)

If you are really looking for cloning the complete disk, including all partitions, then dump/restore is perhaps not the best option. It is somewhat slow, and as wblock pointed out, you have to take care for each partition and the boot block separately.

Quite frequently, I use dd(1)() to clone my system disk to another hdd of the same model. This produces an exact clone, including partitions and its labels.

`dd if=/dev/ada0 of=/dev/ada1 bs=1m conv=noerror`

Here /dev/ada0 is the source disk and /dev/ada1 is the target. On a low profile system (Dual Core Atom, 1.6 Ghz),  cloning of a 3 Tbyte disk (2.1 Tbyte used) takes about 12 h at an overall transfer rate of roughly 70 Mbyte/s. Processor load by dd was about 2-3 % during cloning.

Actually, I always submit the dd command from within tmux(1)(), and then detach from the session.

Best regards

Rolf


----------



## wblock@ (Aug 21, 2012)

rolfheinrich said:
			
		

> If you are really looking for cloning the complete disk, including all partitions, then dump/restore is perhaps not the best option. It is somewhat slow, and as wblock pointed out, you have to take care for each partition and the boot block separately.
> 
> Quite frequently, I use dd(1)() to clone my system disk to another hdd of the same model. This produces an exact clone, including partitions and its labels.
> 
> `dd if=/dev/ada0 of=/dev/ada1 bs=1m conv=noerror`



A couple things:

1. dump(8) can be much faster than dd(1), because most disks aren't entirely full or even close.  dump(8) copies just the blocks that are used, dd(1) copies *every* block.

2. noerror means "if there is an error during the copy, no big deal, just fill it with zeros and keep going".  Do not use noerror unless the copy is from a failing hard drive.


----------



## Anonymous (Aug 21, 2012)

wblock@ said:
			
		

> 1. dump(8) can be much faster than dd(1), because most disks aren't entirely full or even close.  dump(8) copies just the blocks that are used, dd(1) copies *every* block.



dump(8)/restore(8) doesn't even remotely come close to a transfer rate of 70 MByte/s. A recent experiment with the same disks gave 15 MByte/s, and at the same time the processor utilization was quite high.

`# mount -o async /dev/ada1p3 /mnt; cd /mnt`
`# dump -C 24 -0af - /dev/ada0p3 | restore -rf -`


```
1926 root        1  58    0  4324K  2256K piperd  1   0:06 18.90% restore
 1927 root        1  57    0 28004K  7008K sbwait  3   0:05 17.48% dump
 1928 root        1  49    0 27876K 26044K physrd  3   0:02  8.50% dump
 1929 root        1  49    0 27876K 26044K physrd  1   0:02  8.40% dump
 1930 root        1  49    0 27876K 26044K physrd  1   0:02  8.15% dump
```

This means, that cloning my 3 Tb disk would take 12*70/15*2.1/2.7 = 43h 33min, and during this time the system would be also quite in-responsive, because of the load that dump/restore imposes on to the CPUs.

So in my case, dump/restore would be superior to dd only if the disk to be cloned would be only about 21 % full.



			
				wblock@ said:
			
		

> 2. noerror means "if there is an error during the copy, no big deal, just fill it with zeros and keep going".  Do not use noerror unless the copy is from a failing hard drive.



The relevant section of dd(1)() tells:


```
noerror  Do not stop processing on an input error.  When an
         input error occurs, a diagnostic message followed by
         the current input and output block counts will be writâ€
         ten to the standard error output in the same format as
         the standard completion message.  If the sync ...
```

This seems to be exactly what I want/need on disk cloning, i.e., on error don't stop, but continue and drop an error message. If there would be an error, then I want dd to rescue as much of the data as possible. After cheking the error log, I could put that cloned disk safely aside, and try again to get a whole clone using recoverdisk(1)() onto a new disk.

Best regards

Rolf


----------



## wblock@ (Aug 21, 2012)

rolfheinrich said:
			
		

> dump(8)/restore(8) doesn't even remotely come close to a transfer rate of 70 MByte/s. A recent experiment with the same disks gave 15 MByte/s, and at the same time the processor utilization was quite high.
> 
> `# mount -o async /dev/ada1p3 /mnt; cd /mnt`
> `# dump -C 24 -0af - /dev/ada0p3 | restore -rf -`



That will go about two to four times faster if you give dump a -b64 flag.



> The relevant section of dd(1)() tells:
> 
> 
> ```
> ...



If there's an obvious error... well, I still have mixed feelings about this.


----------



## kr651129 (Aug 22, 2012)

Wow, lots of great info, thanks everyone.

So if I'm understanding this right, if I want to go the dd route I would do the following


```
dd if=/dev/targetdrive of=/dev/backupdrive bs=1m conv=noerror
```

Then to restore the data from backup drive I would just reverse the process?


```
dd if=/dev/backupdrive of=/dev/targetdrive bs=1m conv=noerror
```


----------



## wblock@ (Aug 22, 2012)

For that to work, either the drives must be identical in size, or the partitions defined so they fit on the smaller drive.  It must be MBR, not GPT.  The better way to "restore" would be to simply disconnect the failed drive and connect the backup drive instead.

It's not really a backup, it's more of a copy from a while back.  What I mean by that is there is only one copy.  If you want to restore a file that was deleted before the most recent copy was made, it doesn't help.  If you create a new directory and new dumpfiles each time, that file can still be accessed in the older files.

If the concern is the system disk dying, a mirror can be a better option.  If the worry is about incremental convenience backup (recovering accidentally deleted files and old versions), see sysutils/rsnapshot.  None of these is a replacement for an off-site backup, but they do go well with it.


----------



## Anonymous (Aug 22, 2012)

wblock@ said:
			
		

> ...  It must be MBR, not GPT. ...



Are you sure?

I used dd for cloning GPT disks. The partitions are cloned exactly, and I can access the whole file structure on all the partitions. I have to admit that I did not try to restart from the clone. Is there something, that I need to be worried about?


----------



## gpw928 (Aug 22, 2012)

If your source file systems are mounted writable, then both dump and dd have caveats, but I would personally trust dump a lot more than dd in that situation because dump has options to deal with backing up writable file systems.  dd does not (and damage to the target from copying a changing file system with dd is unpredictable).

dd is fine if your disks are identical, and file systems not mounted (writable).

As others have indicated:

because dd does not care about file system data structures, it is provides certain emergency rescue options; 

you may need to benchmark blocksize for both dump and dd to optimise speed; and 

dump requires partitions and boot blocks to be restored manually.
dd has a place for special cases requiring disk duplication, but for regular backups I would always prefer dump.

Having said that, I have not used dump for many years.  These days my backups go to disk (both local and remote) using rsync.  It's *much* faster because once the initial copy is in place, only the changes need be backed up.  Like dump/restore, rsync requires that you manually re-create partitions (and possibly boot block) when restoring.

Cheers,

Phil


----------



## wblock@ (Aug 22, 2012)

rolfheinrich said:
			
		

> Are you sure?
> 
> I used dd for cloning GPT disks. The partitions are cloned exactly, and I can access the whole file structure on all the partitions. I have to admit that I did not try to restart from the clone. Is there something, that I need to be worried about?



Backup GPT partition tables have to be at the end of the disk, not before.  If the disks aren't identical in size, one will have an incorrect backup GPT.


----------



## Anonymous (Aug 22, 2012)

wblock@ said:
			
		

> Backup GPT partition tables have to be at the end of the disk, not the before.  If the disks aren't identical in size, one will have an incorrect backup GPT.



Ah, OK!

I always use exactly the same hdd model for cloning. Next time, I will try dump/restore with -b64 as dump option.

Many thanks for the clarification and best regards

Rolf


----------



## wblock@ (Aug 22, 2012)

gpw928 said:
			
		

> Having said that, I have not used dump for many years.  These days my backups go to disk (both local and remote) using rsync.  It's *much* faster because once the initial copy is in place, only the changes need be backed up.



rsync(1) can use hard links for things that haven't changed.  rsnapshot(1) uses that and rotates copies, so it doesn't take much space.


----------



## kr651129 (Aug 22, 2012)

What I'm trying to do is make a clone of my system after I've got everything setup I like (xfce, preferences, other ports, etc) basically "my base system" that way whenever I want to bring my computer back to square one I don't need to spend time installing FreeBSD, ports, changing files, etc.


----------



## wblock@ (Aug 22, 2012)

So... a snapshot in time.  But wouldn't it be nice to have more than one?  Make a new directory like 2012-08-21 each time you do a backup, and you'll be able to go back in time.


----------



## ddaley (Aug 22, 2012)

I have two hard drives (of different sizes) installed in my computer.  I clone my primary to my backup drive once in a while.  I use this script to do it


```
#!/usr/local/bin/bash

time dump -1 -a -L -f - / | ( cd /mnt/root ; restore ruvf - )
time dump -1 -a -L -f - /var | ( cd /mnt/var ; restore ruvf - )
time dump -1 -a -L -f - /tmp | ( cd /mnt/tmp ; restore ruvf - )
time dump -1 -a -L -f - /usr | ( cd /mnt/usr ; restore ruvf - )
```

Might be a decent starting point for you.  Works for me, though it isn't the fastest thing.  I have moved as many non essential files to a partition that I do not clone.


----------

