# What would cause copy to fail randomly?



## MMacD (Feb 5, 2017)

Running as root, I added a big zfs mirror to my server-of-all-work (running 10.2) to separate out project storage from general storage and gain more working space.  Then, I `cp -R * /projects` to move the files.  But I discovered that not all files were copied.  Even whole subtrees were missed out.

Having installed X on the server for convenience's sake, I tried using drag-and-drop to copy one of the subtrees.  But, again, not all the files in the subtree were copied.

There's no pattern to it that I can see, and I feel quite mystified.

The most scary and annoying part is that, in order to free up the mount point for the new mirror, I'd to `cp -R * /scratch` to begin with, so goddess knows what files were lost in _that_ operation!


----------



## aribi (Feb 5, 2017)

The equivalent of `cp -R * /projects` with verbosity would be

```
tar cf - * | (cd /projects ; tar xfv -)
```
This will give you a list of the files written out at the new location. This can be easily checked with a list of the files in the origin produced by

```
find . -type f -print | sed 's/^\.\///'
```


----------



## ShelLuser (Feb 6, 2017)

MMacD said:


> The most scary and annoying part is that, in order to free up the mount point for the new mirror, I'd to `cp -R * /scratch` to begin with, so goddess knows what files were lost in _that_ operation!


Look into mtree(8), that is a very good tool to quickly check a copied directory hierarchy. One warning though; you may want to disable the use of modification time / access time.

So let's say I copied ~/temp/ (directory) to ~/tree/temp and I want to ensure that all files are there (this may also help you to establish a pattern).

First I run `mtree -ck nochange > ../spec` in ~/tree/temp. This gets me a specification file ~/tree/spec which mtree can use to compare against the original.

I then go to my original location (~/temp) and use: `mtree < ~/tree/spec`. This will tell you exactly which files weren't copied and were missing. Of course you can also pipe the commands if you'd like, but I prefer to take it one step at a time.

Note: I used the nochange specification to merely check if a file was actually there (had been copied). If you need more certainty (establish if a file hasn't been corrupted) then you can also add CRC checksums or other checks. See the manualpage, I could imagine using something like: `mtree -ck nochange,md5`.


----------



## fscorrea (Feb 13, 2017)

Consider using net/rsync instead of `cp`:
`$ rsync -auvrh --progress $(pwd)/* /scratch`


----------

