# Copying a big folder from a near full disk over sftp?



## dieselnutjob (Jan 23, 2021)

Hello
I have a mail server that's nearly full, and another server doing nothing with a bigger disk.
Both servers are running FreeBSD 11.4.
I don't have enough empty space on the mail server to tar up users maildir folders.
Is there a way to somehow send entire folder structures down an ssh/sftp link from one to the other without taring them up first?
Also the only user that can ssh/sftp from one server to the other is my account.  root and the other users are blocked so I need do it with an account different to the user that owns the files.  Only public/private key authentication is supported if that makes a difference.
I can have some downtime to do this (dns changes will take time to propagate anyway) so it doesn't have to be done with the dovecot or postfix daemons running.
thanks


----------



## ralphbsz (Jan 23, 2021)

rsync can copy over ssh, and use ssh key authentication. I do it all the time.


----------



## im (Jan 23, 2021)

FreeBSD user can't write files with a different owner.
Do you have local root access to the destination and source servers?

*pax*: How to copy the files from a remote PC to the local current folder:
Pax is come with FreeBSD.

```
ssh root@source.pc "cd /mnt/SourceFolder && pax -w -X . " | pax -r -v -p eme
```

*rsync*: How to copy from a local folder to the remote destination:

```
#!/bin/sh
date_now=`date "+%Y%m%d-%H%M"`
rsync -Havx --numeric-ids --bwlimit=4000 --log-file=log-${date_now} -e 'ssh -p 22 -i keyfile' /localsource/ root@remote.server:/destination/
# --dry-run --delete --delete-after --delete-excluded --exclude-from=
```

rsync is more useful. You can run rsync on worktime for creating initial copy.
By then, on day-off, stop your mail services and run the same rsync with '--delete'. It sync the copy with actual data.
Be careful with '--delete'.

```
This  option can be dangerous if used incorrectly!
It is a very good idea to first try a run using the
--dry-run option (-n)  to see what files are going to be deleted.
```


----------



## dieselnutjob (Jan 24, 2021)

Yes I have the root password at both ends.


----------



## im (Jan 24, 2021)

dieselnutjob said:


> Yes I have the root password at both ends.


Just enable remote root access on the destination server. Is not it?

In other case, you can create tar.archive from source server directly to the user@destination.
By then, on the destination server' expand the received archive with root privileges. It saves owner and other attributes.
The same idea may be performed using 'pax()'.

Another way:
Install and configure rsync-service on a destionation. Configure it for running with root privileges.
Copy the files from source to the destination's rsync service. No ssh root access are required in this case.

```
rsync -Hav --numeric-ids /home/mail/ 192.168.111.1::mail2/
```



Spoiler: rsyncd.conf





```
# rsyncd.conf - Example file, see rsyncd.conf(5)
#

# Set this if you want to stop rsync daemon with rc.d scripts
pid file = /var/run/rsyncd.pid

# Edit this file before running rsync daemon!!

#uid = nobody
#gid = nobody
uid = root
gid = wheel
#use chroot = yes
#max connections = 4
#syslog facility = local5
address = 192.168.111.1

[mail2]
       path = /home/mail2/
        use chroot = no
        read only = no
        numeric ids = yes
```




Yet another ways:
Run the source server as a jail() on a destination. This way can save your time, if you are familiar with jail().
Share 2nd server storage and mount it from 1st. Just use 2nd as a storage.


----------



## im (Jan 24, 2021)

I have "ready to use" solution:
It creates an archive on the remote computer.
It uses 'regular user' and 'key-based authorization' for ssh-transfer.

*On the source server, run with root privileges:*

```
# cd /sourcefolder/
# pax -w -X . | ssh -p 22022 -i key-admin admin@destination.com "cat >/home/temp/sourcefolder.pax"
```

*On the destination server, run with root privileges:*

```
# cd /home/temp/
# pax -r -v -p eme -f sourcefolder.pax
```

Tune these commands for your case.
Enjoy!


----------



## drhowarddrfine (Jan 24, 2021)

Note: these are "directories" and not the Windows concept of "folders" which is not the same thing.


----------

