# script to run rsync backups syntax help



## ProServ (Sep 1, 2020)

Hi,
Have a script that is supposed to umount /bkup if mounted, fsck /bkup (/dev/da0p1), mount /bkup (/dev/da0p1) and then run multiple rsync commands one at a time. Due to incorrect syntax errors this script is not working as expected.

The idea is to run one command line at a time until it completes. It was my understanding that '&&' means wait until this command is finished before starting next command. But when the script is run, it produces the error "Invalid null command."

The script is run by crontab with /bin/sh /root/scripts/script1.sh  and in part is:
#!/bin/sh
/sbin/umount /bkup &&
/sbin/fsck -y /dev/da0p1 &&
/bin/sleep 2 &&
/sbin/mount /bkup &&
/bin/sleep 2 &&
/usr/local/bin/rsync -azP --ignore-existing -e 'ssh -p 19225' root@host1.my-domain.tld:/root /bkup/dir1/ &&
/usr/local/bin/rsync -azP --ignore-existing -e 'ssh -p 19225' root@host2.my-domain.tld:/etc /bkup/dir2/ &&
.
.
.
/bin/sleep 5 &&
/sbin/umount -f /bkup &&
/bin/sleep 2 &&
/sbin/fsck -y /dev/da0p1

There are 26 rsync commands in this script backing up different servers.
Have tried substituting '||' for '&&' on each line but the error is the same.
So the question is, what is the correct syntax to put into the script so that each line must complete BEFORE going to the next command?

Thanks.


----------



## chrbr (Sep 1, 2020)

You need the && and || things only if you want to have commands nested in a single line. Please try without && and || as

```
echo "Hi"
sleep 2
echo "Again"
sleep 2
echo "Finito"
```


----------



## ralphbsz (Sep 1, 2020)

How about some error handling? What do you want to happen if any of the commands fail?


----------



## chrbr (Sep 1, 2020)

Dear ProServ,
just in case - https://forums.freebsd.org/threads/useful-scripts.737/page-14#post-469449 gives a lot of useful hints and best practices for shell programming.


----------



## CyberCr33p (Sep 1, 2020)

&& runs next command if the previous command completed successfully.

|| runs next command if the previous command failed.

Also there is no need for sleep and fsck.


----------



## richardtoohey2 (Sep 1, 2020)

I think you'll get the same answers as last time you asked this: https://forums.freebsd.org/threads/shell-scripts-with-multiple-rsync-lines-and.76673/#post-475026

You are trying to make it short and smart and it's not working.  Make things simpler.  It will be longer but easier to understand and debug when things go wrong.


----------



## ralphbsz (Sep 2, 2020)

So many questions:

Why do you fsck? Do you suspect that the file system is damaged? And why do you fsck -y? If the file system is damaged, don't you want to know exactly what the damage is, instead of blazing along?

Why all the sleep calls? I think they're a waste of time.

How about logging from this script? It does a lot of useful things. It will occasionally fail (everything on the planet does). When it fails, you need to debug why.

We absolutely need error handling. And you need to check preconditions.

Do you really want to allow root ssh login? That's a security hole big enough to drive a tractor through.

You seem to have very little experience with shell scripts. In your hands, writing scripts that run as root and copy lots of data around is unsafe. An old colleague used to call this kind of thing a "foot-shaped gun": A gun that is most suited for shooting oneself in the foot.

Design hints: Take common things (like the name of the file system, the name of the host, the ssh port, and so on) and put them into variables.

And write some shell functions to encapsulate commonly used operations, like running the rsync/ssh.

And think about using parallelism. Depending on where your bottlenecks are (disk or network? how many disks?), running multiple rsync's in parallel might help (or it might hurt). Or you might not care about performance.


----------



## ProServ (Sep 14, 2020)

Hi, is there code we could add to sh scripts that would check if external drive is already mounted? Trying to get script which does backups to mount /backup if not already mounted  and if not mounted, mount the backup disk.

As for the # fsck -y , have had in the past encountered useless backups due to file getting messed up somehow. Since our backup rsync lines will check dest against current file (--ignore-existing),  the damaged file will be replaced. I don't see how fsck -y will hurt in this case. 

As for root login, pf does a great job for login failures.


----------



## chrbr (Sep 14, 2020)

Dear ProServ,
basically you can use the same methods as if you are doing the task on the console. You have to be more pedantic in error handling when writing scripts. In interactive use the brain can catch simple errors without thinking. A script just runs as programmed. About the question you can run `mount` and check its output. May be by pipe the output to `grep` which filters the output and tells you if something is included or not. That can be `mount|grep backupserver` or so. The result can then be part of an if statement which then run something or not.

What I usually do is to check the commands one by one interactively. One task is to check the expected output. The more difficult thing is to get an idea of the output in case something goes wrong like disk not attached, cable broken, ... You have to think about what to do if something goes wrong. This is more difficult than specifying the good path. Then it is good to test things in steps as small as possible. For that you can already use short scripts using the standard script shell `/bin/sh`. When you are happy with the small building blocks you can add them to improve your existing scripts.

My advise is not to just google, copy&paste but find out what is going on. If something goes wrong you will have to find out why. By copy&paste it is difficult to learn how things work.


----------

