# Syncing one directory with another



## r619 (Jun 27, 2013)

The goal is to minimize writes to /var on SSD.

Mounting /var read-only won't solve it, obviously - it changes too frequently, not only when we decide to update something. So the solution is to mount /var as tmpfs, and sync it with _the_ SSD copy on init (as early as possible, to eliminate the chance of something useful being overwritten by something useless) and SSD copy with tmpfs on shutdown (as late as possible, to make less useless writes)*.*

First task is fa_ir_ly simple - we just place `cp -R /somedir/var.bak/ /var` somewhere in rc.d.

The second task is harder: we have to compare _the_ last modification timestamp of each file and directory contents. _A c_ustom script which would parse several `ls -c` outputs and then recursively check dir_ectories_s and then copy new files doesn't seem to be right way, also I'm not yet familiar enough with `sh` syntax.
_Are_ there some other solutions?


----------



## SirDice (Jun 27, 2013)

r619 said:
			
		

> Is there some other solutions?


How about net/rsync? You can also use it for local directories.


----------



## ondra_knezour (Jun 27, 2013)

There is mount_unionfs() which can resolve problems with initial sync, but according to BUGS section of _the_ manpage this filesystem seems to be broken from stone age. 

NanoBSD uses some tricks to save it's configuration files to read-only partitions, check there for inspiration.


----------



## throAU (Jun 27, 2013)

Given the improvements in SSD reliability and reduced cost, considered just mirroring /var?

I'm guessing even a pair of SSDs is a lot cheaper than your time spent to craft some custom hack that isn't in common use, and deal with the fallout when THAT breaks?

If there's one thing the industry has taught me it is this:  don't try and make things more complex than they need to be (they are already complex enough!).

I'd suggest it is far more likely that your custom sync solution will be more likely to cause problems or fail in some way (un-clean shutdown for example) more spectacularly than simply wearing the cost of fault tolerance and replacement for SSD reliability.


----------



## Anonymous (Jun 27, 2013)

r619 said:
			
		

> ... Are there some other solutions?



Perhaps sysutils/clone in sync-mode. Note, I am biased here, since I wrote this utility. `# clone -s /var /somedir/var.bak`

Anyway, for any kind of synchronization, you need to tell the initial copy to maintain the file/dir attributes. So for cp(1)() you want to use the -p option, in addition to -R.


```
man cp
...
     -p    Cause cp to preserve the following attributes of each source file
           in the copy: modification time, access time, file flags, file mode,
           user ID, and group ID, as allowed by permissions. ...
```

`# cp -pR /somedir/var.bak/ /var`

However, you could use also clone for the initial copy: `# clone /somedir/var.bak /var`


----------



## wblock@ (Jun 27, 2013)

I'm with @throAU on this.  If the system loses power, changes to /var would be lost.  I also don't feel it's worth taking special precautions to limit writes to SSDs.  They are there to be used.  If a system writes so much to /var that it really could wear out an SSD, using a hard drive or remote syslog server would be appropriate.  But really, very few systems need that.


----------



## phoenix (Jun 27, 2013)

Even consumer-grade SSDs are rated for multiple full-disk writes per day (usually 3-5 on the crappiest SSDs; 10-20 on the good ones).  /var doesn't change that much on a daily basis, so it won't be reducing the lifespan of the SSD by more than a few days or weeks.


----------



## r619 (Jun 27, 2013)

throAU said:
			
		

> Given the improvements in SSD reliability and reduced cost, considered just mirroring /var?
> 
> I'm guessing even a pair of SSDs is a lot cheaper than your time spent to craft some custom hack that isn't in common use, and deal with the fallout when THAT breaks?


Still, it's a nice way to make yourself familiar with the new system*.*



			
				throAU said:
			
		

> I'd suggest it is far more likely that your custom sync solution will be more likely to cause problems or fail in some way (un-clean shutdown for example) more spectacularly than simply wearing the cost of fault tolerance and replacement for SSD reliability.


Backups, in case of this. At worst, *I* would lose just the package database.


----------



## r619 (Jun 28, 2013)

rolfheinrich said:
			
		

> Perhaps sysutils/clone in sync-mode. Note, I am biased here, since I wrote this utility.



A bug, presumably:

```
# mkdir source dest && touch ./dest/file1 ./source/file2 && chmod 777 ./dest/file1 && clone -s source dest
.
dest/file1 is of unknown type, it could not be deleted.

No items copied.
```


----------



## Anonymous (Jun 28, 2013)

r619 said:
			
		

> A bug, presumably:
> 
> ```
> ...
> ...



A regression in the course of 1.0.0 to 1.0.1. Earlier today, I fixed this already upstream.

The port update is on the way.


----------



## throAU (Jun 28, 2013)

r619 said:
			
		

> Still, it's a nice way to make yourself familiar with the new system



Well if it's a project to learn how to do replication, then fair enough.  But if it's a production system...



> Backups, in case of this. At worst, *I* would lose just the package database.



Unless you've got your logs elsewhere via remote syslog, they are gone also.  Along with your mail spool.  You've also got downtime while you recover from backup.


----------



## r619 (Jun 29, 2013)

Wow, just look what I've found.

That makes the task quite easier - I may simply symlink some important dirs (portsnap, pkg database) on init to /mnt/somehdd if it exists. (Yup, I don't think *I* need old logs.) Everything else is already done and could be explicitly turned on by pushing "populate_var=YES" and "varmfs=YES" strings in /etc/rc.conf.

And now I also understand how rc.conf works.


----------

