# Is swap on zfs safe?



## VK (Nov 20, 2011)

Hello,

I've read a couple of years ago that swapping on zfs wasn't safe [at that time] due to the fact zfs itself might need allocating memory for writing to the swap dataset. I can't find anything related to this issue right know. 

Is swap over zfs safe on FreeBSD?

Thanks.


----------



## graudeejs (Nov 21, 2011)

zfs swap works perfectly fine for me


----------



## Sebulon (Nov 21, 2011)

Same here

/Sebulon


----------



## gkontos (Nov 21, 2011)

The only disadvantage is that it can not be used as a dump device.

Other than that it is absolutely safe. Just make sure you set the checksum property to off.

George


----------



## graudeejs (Nov 21, 2011)

gkontos said:
			
		

> The only disadvantage is that it can not be used as a dump device.
> 
> Other than that it is absolutely safe. Just make sure you set the checksum property to off.
> 
> George



That is not required


----------



## gkontos (Nov 21, 2011)

graudeejs said:
			
		

> That is not required


Sorry, what is not required ?


----------



## graudeejs (Nov 21, 2011)

gkontos said:
			
		

> Sorry, what is not required ?



It is not required to turn off checksum property.
However (turning it off might increase performance. I never noticed anything. So I decided to leave it on. At least I'll know when my disk start to die, even if it's swap area. I might be wrong  )


----------



## gkontos (Nov 21, 2011)

graudeejs said:
			
		

> It is not required to turn off checksum property.
> However (turning it off might increase performance. I never noticed anything. So I decided to leave it on. At least I'll know when my disk start to die, even if it's swap area. I might be wrong  )


I have seen cases where the checksum has caused weird messages and false alarms. IMHO you are much better off without it. 

George


----------



## fluca1978 (Nov 21, 2011)

Sorry, I cannot see the real advantage of having swap over ZFS. I mean, ok, ZFS can help finding out disk problems, but is it really needed to have swap over such an important file system? Or are you talking about adding swap files on a ZFS file system?


----------



## graudeejs (Nov 21, 2011)

I don't see disadvantages either


----------



## fluca1978 (Nov 21, 2011)

graudeejs said:
			
		

> I don't see disadvantages either



Of course there are no drawbacks, even if having the capability to snapshot swap, do a checksum on it, and all the other features that ZFS provides sounds a bit too much to me.


----------



## kpa (Nov 21, 2011)

The advantage is that you don't have to make separate partition for swap. You may not even have partitions on the disks that make up your zfs pool and your system disk is so minimal that you don't want to create more than one partition on it. For example on my fileserver the OS is on two 2GB usb memory sticks configured as gmirror(8) RAID1, there just isn't enough room for swap and I didn't want to make extra partitions on the zfs pool disks for swap so I use swap on a ZVOL.


----------



## bbzz (Nov 21, 2011)

Exactly; for me the only reason to have it on ZFS is because I don't have to slice up disk.


----------



## Crest (Nov 21, 2011)

Not checksumming swap is braindead (unless you trust your disks for some reason and want to save a few cycles) as it contains. If ZFS complains about "strange checksum errors" it most likly found disk I/O errors.


----------



## gkontos (Nov 21, 2011)

There have been many arguments regarding the use of ZFS swap. Either way has its pros and cons. Allocating swap space in a ZFS pool has many advantages. For example you don't need to create separate partitions nor to load any other software mirroring module. The only disadvantage is that you can't store core dumps. This could be easily bypassed by allocating a separate device. 
On the other hand, there is no point in using checksums for swap. On the contrary you are adding an extra meaningless burden. You don't rely on swap checksums to monitor the health of your drives. 

PS @crest be a little more careful with your expressions.


----------



## phoenix (Nov 21, 2011)

Not sure if this has been fixed yet or not, but there's one little niggle to using swap on ZFS:  you need memory to track disk usage and whatnot in the ARC.  So, if you get into a situation where you need to swap something to disk, but don't have any non-wired memory to use to track the writes and whatnot ... you end up with a locked up system.

This was a big problem with ZFSv6 and ZFSv14.  Not sure if it's still a problem.


----------



## Crivens (Nov 22, 2011)

gkontos said:
			
		

> I have seen cases where the checksum has caused weird messages and false alarms. IMHO you are much better off without it.
> George



Have you heard that they are now removing the FriendOrFoe alert from missile systems? There have been weird beebing noises and false alarms...  

Seriously, for all disks which gave out in my systems in the last years ZFS got the message way before SMART was able to find anything. The only benefit for turning off checksums for swap would be that a read error from the disk does not shoot up the system. Maybe the page is to be written, then you are save. But I do not play russian roulette. If it does not fail at once you get panics, maybe hours or days later, and have no idea where they come from. 

I prefer a faulty system to halt at once so I can see what the trouble is and so I am motivated to fix it. Turning that off is IMHO like playing russian roulette. With an automatic.

Sorry if this seems harsh, but I see what a certain "this will work, 'guv" stance can get you into each day @work. Do the right thing, do not overdo it, consider each part to be a fail-waiting-to-happen. Including yourself.


----------



## gkontos (Nov 22, 2011)

Crivens said:
			
		

> Have you heard that they are now removing the FriendOrFoe alert from missile systems? There have been weird beebing noises and false alarms...



No, I haven't but I take your word for it since you seem to be the expert here 



			
				Crivens said:
			
		

> Seriously, for all disks which gave out in my systems in the last years ZFS got the message way before SMART was able to find anything. The only benefit for turning off checksums for swap would be that a read error from the disk does not shoot up the system. Maybe the page is to be written, then you are save. But I do not play russian roulette. If it does not fail at once you get panics, maybe hours or days later, and have no idea where they come from.



SMART has nothing to do with ZFS. SMART will report a failure only after the failure has occurred. ZFS also doesn't get any message from disks. It simply rely on checksums to check the integrity of data.

Your point is that while your data is being constantly checked by checksums, you rely on SWAP which is being used as an alternative to memory, as a tool for data integrity.

In addition you are adding extra hogs to a system at a time when the system is over loaded.     



			
				Crivens said:
			
		

> Sorry if this seems harsh, but I see what a certain "this will work, 'guv" stance can get you into each day @work. Do the right thing, do not overdo it, consider each part to be a fail-waiting-to-happen. Including yourself.



My opinion and implementation is based upon technical facts and experience. Of course I could be wrong. But I at least I am not arrogant. I see the arrogance in some peoples work each time I am called to fix their catastrophes


----------



## Crivens (Nov 22, 2011)

gkontos said:
			
		

> No, I haven't but I take your word for it since you seem to be the expert here


Not for ZFS, there are others here who know more about what to do with it. Phenix comes to mind, for example.

The joke is that the missile systems, at least those I had the questionable honor to provide service for, do not have safety measures. This reduces parts and also makes certain that whoever operates the damned thing does not rely on safety catches and is widely awake while operating it.


> SMART has nothing to do with ZFS. SMART will report a failure only after the failure has occurred. ZFS also doesn't get any message from disks. It simply rely on checksums to check the integrity of data.


Yes, it checks your data. And it does this IMHO much better than the SMART code in the HD. You get the notification from ZFS that something does not work out before the HD sees anything wrong. You are free to ignore such warnings if you want.


> Your point is that while your data is being constantly checked by checksums, you rely on SWAP which is being used as an alternative to memory, as a tool for data integrity.


 No. My point is that if you can have a warning that the disks are about to fail, I want that warning ASAP. Your suggestion to disable checksums to shave a few cycles would help how exactly?


> In addition you are adding extra hogs to a system at a time when the system is over loaded.


Yes, and proudly. There is a fundamental difference between a slow system and a crashed system. Also, if the pool which is hosting the swap device or file is supplying redundancy you can survive even a read error upon page-in. Again, it can do that only when checking is enabled.


> My opinion and implementation is based upon technical facts and experience. Of course I could be wrong. But I at least I am not arrogant. I see the arrogance in some peoples work each time I am called to fix their catastrophes


Which is more or less exactly what I wrote.


----------



## graudeejs (Nov 22, 2011)

BTW, it I never notice when system writes to swap when I use ZFS, however I remember how I felt lags when I was using swap partition


----------



## Slurp (Nov 22, 2011)

To those who say that they use swap on ZFS and it's safe, did you try to stress it?
I.e. write a program that randomly accesses 2x physical memory for a couple of minutes.


----------



## fluca1978 (Nov 23, 2011)

Uhm...if the point of having swap on ZFS is that it can help finding problems on disks, why not doing a memory disk using ZFS? That will help finding errors in memory too. I'm not joking, that is one of the ZFS strength.
However, I agree that using ZFS for swap means using a _fat_ system when the whole machine is overloaded, so it does not seems the right choice to do. It is also true that, my experience, people tend to use ZFS on a set of disks, not a single disk, and here the RAID could split I/O and make swapping less overloading that the single-partition counterpart. 
Is there any real stress-test/comparison about using a slice vs zfs swap? That could give us some real answers...


----------



## graudeejs (Nov 23, 2011)

Right, and you expect swap to be use all the time... I don't I expect swap to not be used... If it's used too much, it means that you need more RAM


----------



## Slurp (Nov 23, 2011)

graudeejs said:
			
		

> Right, and you expect swap to be use all the time... I don't I expect swap to not be used... If it's used too much, it means that you need more RAM



I have a workload that could benefit from petabytes of RAM. I can't afford it, so I push what I have. For this reason, in the predictable future my swap will be used heavily.


----------



## Crivens (Nov 23, 2011)

Slurp said:
			
		

> I have a workload that could benefit from petabytes of RAM. I can't afford it, so I push what I have. For this reason, in the predictable future my swap will be used heavily.



Fluid dynamics simulations?  (I know when _my_ system starts to hit the page store )


----------



## Slurp (Nov 23, 2011)

Nah, compression.


----------

