# Cannot decrypt ZFS bootpool after hardware change



## sotehnn (Jun 27, 2014)

Hi. I have a fresh install of 10-RELEASE sitting on an encrypted ZFS filesystem. The encryption was handled automatically for me during installation. I have a number of other zpools mounted on the same machine but none of them are encryted. All disks are standard SATA-II drives.

My problem is that any reconfiguration of my disk drives (adding or removing non-system drives) causes me to not be able to decrypt and mount my system during boot (password mismatch). The only way I can start my machine is to return my machine to its original configuration (remove the additional HDD's), reboot, and then enter my passphase.

So currently the only way I've been able to expand my storage is to hot-swap my extra drives after boot.

I assume that this is a protection feature and is by-design. But I haven't been able to find any information about how to reconfigure the system after a drive change. There doesn't seem to be any mention of this in the handbook or the GELI man page. 


Could somebody please confirm that what I'm experiencing is indeed normal and point me towards the correct documentation to help me solve it (or give me the command to run if it's something simple). 

Thanks so much for your time.


----------



## SirDice (Jun 28, 2014)

It's probably because adding or removing disks may change the order of the ada(4) or other disk devices. GELI may try to decrypt, for example, ada4 when it should do ada0 or vise verse.  The trick is to use labels instead of devices/partition references. Those labels will 'stick' regardless of the underlying device or partition name. 

Have a look at Handbook: 20.8. Labeling Disk Devices, in particular the `tunefs -L` command.


----------



## allanjude@ (Jun 28, 2014)

You likely will need to update your /boot/loader.conf (on the unencrypted bootpool) to identify the location of the new drives.


```
geli_%s_keyfile0_load
geli_%s_keyfile0_type
geli_%s_keyfile0_name
```

where %s is the partition (ada0p3).

I am not sure if this happens before or after any label providers, so you might be able to use a partition label so these track automatically.


----------



## sotehnn (Jun 28, 2014)

Okay thanks nearsourceit & SirDice. That seems reasonable, I'll play around with those settings.

I assumed that the device ID's remained constant as the disks remained in the same physical port (in fact I thought they were actually the port ID's). I'll try labelling the disks as well.

Thanks again.


----------



## allanjude@ (Jun 29, 2014)

sotehnn said:
			
		

> Okay thanks nearsourceit & SirDice. That seems reasonable, I'll play around with those settings.
> 
> I assumed that the device ID's remained constant as the disks remained in the same physical port (in fact I thought they were actually the port ID's). I'll try labelling the disks as well.
> 
> Thanks again.



The devices are named based on the port number, but if you have device 3 and 4 plugged in, they will be ada0 and ada1.

If you then plug in device 0 and 1, those will be ada0 and 1, and 3/4 become ada2/ada3.


----------



## wblock@ (Jun 29, 2014)

In other words, ada(4) devices use relative numbering.  The first one found is always ada0, the next one always ada1.  This is one reason labels are handy.


----------



## cracauer@ (Jun 30, 2014)

Careful now. There is a difference between ZFS and other filesystems here, and that is that a kernel booting a system with root on ZFS is still hardcoded to boot off (use as root) the first ZFS it finds.  I found that no amount of loader.conf editing, GELI keys or otherwise, would make you be able to boot off a non-first ZFS (or non-first-disk ZFS, I forgot?) the way you can do it with GELI below FFS/UFS.

So the HD fiddling must ensure that the desired boot disk is first to be found on the controller, which might be different from the port numbering on the circuit board, especially if it is an Asus board.


----------

