# moving ZFS mirror root from ATA to ATA CAM



## akitaro (Oct 31, 2010)

Hello!

I use a box with two atapi hard drives gpt-partitioned as:


```
zoom# gpart show
=>       34  156301421  ad0  GPT  (75G)
         34        128    1  freebsd-boot  (64K)
        162    3145728    2  freebsd-swap  (1.5G)
    3145890  153155565    3  freebsd-zfs  (73G)

=>       34  156301421  ad1  GPT  (75G)
         34        128    1  freebsd-boot  (64K)
        162    3145728    2  freebsd-swap  (1.5G)
    3145890  153155565    3  freebsd-zfs  (73G)
```

ad0p3 and ad1p3 are mirrored with ZFS mirror pool:


```
zoom# zpool status
  pool: zroot
 state: ONLINE
 scrub: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        zroot          ONLINE       0     0     0
          mirror       ONLINE       0     0     0
            gpt/disk0  ONLINE       0     0     0
            gpt/disk1  ONLINE       0     0     0

errors: No known data errors
```

Everything works fine, but now I'd like to switch from ata driver to ata cam. I build new kernel with 
	
	



```
device atapicam
```
 and 
	
	



```
options ATA_CAM
```
 It starts but after hardware detection it breaks to loader because of root fs mount fail.

I think this is because there are no ad0 and ad1 drives now, but ada0 and ada1. So my question is how can I now reconfigure zfs pool to use ada â€” not ad â€” disks?

Thank you.

andy


----------



## Galactic_Dominator (Oct 31, 2010)

That's shouldn't matter at all.  Is 

```
vfs.root.mountfrom="zfs:<zpool_name>"
```
in your /boot/loader.conf?


----------



## akitaro (Nov 2, 2010)

Galactic_Dominator said:
			
		

> That's shouldn't matter at all.  Is
> 
> ```
> vfs.root.mountfrom="zfs:<zpool_name>"
> ...



Sure! How does my current (atapi, not cam) kernel work? Of course, this line present in /boot/loader.conf.


----------



## Galactic_Dominator (Nov 2, 2010)

I don't really know, perhaps your new kernel is broken in other ways which causes the error.  You do have 
	
	



```
device ata
```
?

ATA_CAM is not really an upgrade though, you'd be better of returning to GENERIC kernel, and loading ahci from loader and see if that works for you.


----------



## akitaro (Nov 5, 2010)

Galactic_Dominator said:
			
		

> I don't really know, perhaps your new kernel is broken in other ways which causes the error.  You do have
> 
> 
> 
> ...



1st: According to /usr/src/sys/conf/NOTES

```
# ATA_CAM:              Turn ata(4) subsystem controller drivers into cam(4)
#                       interface modules. This deprecates all ata(4)
#                       peripheral device drivers (atadisk, ataraid, atapicd,
#                       atapifd. atapist, atapicam) and all user-level APIs.
#                       cam(4) drivers and APIs will be connected instead.
```

2nd: I need no ahci, this box has no sata controller (it is a little-bit old).

I've just got the experiment, building new GENERIC kernel with just single modification â€” add 
	
	



```
options      ATA_CAM
```
 It starts, then reports
	
	



```
Trying to mount root from zfs:root
ROOT MOUNT ERROR
```
and breaks. So the problem is not in my custom kernel.


PS: RELENG_8, synced and built four days ago.


----------



## Savagedlight (Nov 5, 2010)

I see you're trying to mount root from zfs:root, while your pool is named zroot.
Shouldn't you be trying to mount root from zfs:*z*root?


----------



## akitaro (Nov 6, 2010)

Savagedlight said:
			
		

> I see you're trying to mount root from zfs:root, while your pool is named zroot.
> Shouldn't you be trying to mount root from zfs:*z*root?



It was my mistype just here (I can't cut text from loader, can you? 

Read my above messages and you'll see â€” everything works fine with GENERIC kernel (and this fact proves that /boot/loader.conf is correct), but does not after rebuilding GENERIC with ATA_CAM option.


----------



## t1066 (Nov 6, 2010)

```
# ATA_CAM:              Turn ata(4) subsystem controller drivers into cam(4)
#                       interface modules. This deprecates all ata(4)
#                       peripheral device drivers (atadisk, ataraid, atapicd,
#                       atapifd. atapist, [U][B]atapicam[/B][/U]) and all user-level APIs.
#                       cam(4) drivers and APIs will be connected instead.
```

As you can see atapicam is deprecate with ATA_CAM, I would suggest remove


```
device atapicam
```

from kernelconf and try again.


----------



## akitaro (Nov 6, 2010)

t1066 said:
			
		

> ```
> # ATA_CAM:              Turn ata(4) subsystem controller drivers into cam(4)
> #                       interface modules. This deprecates all ata(4)
> #                       peripheral device drivers (atadisk, ataraid, atapicd,
> ...



As I've mentioned above I use GENERIC kernel with 
	
	



```
options    ATA_CAM
```
 GENERIC kernel does not include atapicam.


----------



## phoenix (Nov 6, 2010)

The problem is a bit of a chicken-and-egg situation.

ata(4) devices are *ad#*, while *ATA_CAM* devices are *ada#*.

When you change the name of the devices underneath a ZFS pool, you need to export/import the pool.  _However_, with / on the pool, you can't export the pool.

You need to find a LiveCD that uses ATA_CAM, so that you can boot, detect your drives as ada devices.  Then you can export the pool, import the pool so that ZFS re-reads the metadata on the drives and configures things with the ada devices, and then you can boot normally.

Or, just continue to use a non-ATA_CAM kernel.


----------



## akitaro (Nov 6, 2010)

phoenix said:
			
		

> The problem is a bit of a chicken-and-egg situation.
> 
> ata(4) devices are *ad#*, while *ATA_CAM* devices are *ada#*.
> 
> ...



There was my concerns, bu I was hoping for gpt labels (I use gpt/disk1 and gpt/disk2, not /dev/daNNN for creating zfs pool, as you can see). Alas, they do not work. Perhaps I do not understand them enough.

Thank you, phoenix!


----------



## nsayer (Nov 28, 2010)

I'm not 100% sure this will help, but I have my zpool set up to use gptid. In principle this means that I can even swap the disks around between controllers and it should still work.

When I migrated from UFS, what I did was set up the zfs tank on ada1, then swapped it around and booted from it, then relabeled what was now ada1 (the old ada0) to be the same partitioning scheme as ada0, then added the 2nd disk's zfs partition as a mirror.

When I first did this, I noticed that the system had decided that it was booting from /dev/gptid/blah-blah-blah. When I attached the 2nd one, I used 'glabel list' to find it's GPT ID and added it that way.

If you have a mirrored system, you might try detaching one of them and reattaching it by it's gpt id. When it's done resilvering, you should be able to detach and reattach the other one. Once that's done, you should be able to swap out your kernel and the device name changes shouldn't matter.


----------



## sash (Jan 17, 2011)

phoenix said:
			
		

> The problem is a bit of a chicken-and-egg situation.
> 
> ata(4) devices are *ad#*, while *ATA_CAM* devices are *ada#*.
> 
> ...



In fact it will NOT help. Replace kernel with ATA_CAM kernel is enough. But now it doesn't work because of bug in kernel PR kern/153996


----------



## Sebulon (Feb 18, 2011)

To whom it may concern...

I have now successfully moved all my zpools from ATA to ATA_CAM, including my root pool, using this as a how to, and it works, flawlessly! ItÂ´s beautiful, I can hot-swap my drives without having to fiddle with atacontrol any more. I just wanted to express my gratitude for this.

/Sebulon


----------

