# Add a second zfs boot disk after install



## pos (Jul 15, 2019)

Hi

I have a firewall with a 32 GB msata disk. I have another and new disk on the shelf that I will add to the firewall. 
I want this to be a 100% moirrored disk that I can boot from by just changing boot order in bios.


This is what it looks like before I have added the second msata....


```
13:07:26 xanadu:~ # gpart show -l
=>      40  61865904  ada0  GPT  (30G)
        40      1024     1  gptboot0  (512K)
      1064       984        - free -  (492K)
      2048   4194304     2  swap0  (2.0G)
   4196352  57667584     3  zfs0  (27G)
  61863936      2008        - free -  (1.0M)


13:05:48 xanadu:~ # zpool list -v
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot         27G  5.38G  21.6G        -         -     7%    19%  1.00x  ONLINE  -
  ada0p3      27G  5.38G  21.6G        -         -     7%    19%
13:06:30 xanadu:~ #
```



Now lets say I have added the new second disk...... I need some help with the steps to avoid reinstall  
I have never done this with zfs+FreeBSD on the root disk and just want to be careful...


# 1 # - Clone partition table
I assume I first have to clone my partition table. According to the man page of gpart I could do it with:
`/sbin/gpart backup ada0 | /sbin/gpart restore -F ada1`

# 2 # - Set up mirror
I guess this will work directly, but I cannot boot on the new disk if trying as I have no boot block. Right?
Will this use just partition 3 and leave the others (i.e not destroying my other partitions)? I mean... If the 
second disk would have been blank, it would have created the partion with this command.
`zpool attach zroot ada0 ada1`

# 3 # - Make new disk bootable
Well... what is the proper way for me doing this based on my printed disk data above?
`gpart bootcode <what...> ada1`


# 4 # - swap
will the swap work out of the box if booting on the second disk as the partiton is cloned? Or does it need any other prepp?

# 5 # - Make second disk the second boot option in bios



Any pro:s that can help me avoid a disk recovery caused by me doing the wrong things  

Tnx in advance
/Peo


----------



## SirDice (Jul 15, 2019)

1) Don't clone them exactly, the filesystem labels will clash (two partitions with the same label, one on each disk). It'll be better if you use an intermediate file; `gpart backup ada0 > part-table.txt`; then check and modify where needed before restoring to the other disk. 

3) Same command as for one disk; `/sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1`


----------



## ucomp (Jul 15, 2019)

pos said:


> ...
> 
> I cannot boot on the new disk ...




```
zpool get bootfs
```


```
zpool set bootfs
```


----------



## pos (Jul 15, 2019)

SirDice said:


> 1) Don't clone them exactly, the filesystem labels will clash (two partitions with the same label, one on each disk). It'll be better if you use an intermediate file; `gpart backup ada0 > part-table.txt`; then check and modify where needed before restoring to the other disk.
> 
> 3) Same command as for one disk; `/sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1`



Thanks. I will do a backup and then import to the new disk.  And I will rewrite the labels in between... 

Apart from what you already have answered, is there any other major thing I need to think of? I mean, anything I have not asked due to lack of knowledge of the FreeBSD+bootzfs+mirror  ?


PS
Should the bootcode index be "1" on disk 2?

Tnx
/Peo


----------



## pos (Jul 15, 2019)

ucomp said:


> ```
> zpool get bootfs
> ```
> 
> ...



Hmmm. Any reason why this should be needed? The zfs pool already exist.


----------



## SirDice (Jul 15, 2019)

pos said:


> Should the bootcode index be "1" on disk 2?


Look at the index of the freebsd-boot partition, that's where it needs to go. Typically it has index 1 but this doesn't have to be the case. The 'index', in this case, refers to the numbering of the partitions on the disk, not the boot order or something similar.

In my case for example, the freebsd-boot partition has index 2, because it's the _second_ partition on disk.

```
dice@williscorto:~ % gpart show
=>       40  625142368  ada0  GPT  (298G)
         40     409600     1  efi  (200M)
     409640       1024     2  freebsd-boot  (512K)
     410664        984        - free -  (492K)
     411648   16777216     3  freebsd-swap  (8.0G)
   17188864  607952896     4  freebsd-zfs  (290G)
  625141760        648        - free -  (324K)
```


----------



## pos (Jul 15, 2019)

SirDice said:


> Look at the index of the freebsd-boot partition, that's where it needs to go. Typically it has index 1 but this doesn't have to be the case. The 'index' refers to the numbering of the partitions, not the boot order or something similar.



Ahhh, yes... Of course   It´s the partition number for freebsd-boot. Now it is 1. And if I clone the disk it will be the same. I.e 1.

Tnx


----------



## ucomp (Jul 15, 2019)

pos said:


> Any reason ...


Many roads lead to Rome as it seems , I mostly go
this one , if necessary.


----------



## pos (Jul 15, 2019)

I have now added the second disk and done the following, and it at least seems ok....


`gpart backup ada0 > /root/gpart-backup-ada0-part-table.txt`
vi /root/gpart-backup-ada0-part-table.txt (change lables names from <name>0 to <name>1. Maybw not needed)
`gpart restore -l -F ada1 < /root/gpart-backup-ada0-part-table.txt`



```
18:17:59 xanadu:~ # gpart show -l
=>      40  61865904  ada0  GPT  (30G)
        40      1024     1  gptboot0  (512K)
      1064       984        - free -  (492K)
      2048   4194304     2  swap0  (2.0G)
   4196352  57667584     3  zfs0  (27G)
  61863936      2008        - free -  (1.0M)

=>      40  61865904  ada1  GPT  (30G)
        40      1024     1  gptboot1  (512K)
      1064       984        - free -  (492K)
      2048   4194304     2  swap1  (2.0G)
   4196352  57667584     3  zfs1  (27G)
  61863936      2008        - free -  (1.0M)
```



18:05:39 xanadu:~ # /`sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1`
partcode written to ada1p1




18:10:01 xanadu:~ #`zpool attach zroot ada0p3 ada1p3`
Make sure to wait until resilver is done before rebooting.

If you boot from pool 'zroot', you may need to update
boot code on newly attached disk 'ada1p3'.

Assuming you use GPT partitioning and 'da0' is your new boot disk
you may use the following command:

    gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

18:10:37 xanadu:~ #


```
18:14:35 xanadu:~ # zpool list -v
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot         27G  5.38G  21.6G        -         -     7%    19%  1.00x  ONLINE  -
  mirror      27G  5.38G  21.6G        -         -     7%    19%
    ada0p3      -      -      -        -         -      -      -
    ada1p3      -      -      -        -         -      -      -
18:14:39 xanadu:~ #
18:14:39 xanadu:~ #
18:14:40 xanadu:~ # zpool status
  pool: zroot
state: ONLINE
  scan: resilvered 5.38G in 0 days 00:02:02 with 0 errors on Mon Jul 15 18:12:38 2019
config:

    NAME        STATE     READ WRITE CKSUM
    zroot       ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        ada0p3  ONLINE       0     0     0
        ada1p3  ONLINE       0     0     0

errors: No known data errors
18:14:41 xanadu:~ #
```


Would this be ok and safe to reboot on the other or any of these two disks now?

Note that the zpool attach suggests "-b /boot/pmbr" when using the gpart bootcode command. Maybe this is not applicable here? Or is it?

*I guess I have to also add "-b /boot/pmbr" as I have a GPT partitioning scheme and boot with bios that if I do not have the wrong info has the following boot sequence...
BIOS -> /boot/pmbr -> /boot/gptboot (or /boot/gptzfsboot) -> /boot/loader (or /boot/zfsloader) -> FreeBSD Kernel

Please enlighten me...*

/Peo


----------



## pos (Aug 3, 2019)

To close the thread and maybe help others... 


I did exactly as above, but used 
	
	



```
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
```
 instead (i.e added -b) as I use bios boot.

Works flawlessly to boot on any of the disks. 
So... SOLVED !


----------

