# gmirror replacing drive



## william21 (Sep 10, 2012)

I have a gmirror setup

```
mirror/gm4  COMPLETE  ad4
                      ad6
```
ad4 has failed a test 


```
1  Extended offline    Completed: read failure       90%     22781         58748720
```

so as a precaution I want too replace it.
If I power the system down and swap ad4 I know the system wont boot as it is the primary drive.
Do I move ad6 across too ad4 then plug the new drive in as ad6 ?
Thanks


----------



## michaelrmgreen (Sep 10, 2012)

According to this: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html
the system should boot (BIOS options notwithstanding). Can't say I've tried it myself. Backups agogo.


----------



## wblock@ (Sep 10, 2012)

Use the BIOS boot menu to choose ad6.


----------



## william21 (Sep 18, 2012)

Just in case it helps any one changing the bios too boot off ad6 did not work as it complained it could not boot off ad4.
Recabling ad6 onto the ad4 port and fitting a new drive on the ad6 port it booted perfectly.
Then I replaced the drive as per the docs

```
# gmirror forget gm4

# gmirror insert gm4 /dev/ad6
```

It is not happily rebuilding


```
gmirror status
      Name    Status  Components
mirror/gm4  DEGRADED  ad4
                      ad6 (73%)
```


----------



## william21 (Sep 18, 2012)

It is now rather


----------



## michaelrmgreen (Sep 28, 2012)

Hmmm, I had another look at the Handbook section about this (20.4.3 fact fans) and sure enough the failed drive in the example isn't the boot drive (/dev/da1). Now it rather seems to me that, notwithstanding BIOS objections, the system should be able boot from the non-boot drive when adjusted to do so.

Can anyone with experience of this failure mode (boot drive) shed any light on whether rearranging the cabling as per william21 is always necessary or whether there's another way.

[Edit] In fact this, slightly out-of-date, article: http://lantech.geekvenue.net/chucktips/jason/chuck/1175552464/index_html
does suggest that the system will restart with a failed boot disk in at least some circumstances.


----------



## wblock@ (Sep 28, 2012)

There are two things going on.  The first is when the computer boots, the BIOS doesn't know about gmirror, it's just looking for a single drive.  With a functioning mirror, you can boot from any of the drives in the mirror.  Since they are all identical, it doesn't matter which is used.

The second is the misuse of gmirror forget.  That is meant to remove drives that are no longer attached, not remove a particular drive that is still attached.  When a drive is failing, power down, disconnect the bad drive, and connect a good blank* drive at least as large.  Boot from the remaining drive.  The mirror status will be degraded.  Then do the forget:
`# gmirror forget gm0`
Note that you don't tell it which drive to forget, it just forgets all that aren't currently connected.

The mirror status will be active, and the bad drive is no longer part of the mirror.  Then insert the new drive into the mirror:
`# gmirror insert gm0 ada2`

The mirror will rebuild.

*: it's important that the new drive not have any old metadata on it before adding it to the mirror, which can make for mysterious errors when rebooting later.  dd(1) can be used to erase the last 34 blocks or so, which will erase the common ones: old gmirror, glabel, or GPT metadata.  For example, to erase that metadata on ada2:

```
# diskinfo -v ada2 | grep 'mediasize in sectors'
	500118192   	# mediasize in sectors
# dd if=/dev/zero of=/dev/ada2 seek=500118158
```
Calculate that seek value as mediasize minus 34.


----------



## william21 (Sep 28, 2012)

wblock@ said:
			
		

> There are two things going on.  The first is when the computer boots, the BIOS doesn't know about gmirror, it's just looking for a single drive.  With a functioning mirror, you can boot from any of the drives in the mirror.  Since they are all identical, it doesn't matter which is used.
> 
> The second is the misuse of gmirror forget.  That is meant to remove drives that are no longer attached, not remove a particular drive that is still attached.  When a drive is failing, power down, disconnect the bad drive, and connect a good blank* drive at least as large.  Boot from the remaining drive.  The mirror status will be degraded.  Then do the forget



That was what I tried. Shutdown replaced ad4 with brand new  drive. Configured bios too boot off ad6 which it started too do then fell over complaining about missing ad4. I should have noted the exact error but was under pressure too get system back up.
But moving ad6 over too ad4 was fine.
The way I see it the drives are not equal and ad6 is just a clone of ad4 but as you can see i am no expert on these matters. But I did read some where that it was necessary if the boot drive failed too move the other one across


----------



## kpa (Sep 28, 2012)

As far as the gmirror(8) service is concerned, the drives are equal. The BIOS and the boot loaders (that know nothing of the mirror) may get confused if the first disk is unreadable in a certain way that allows loading of the first stage of the boot loader but prevents booting any further than that.


----------



## william21 (Sep 28, 2012)

In that case maybe it was the way I configured in the first place that is wrong as it was the boot loader part that fell over


----------



## wblock@ (Sep 28, 2012)

Maybe forgot to change /etc/fstab to point to the mirror instead of a particular drive?  That would have had to be when the mirror was created, and assumes it was a copy of a single drive, but it's a possibility.


----------



## william21 (Sep 28, 2012)

No fstab is correct


```
more /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/mirror/gm4s1b              none            swap    sw              0       0
/dev/mirror/gm4s1a              /               ufs     rw              1       1
/dev/mirror/gm4s1e              /tmp            ufs     rw              2       2
/dev/mirror/gm4s1f              /usr            ufs     rw              2       2
/dev/mirror/gm4s1d              /var            ufs     rw              2       2
/dev/acd0               /cdrom          cd9660  ro,noauto       0       0
```


```
gmirror status
      Name    Status  Components
mirror/gm4  COMPLETE  ad4
                      ad6
```


----------

