# Need to rescue GMIRROR soft RAID



## FreeDomBSD (Sep 17, 2013)

```

```
I have four disks that are divided into two GMIRROR RAIDs.

This is a really old system that had malfunctioning hardware and I retired it few years ago. I'd like to regain access to the files and copy them onto a new hard drive.

How to I do that with four hard drives and one OS drive only (on another system)?


----------



## wblock@ (Sep 17, 2013)

It depends on how the system failed.  If the drives are corrupted or failing, it's a recovery operation.  If the drives and data are okay, they can be connected to a current system.

gmirror(8) support is part of the standard kernel.


----------



## FreeDomBSD (Sep 17, 2013)

Another headache is that my facilities to perform this rescue are Ubuntu and Windows. I have a multi-drive external USB enclosure that I was planning to load the four hard drives into and perform a rescue in Ubuntu.

Is gmirror(8) supported under Ubuntu?


----------



## J65nko (Sep 17, 2013)

FreeDomBSD said:
			
		

> Is gmirror(8) supported under Ubuntu?


You are joking, right 

You could use a installation DVD/memstick and choose Live CD. That gives you a root shell.

The alternative would be to install FreeBSD on an USB stick, boot it and you have all the FreeBSD utilities available.


----------



## wblock@ (Sep 17, 2013)

It's not necessary to connect all four drives at once.  In fact, you only need one drive from each mirror, and both mirrors don't have to be done at the same time.  I'd install FreeBSD on a blank, new drive, then connect one of the old mirror drives, mount it read-only, and copy off any usable data.  If the drive was corrupted, use dd(1) to make an image into a file on the new FreeBSD drive for recovery.  Then repeat for one drive from the second mirror.


----------



## FreeDomBSD (Sep 17, 2013)

wblock@ said:
			
		

> It depends on how the system failed.  If the drives are corrupted or failing, it's a recovery operation.  If the drives and data are okay, they can be connected to a current system.
> 
> gmirror(8) support is part of the standard kernel.



Thank you for that link. Would `gmirror rebuild raid` be the correct command?


----------



## wblock@ (Sep 17, 2013)

If the drives are okay, no rebuild is necessary.


----------



## tingo (Sep 17, 2013)

And for drives that are failing, I recommend sysutils/dd_rescue, it works nicely even on drives with heat failures (do them with repeated attempts, with cool down time between).


----------



## FreeDomBSD (Sep 27, 2013)

The single drive showed up couple of times under /dev/mirror/raid1 but as I took the drive in and out of the hotswap bay it stopped showing up. I was trying to figure out which drive was bad and how to mount the remaining drive on the new freebsd FreeBSD system. Well, I figured out which drive was bad, but gmirror stopped appearing so now I'm really confused. How to call gmirror back? How to mount the gmirror drive?


----------



## J65nko (Sep 27, 2013)

Do `# gmirror status` or `# gmirror list` produce anything?


----------



## FreeDomBSD (Sep 27, 2013)

Negative. Both commands are unavailable. Drive is detected though.


----------



## J65nko (Sep 27, 2013)

Please verify whether the the kernel module geom_mirror has been loaded:

```
[cmd=#] kldstat[/cmd]
Id Refs Address            Size     Name
 1    9 0xffffffff80200000 1323780  kernel
 2    1 0xffffffff81524000 21910    geom_mirror.ko
```

This module can be loaded manually with `# kldload geom_mirror` or by having the following in /boot/loader.conf:

```
geom_mirror_load="YES"
```


----------



## FreeDomBSD (Sep 27, 2013)

Success. How do I mount the mirror in a folder? I tried `mount /dev/mirror/raid1 /1` with no avail.


----------



## wblock@ (Sep 27, 2013)

The mirror acts just like a drive.  Drives usually have more than one partition.  So don't mount the mirror, mount the partitions of the mirror.  Use `gpart show` to see the device names, then use it again on each slice to check for FreeBSD partitions, like `gpart show /dev/mirror/raid1s1`.


----------



## FreeDomBSD (Sep 27, 2013)

When I tried `mount /dev/mirror/raid1 /1` the complaint was that the file system is not clean. I ran `fsck -y /dev/da0` and got a complaint of not able to determine file system type. This is consistent with just running `gpart show` when it lists the drive in question as free of a file system and `gpart show /dev/mirror/raid1s1` and `gpart show /dev/mirror/raid1` get a complaint from gpart

```
No such geom: /dev/mirror/raid1
```


----------



## J65nko (Sep 27, 2013)

Now that the kernel mirror module has been loaded, what is the output of the two commands I mentioned in https://forums.freebsd.org/showpost.php?p=234232&postcount=10 ?

For example on one of my systems:


```
[cmd=#]gmirror status[/cmd]
      Name    Status  Components
mirror/gm0  COMPLETE  ada0 (ACTIVE)
                      ada1 (ACTIVE)
```
From this we learn that mirror/gm0 is the name of the mirror name in /dev. Using that name to find more info: 

```
[cmd=#] gpart show mirror/gm0[/cmd]
=>       63  781422704  mirror/gm0  MBR  (372G)
         63    8388576           1  freebsd  (4G)
    8388639  773034066           2  freebsd  [active]  (368G)
  781422705         62              - free -  (31k)
```
So here we have two slices mirror/gm0s1 and mirror/gm0s2 On slice one I have a rescue system of 4 GB, the server install is on the second slice:


```
[cmd=#] gpart show mirror/gm0s1[/cmd]
=>      0  8388576  mirror/gm0s1  BSD  (4G)
        0  7905280             1  freebsd-ufs  (3.8G)
  7905280   483296             2  freebsd-swap  (236M)

[cmd=#] gpart show mirror/gm0s2[/cmd]
=>        0  773034066  mirror/gm0s2  BSD  (368G)
          0    2097152             1  freebsd-ufs  (1.0G)
    2097152   20971520             4  freebsd-ufs  (10G)
   23068672   41943040             5  freebsd-ufs  (20G)
   65011712   10485760             6  freebsd-ufs  (5.0G)
   75497472  629145600             7  freebsd-ufs  (300G)
  704643072   62914560             8  freebsd-ufs  (30G)
  767557632    5476434             2  freebsd-swap  (2.6G)
```
Now we see the BSD disklabels where 1 => a, 2 => b, 3 => c (not used), 4 => d etc.

A snippet of /etc/fstab

```
# dev                   mount           FStype  Options                                 Dump    Pass#
#-----------            -----           -----   -------------------                     ----    ----
/dev/mirror/gm0s2a      /               ufs     rw,noatime                              1       1
/dev/mirror/gm0s2b      none            swap    sw                                      0       0
/dev/mirror/gm0s2d      /usr            ufs     rw,noatime                              2       2
```


```
[cmd=#]df -h[/cmd]
Filesystem            Size    Used   Avail Capacity  Mounted on
/dev/mirror/gm0s2a    991M    481M    430M    53%    /
devfs                 1.0k    1.0k      0B   100%    /dev
/dev/mirror/gm0s2d    9.7G    653M    8.3G     7%    /usr
```


----------



## FreeDomBSD (Sep 27, 2013)

`# gmirror status`


```
mirror/raid1  DEGRADED  da0 (ACTIVE)
```
 
`#  gpart show mirror/raid1`


```
No such geom: mirror/raid1
```


----------



## wblock@ (Sep 27, 2013)

I've seen similar confusing errors with mfsBSD, which somehow does not have all the kernel modules or code for mirrors, but shows them anyway.

You may have to boot into the live CD mode from an installer disk.


----------



## tingo (Sep 27, 2013)

What (if anything) does `gmirror list` give as output?


----------



## FreeDomBSD (Oct 6, 2013)

`# kldload geom_mirror`, `# mount /dev/mirror/raid /home/raid`, and got this error:


```
mount: /dev/mirror/raid: R/W mount of /usr/home/raid is denied. Filesystem is not clean - run fsck.: Operation not permitted
```

Then I struggled with the errors because fsck did not know the filesystem it was working with *I* e*v*entually excecuted this command: `# fsck -t ufs /dev/mirror/raid` and after it was done cleaning the filesystem: `# mount /dev/mirror/raid /home/raid`.


----------

