# "Invalid argument" when mounting ext2fs on a zvol snapshot



## jimbobmcgee (May 23, 2014)

I'm having trouble mounting an EXT2 partition, and am not sure what else to try.  On my FreeNAS (FreeBSD 9.2) lab device, I have a ZFS pool with two zvols (zvol1 and zvol2).  I have shared these with iSCSI to a Linux machine (CentOS 6.4).  The Linux machine sees these as /dev/sdb and /dev/sdc.

Using the Linux machine, I have formatted /dev/sdb with an EXT2 filesystem, directly to the device (no partitioning).  If I then make a snapshot on the NAS, I can mount that snapshot with `mount -r -t ext2fs /dev/zvol/pool/zvol1@snap /tmp/zvol1`.  All well and good.

However, with /dev/sdc, I have labelled and partitioned it with a msdos/primary partition, then formatted  /dev/sdc1 with EXT2.  If I make a snapshot of that zvol, I can see two new block devices: /dev/zvol/pool/zvol2@snap and /dev/zvol/pool/zvol2@snaps1.  I guessed that these represented the 'disk' and the 'partition', and that *s1 is the partition, but when I run `mount -r -t ext2fs /dev/zvol/pool/zvol2@snaps1 /tmp/zvol2`, it fails with "Invalid argument".

`dmesg` additionally shows:

```
ext2fs: zvol/vol01/zvol2@snaps1: wrong magic number 0 (expected 0xef53)
```

A `gpart show` of the snapshot clearly shows the Linux partition:

```
=>      63  16777153  zvol/pool/zvol2@snap  MBR  (8.0G)
        63       193                           - free -  (96k)
       256   2096896                        1  linux-data  (1G)
   2097152  14680064                           - free -  (7.0G)
```

...but the output of `file -s` suggests I am wrong about what that *s1 block device actually is:

```
# file -s /dev/zvol/pool/zvol1@snap
/dev/zvol/pool/zvol1@snap: Linux rev 1.0 ext2 filesystem data (mounted or unclean), UUID=a756aca7-470d-47d2-a021-97b904aa2f5b (large files)

# file -s /dev/zvol/pool/zvol2@snap
/dev/zvol/pool/zvol2@snap: x86 boot sector; partition 1: ID=0x83, starthead 4, startsector 256, 2096896 sectors, code offset 0xb8

# file -s /dev/zvol/pool/zvol2@snaps1
/dev/zvol/pool/zvol2@snaps1: data
```

So if that *s1 block device is _not_ the partition, what is it, where _is_ my EXT2 partition's block device, and what should I actually be doing to mount the filesystem within FreeBSD?

(Note that I have also tried with a clone of the snapshot, with the same result)


----------



## jimbobmcgee (Jun 4, 2014)

*Re: "Invalid argument" when mounting ext2fs on a zvol snapsh*

APologies for the bump, but it's been a couple of weeks now, and I'm still struggling with this...


----------



## jimbobmcgee (Jun 6, 2014)

*Re: "Invalid argument" when mounting ext2fs on a zvol snapsh*

OK, I'm a bit closer.  I think it's a 4K sector-size issue.

If I do `geom part list`, among the various devices, I can see:

```
Geom name: zvol/vol01/zvol2@snap
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 16777215
first: 63
entries: 4
scheme: MBR
Providers:
1. Name: zvol/vol01/zvol2@snaps1
   Mediasize: 1073610752 (1G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 131072
   Mode: r0w0e0
   rawtype: 131
   length: 1073610752
   offset: 131072
   type: linux-data
   index: 1
   end: 2097151
   start: 256
Consumers:
1. Name: zvol/vol01/zvol2@snap
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Mode: r0w0e0
```

I can see from this that `geom` thinks my sector size is 512.  I'm pretty sure I created the original zvol with a block size of 4K, which I _think_ iSCSI has advertised as a 4K-sector device to the CentOS VM.  

If I do: `gnop create -o 1m -S 4k -s 1073610752 /dev/zvol/vol01/zvol2@snap` nad then examine that nop device with `file -s [email=/dev/zvol/vol01/zvol2@snap.nop]/dev/zvol/vol01/zvol2@snap.nop[/email]`, I can see more "Linux-y" output (rather than the generic 'data'):

```
/dev/zvol/vol01/zvol2@snap.nop: Linux rev 1.0 ext2 filesystem data (mounted or unclean), UUID=86741afe-5a22-4d9a-af14-8452f794298c (large files)
```

I would think this might get me closer, but mounting the nop device with `mount -t ext2fs -r [email=/dev/zvol/vol01/zvol2@snap.nop]/dev/zvol/vol01/zvol2@snap.nop[/email]` still yields "Invalid argument" (this time with no additional message in `dmesg`.

I will create a zvol with a 512 block size and see if the problem persists...

In the meantime, what else can I try?


----------



## bsdkeith (Jun 6, 2014)

*Re: "Invalid argument" when mounting ext2fs on a zvol snapsh*

I'm new to FreeBSD, but if I am reading your problem right, you want to get at data that was written by a Linux machine/distro to your disk. 
I see a 'mounted or unclean' in amongst your info posted that suggests to me that it needs the filesystem checked, & I think you will have to do this using a Linux distro. 
So my suggestion is to get hold of a 'live' distro to try & fsck your disk & recover the data; how valuable is that data(?) this 'could' maybe have bad consequences on the actual disk filesystem as per any such action.
 If you don't know any linux distros, may I suggest antiX http://antix.mepis.org/index.php?title= ... #Downloads (a Debian based live distro), put it on a cdrom or a pendrive & boot from it to check the filesystem. I hope I'm not 'trying to teach my grandmother how to suck eggs'; just that you don't seem to have had any suggestions.


----------



## jimbobmcgee (Jun 9, 2014)

*Re: "Invalid argument" when mounting ext2fs on a zvol snapsh*

I ran some tests over the weekend, sharing various zvols with iSCSI (istgt), with a mix of `BlockLength 512` and `BlockLength 4096` in the istgt.conf file.

I made the following observations:

 If I wrote an ext2 filesystem directly to a 4K target, I could snapshot and `mount -r` locally without issue
 If I wrote an ext2 filesystem directly to a 512 target, I could snapshot and `mount -r` locally without issue
 If I wrote a partition table to a 4K target, mounting the ext2 filesystem on the snapshot's *s1 device resulted in "Invalid argument"
 If I wrote a partition table to a 512 target, I could snapshot and `mount -r` the *s1 device locally without issue
Given this, I'm inclined to believe this might be one of two things:

 a problem with the discovery process that automatically creates the block devices for partitions (devd?).  Once I really started looking at the output of `gpart list`, I could see a number of discrepancies: Sectorsize, Mediasize, Stripeoffset all looked wrong on the 4K snapshots and all looked correct on the 512 snapshots (although that doesn't mean I am reading them correctly).
 a problem with GNU parted (on the CentOS VM) during creation of the partition table on a 4K device.  It looks like real, physical 4K disks have a 512-emulating layer over the top of them for exactly this reason (which is shown as 512 logical / 4K physical in parted), but the iSCSI target just presents itself a 4K logical / 4K physical device, which GNU parted does not yet fully understand.
I wonder if there is a sure-fire way to tell which one is the problem?

I also wonder if there is a `gnop` command that might actually identify the data correctly.  I have tried `gnop create -v -S 4096 -o 1048576 -s 8588886016 /dev/zvol/vol01/zvol2@snap` -- which is the correct offset, sector size and size according to parted -- to no avail.

Or does `gnop` not do what I think it does?


----------

