# ZFS send/receive between FreeBSD 9 and a Solaris box (stream unsupported version)



## Boeri (Jun 29, 2011)

Dear all,

I'm getting the following error when I try to do a zfs send/receive on a solaris (pool v26) to freebsd box (pool v28):
"cannot receive: stream is unsupported version 18"
Any solution for this? 

- My questions:
Why can't I create a pool version lower then v28 on zfs? Why do I get the maximum allowed pool version is v15 with the zfs upgrad command? If a create a default pool it is v28, this is normal on freebsd current. 


solaris host
--------------

```
root@solaris:~# zfs send -R firstbackup/users@send | ssh root@192.168.250.14 zfs receive -dF remotepool
Password:
cannot receive: stream is unsupported version 18

root@solaris:~# zpool get version firstbackup                                  NAME         PROPERTY  VALUE    SOURCE
firstbackup  version   26       default
```

freebsd  host
---------

```
[root@ ~]# zpool get version remotepool
NAME        PROPERTY  VALUE    SOURCE
remotepool  version   28       default

[root@ ~]# zpool create -o version=26 remotepool /dev/ada1
cannot create 'remotepool': property 'version' number 26 is invalid.

[root@ ~]# zpool upgrade -v
This system is currently running ZFS pool version 15.

[root@ ~]# uname -a
FreeBSD  9.0-CURRENT FreeBSD 9.0-CURRENT #1: Tue Jun 28 14:40:11 UTC 2011     root@:/usr/obj/usr/src/sys/SANKERNEL  amd64
```


The following versions are supported:


----------



## AndyUKG (Jun 29, 2011)

Boeri said:
			
		

> ```
> [root@ ~]# zpool upgrade -v
> This system is currently running ZFS pool version 15.
> ```



That looks like your system is only running pool version 15, odd as the pool itself is reporting 28. What is the output from: `# grep -i zfs /var/run/dmesg.boot`

Andy.


----------



## usdmatt (Jun 29, 2011)

Is this a FreeBSD 8.2 machine that you have upgraded to 9.0?
If so I'd say that the problem is not with ZFS itself but with the FreeBSD upgrade not completing correctly.
As AndyUKG said the system still thinks it's running v15.

I have a test machine running 9.0 that was installed from scratch using a 9.0 snapshot ISO and have the expected output from the commands you ran:


```
test# zpool upgrade -v |head
This system is currently running ZFS pool version 28.
```


```
test# zpool create -o version=26 test md0
test# zpool get all test
NAME  PROPERTY       VALUE       SOURCE
test  size           59.5M       -
test  capacity       0%          -
test  altroot        -           default
test  health         ONLINE      -
test  guid           17052720165871221467  local
test  version        26          local
...
```

I still don't know how you managed to get a v28 pool on there though.


----------



## Boeri (Jun 29, 2011)

```
[root@ ~]# grep -i zfs /var/run/dmesg.boot
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
            to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version 5
ZFS storage pool version 28
```


----------



## AndyUKG (Jun 29, 2011)

Ok, so the output of zpool upgrade is wrong, but when ZFS starts at boot its showing v28. Definately something amiss with your installation, I'd suggest a make world is probably necessary, think this is the relevant info:

http://www.freebsd.org/doc/handbook/makeworld.html


----------



## Boeri (Jun 29, 2011)

Great guys!
It is working now. This system was indeed upgraded from 8.2 version. I forgot the "make installworld". 


```
[root@ ~]# zpool upgrade -v
This system is currently running ZFS pool version 28.
```

 Thanks!


----------



## SirDice (Jun 29, 2011)

Don't forget to run mergemaster(8) too.


----------



## Boeri (Jun 29, 2011)

Thanks. I ran the *mergemaster* command. 

Replication is fine, but I have a problem accessing my replicated volume. When I'm typing and autocomplete with tab key the command *cd /remotepool/us* (for /remotepool/users) I get a panic.

  -> autocomplete tab does a *ls* in the background, I think 

I think there is a problem with NFSv4 acl/id mapping. Normal zfs (inititally created on the FreeBSD box) file systems are working fine. 








```
[root@ ~]# zfs get all remotepool/users
NAME              PROPERTY              VALUE                    SOURCE
remotepool/users  type                  filesystem               -
remotepool/users  creation              Wed Jun 29 14:42 2011    -
remotepool/users  used                  9.06G                    -
remotepool/users  available             187G                     -
remotepool/users  referenced            9.06G                    -
remotepool/users  compressratio         1.00x                    -
remotepool/users  mounted               yes                      -
remotepool/users  quota                 none                     default
remotepool/users  reservation           none                     default
remotepool/users  recordsize            128K                     default
remotepool/users  mountpoint            /remotepool/users        default
remotepool/users  sharenfs              off                      default
remotepool/users  checksum              on                       default
remotepool/users  compression           off                      default
remotepool/users  atime                 on                       default
remotepool/users  devices               on                       default
remotepool/users  exec                  on                       default
remotepool/users  setuid                on                       default
remotepool/users  readonly              off                      default
remotepool/users  jailed                off                      default
remotepool/users  snapdir               hidden                   received
remotepool/users  aclinherit            passthrough              received
remotepool/users  canmount              on                       default
remotepool/users  xattr                 off                      temporary
remotepool/users  copies                1                        default
remotepool/users  version               5                        -
remotepool/users  utf8only              off                      -
remotepool/users  normalization         none                     -
remotepool/users  casesensitivity       insensitive              -
remotepool/users  vscan                 off                      default
remotepool/users  nbmand                on                       received
remotepool/users  sharesmb              name=users,guestok=true  received
remotepool/users  refquota              none                     default
remotepool/users  refreservation        none                     default
remotepool/users  primarycache          all                      default
remotepool/users  secondarycache        all                      default
remotepool/users  usedbysnapshots       0                        -
remotepool/users  usedbydataset         9.06G                    -
remotepool/users  usedbychildren        0                        -
remotepool/users  usedbyrefreservation  0                        -
remotepool/users  logbias               latency                  default
remotepool/users  dedup                 off                      default
remotepool/users  mlslabel                                       -
remotepool/users  sync                  standard                 default
```


----------



## SirDice (Jun 29, 2011)

Why are you running -CURRENT? If it's only for ZFS, version 28 has been MFC'ed to -STABLE recently.

Even though -CURRENT will soon be 9.0-RELEASE it is still a work in progress.


----------



## Boeri (Jun 29, 2011)

Fine, I'll try a replication to a v28 pool on a virtual FreeBSD 8.2 box. I'll do it tomorrow. 

We want to move from an Active Directory integrated Solaris/Nexenta box to a FreeBSD box with Samba and AD integration using the same (replicated) pool. So the replication with the NFSv4 ACL's included is very important.


----------



## AndyUKG (Jun 29, 2011)

I think NFSv4 style ACLs are fundamental to ZFS, so there's no question of Solaris having this and FreeBSD not.
Regarding changing from CURRENT, 8.2 is the current RELEASE which doesn't have ZFS v28 (you can add it via a patch though). The other alternative which I think SirDice was suggesting is STABLE which will be FreeBSD 9 also, but a more "stable" version than CURRENT...

Andy.


----------



## SirDice (Jun 30, 2011)

AndyUKG said:
			
		

> The other alternative which I think SirDice was suggesting is STABLE which will be FreeBSD 9 also, but a more "stable" version than CURRENT...


No, -STABLE is currently 8.2-STABLE. At a certain point in time 8.3-RELEASE will be made from it and -STABLE will move to 8.3-STABLE[*].

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html


[*] That's assuming there will eventually be a 8.3-RELEASE.


----------



## Boeri (Jun 30, 2011)

I've installed the ISO from http://mfsbsd.vx.sk/ "8.2-RELEASE-p2 amd64 with ZFSv28 special edition (100MB)" in a virtual machine.

After replication, the machine also gives a panic. This is a big problem because I'cant replicate the NFSv4 acl's with rsync, only with zfs send/receive.


----------



## AndyUKG (Jun 30, 2011)

I haven't experimented with any cross-platform stuff on ZFS myself, but I believe it should work in theory. Probably worth posting the issue on "zfs discuss" as you will find more ZFS experts there plus Solaris people.


----------



## Boeri (Jul 1, 2011)

Thanks. I've followed your advice and opened a topic on zfs-discuss. 
Maybe rsync is a temporary solution, and copy the acl's with the windows (i)cacls tool... 
http://opensolaris.org/jive/thread.jspa?threadID=140181&tstart=0


----------

