# Freebsd 13.0 with root on zfs on hyper-v 2012



## Diman (Jul 5, 2021)

Can't boot. Can't load kernel.
From comand prompt ls, lszfs, more work fine, but load /boot/kernel/kernel fails.

But it can boot if add one more disk with FreeBSD 13.0 installed on it.
Both base and kernel must be installed. With only kernel it can load kernel at boot, but can't load modules.

How to boot system with only one disk?


----------



## Alain De Vos (Jul 5, 2021)

Can you boot from a downloaded .iso/.img of a freebsd-install boot CD/USB image ?
I don't now how vhd(x) is presented to the freebsd kernel. But if you can boot from that you can layout partition schemes and install bootcode where hyper-v is looking.


----------



## SirDice (Jul 5, 2021)

Diman said:


> Can't boot. Can't load kernel.


Those are actually two different things. Can't boot means it is not even able to load loader(8), but not able to find the kernel is a loader(8) error message. So it does boot, it's just that loader(8) can't seem to find the kernel. What is the disk layout? And what's in /boot/loader.conf?


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> Can you boot from a downloaded .iso/.img of a freebsd-install boot CD/USB image ?


Yes, boot normally.



SirDice said:


> What is the disk layout? And what's in /boot/loader.conf?


gpt with 2 partitions, first boot and second system.
loader.conf:
zfs_load="YES"
vfs.root.mountfrom="zfs:sys"


----------



## SirDice (Jul 6, 2021)

Diman said:


> gpt with 2 partitions, first boot and second system.


So it's freebsd-boot and freebsd-zfs then?



Diman said:


> ```
> zfs_load="YES"
> vfs.root.mountfrom="zfs:sys"
> ```


Remove the `vfs.root.mountfrom`. The dataset to boot from should be set with `zpool set bootfs`.

```
root@molly:~ # zpool get bootfs zroot
NAME   PROPERTY  VALUE               SOURCE
zroot  bootfs    zroot/ROOT/default  local
```


----------



## Alain De Vos (Jul 6, 2021)

Is it a legacy-boot or an efi-boot ?


----------



## Diman (Jul 6, 2021)

SirDice said:


> So it's freebsd-boot and freebsd-zfs then?


Yes.



SirDice said:


> Remove the vfs.root.mountfrom. The dataset to boot from should be set with zpool set bootfs.


Does not help.
Loading kernel...
ZFS: i/o error - all block copies unavailable


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> Is it a legacy-boot or an efi-boot ?


legacy.
hyper-v 2012 does not support efi.


----------



## SirDice (Jul 6, 2021)

Diman said:


> ZFS: i/o error - all block copies unavailable


Is there an actual ZFS pool on freebsd-zfs?


----------



## Alain De Vos (Jul 6, 2021)

There are 3 parameter you can use in /boot/loader.conf
For my legacy boot of zfs it is

```
loaddev="disk2p9:"
### Selects the default device to loader the kernel from
currdev="zfs:ZT/ROOT/default:"
### Specify the root partition to mount
vfs.root.mountfrom="zfs:ZT/ROOT/default"
```


----------



## Diman (Jul 6, 2021)

SirDice said:


> Is there an actual ZFS pool on freebsd-zfs?


Yes, working pool. I can boot this pool, if add one more disk with freebsd(copy of first with same disk layout).


----------



## SirDice (Jul 6, 2021)

Diman said:


> I can boot this pool, if add one more disk with freebsd(copy of first with same disk layout).


This makes zero sense. What does `zpool status` show for that pool?


----------



## Alain De Vos (Jul 6, 2021)

& "zpool list -v" when you use the two pools.


----------



## Diman (Jul 6, 2021)

```
zpool status -v
  pool: sys
 state: ONLINE
config:

    NAME        STATE     READ WRITE CKSUM
    sys          ONLINE       0     0     0
      da1p2     ONLINE       0     0     0

errors: No known data errors

zpool list -v
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
sys         31.5G  8.83G  22.7G        -         -     1%    28%  1.00x    ONLINE  -
  da1p2    31.5G  8.83G  22.7G        -         -     1%  28.0%      -  ONLINE
```


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> There are 3 parameter you can use in /boot/loader.conf
> For my legacy boot of zfs it is
> 
> ```
> ...


Does not help.
But it change one thing - ZFS: i/o error appear before loading kernel.


----------



## Alain De Vos (Jul 6, 2021)

Status and list seems ok.
What's the output of:

```
zfs list -o name,canmount,mountpoint
```
And

```
gpart show -p
```


----------



## Diman (Jul 6, 2021)

Just copied disk to hyper-v 2012r2
Works fine.


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> Status and list seems ok.
> What's the output of:
> 
> ```
> ...




```
zfs list -o name,canmount,mountpoint
NAME                    CANMOUNT  MOUNTPOINT
sys                      on        /
sys/swap                 -         -
sys/tmp                  on        /tmp
sys/usr                  on        /usr
sys/usr/home             on        /usr/home
sys/usr/local            on        /usr/local
sys/usr/ports            on        /usr/ports
sys/usr/src              on        /usr/src
sys/var                  on        /var
sys/var/db               on        /var/db
sys/var/db/mysql         on        /var/db/mysql
sys/var/db/mysql/ib_log  on        /var/db/mysql/ib_log
sys/var/log              on        /var/log
sys/var/tmp              on        /var/tmp

gpart show -p
=>     40  4194224    da0  GPT  (2.0G)
       40     1024  da0p1  freebsd-boot  (512K)
     1064  4193200  da0p2  freebsd-zfs  (2.0G)

=>      40  67108784    da1  GPT  (32G)
        40      1024  da1p1  freebsd-boot  (512K)
      1064  67107760  da1p2  freebsd-zfs  (32G)
```


----------



## Alain De Vos (Jul 6, 2021)

1064  4193200  da0p2  freebsd-zfs  (2.0G)
Looks a bid small to me.


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> 1064  4193200  da0p2  freebsd-zfs  (2.0G)
> Looks a bid small to me.


OS only, this disk needed to boot.


----------



## Alain De Vos (Jul 6, 2021)

Which zpool resides on that partition ?

loader.conf would be something like, but loaddev and currdev can mostly be omitted.

```
loaddev="disk2p2:"
currdev="zfs:sys:"
vfs.root.mountfrom="zfs:sys"
```


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> I would try loader.conf:
> 
> ```
> loaddev="disk2p2:"
> ...


Does not help.


----------



## Alain De Vos (Jul 6, 2021)

What is the filesystem on da0p2 ?


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> What is the filesystem on da0p2 ?


Both disks with zfs.


----------



## Alain De Vos (Jul 6, 2021)

What is the zpool pool name ?


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> What is the zpool pool name ?


boot


----------



## Alain De Vos (Jul 6, 2021)

I don't see "boot",

```
zpool list -v
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
sys         31.5G  8.83G  22.7G        -         -     1%    28%  1.00x    ONLINE  -
  da1p2    31.5G  8.83G  22.7G        -         -     1%  28.0%      -  ONLINE
```


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> I don't see "boot",
> 
> ```
> zpool list -v
> ...


This pool needed for boot only, it does not imported.
I don't think the problem in my pools. Fresh install of FreeBSD 13.0 is also want boot. 12 works fine.


----------



## Alain De Vos (Jul 6, 2021)

Do you see this screen or does booting stops before that stage?


			https://forums.freebsd.org/attachments/boot-png.5481/
		


Was the boot zpool and contents created with FreeBSD-13 ?


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> Do you see this screen or does booting stops before that stage?
> 
> 
> https://forums.freebsd.org/attachments/boot-png.5481/
> ...


Yes, i see this screen, boot stop after trying to load kernel.
boot zpool created with FreeBSD 13.0, i just untar'ed base.txz and kernel.txz to it.


----------



## Alain De Vos (Jul 6, 2021)

I'm out of ideas.
Just these two links,








						Best practices for running FreeBSD on Hyper-V
					

Provides recommendations for running FreeBSD on virtual machines



					docs.microsoft.com
				











						Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
					

Describes features that affect core components such as networking, storage, memory when using Linux and FreeBSD on a virtual machine



					docs.microsoft.com


----------



## Alain De Vos (Jul 6, 2021)

There is one last thing to try,
import the root zpool and copy the file /etc/zfs/zpool.cache to the boot zpool in directory /boot/zfs/


----------



## SirDice (Jul 6, 2021)

Zpool shows ad1p2 as part of the pool. I presume with one disk there's only ad0p2. Adding the second disk makes ad1p2 appear.


----------



## Diman (Jul 6, 2021)

Alain De Vos said:


> There is one last thing to try,
> import the root zpool and copy the file /etc/zfs/zpool.cache to the boot zpool in directory /boot/zfs/


Does not help.


----------



## Diman (Jul 6, 2021)

Seems time to post this on bugs.freebsd.org...


----------



## grahamperrin@ (Jul 9, 2021)

__





						257080 – [hyper-v] FreeBSD 13.0 can't boot on hyper-v 2012 with root on zfs
					






					bugs.freebsd.org
				






Diman said:


> … hyper-v 2012​



Is that, in Windows Server 2012? 

(I'm familiar only with Hyper-V in Windows 10. I installed some FreeBSD guests, probably including 13.0 on ZFS, before switching to VirtualBox.)









						Hyper-V - Wikipedia
					






					en.wikipedia.org


----------



## Alain De Vos (Jul 9, 2021)

According to the bugzilla report root on ufs works. 
But does it work with only boot on ufs and root on zfs seems an interesting question.


----------



## Diman (Jul 13, 2021)

Alain De Vos said:


> But does it work with only boot on ufs and root on zfs seems an interesting question.


Yes, it works.


----------

