# FreeBSD stops while booting (on ARM)



## balanga (Dec 2, 2017)

Not sure if this qualifies as an 'Installing FreeBSD' problem since it is specific to running it on ARM arch... but someone might be able to provide a pointer...


```
FreeBSD/arm U-Boot loader, Revision 1.2
(Thu Nov 30 14:06:37 GMT 2017 root@Test)

DRAM: 128MB
Number of U-Boot devices: 2
U-Boot env: loaderdev not set, will probe all devices.
Found U-Boot device: disk
  Probing all disk devices...
  Checking unit=0 slice=<auto> partition=<auto>... good.
Booting from disk0s2:
/boot/kernel/kernel data=0x443064+0x30f9c syms=[0x4+0x932d0+0x4+0x667c5]

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel] in 9 seconds... Booting [/boot/kernel/kernel] in 8 seconds... Booting [/boot/kernel/kernel] in 7 seconds... Booting [/boot/kernel/kernel] in 6 seconds... Booting [/boot/kernel/kernel] in 5 seconds... Booting [/boot/kernel/kernel] in 4 seconds... Booting [/boot/kernel/kernel] in 3 seconds... Booting [/boot/kernel/kernel] in 2 seconds... Booting [/boot/kernel/kernel] in 1 second... Booting [/boot/kernel/kernel]...               
Using DTB compiled into kernel.
Kernel entry at 0x1200100...
Kernel args: (null)
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
   The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-RELEASE #0: Thu Nov 30 14:22:01 GMT 2017
   root@Test:/usr/obj/arm.arm/usr/src/sys/DB-88F6XXX arm
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
module mvs already present!
CPU: Feroceon 88FR131 rev 1 (**unknown 4** core)
  Little-endian DC enabled IC disabled WA disabled DC streaming enabled
  BTB disabled L2 enabled L2 prefetch enabled
  WB enabled LABT branch prediction disabled
  16KB/32B 4-way instruction cache
  16KB/32B 4-way write-back-locking-C data cache
real memory  = 134213632 (127 MB)
avail memory = 123895808 (118 MB)
SOC: Marvell 88F6281 rev A1, TClock 200MHz
  Instruction cache prefetch disabled, data cache prefetch disabled
  256KB 4-way set-associative write-through unified L2 cache
random: entropy device external interface
ofwbus0: <Open Firmware Device Tree>
simplebus0: <Flattened device tree simple bus> on ofwbus0
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0
localbus0: <Marvell device bus> on ofwbus0
nand0: <Marvell NAND controller> mem 0xf9300000-0xf93fffff on localbus0
nandbus0: <NAND bus> on nand0
ic0: <Marvell Integrated Interrupt Controller> mem 0x20200-0x2023b on simplebus0
timer0: <Marvell CPU Timer> mem 0x20300-0x2032f irq 1 on simplebus0
Event timer "CPUTimer0" frequency 200000000 Hz quality 1000
Timecounter "CPUTimer1" frequency 200000000 Hz quality 1000
gpio0: <Marvell Integrated GPIO Controller> mem 0x10100-0x1011f irq 35,36,37,38,39,40,41 on simplebus0
rtc0: <Marvell Integrated RTC> mem 0x10300-0x10307 on simplebus0
twsi0: <Marvell Integrated I2C Bus Controller> mem 0x11000-0x1101f irq 43 on simplebus0
iicbus0: <Philips I2C bus> on twsi0
iic0: <I2C generic I/O> on iicbus0
mge0: <Marvell Gigabit Ethernet controller> mem 0x72000-0x73fff irq 12,13,14,11,46 on simplebus0
mge0: PHY8 attached, phy_sc points to mge0
mge0: Ethernet address: 52:3b:20:9c:11:51
mge0: MII failed to find PHY
device_attach: mge0 attach returned 6
uart0: <16550 or compatible> mem 0x12000-0x1201f irq 33 on simplebus0
uart0: console (1056,n,8,1)
uart1: <16550ops or compatible> mem 0x12100-0x1211f irq 34 on simplebus0
cesa0: <Marvell Cryptographic Engine and Security Accelerator> mem 0x30000-0x30fff,0x3d000-0x3dfff irq 22 on simplebus0
ehci0: <Marvell Integrated USB 2.0 controller> mem 0x50000-0x50fff irq 48,19 on simplebus0
usbus0: EHCI version 1.0
usbus0 on ehci0
mvs0: <Marvell 88F6281 SATA controller> mem 0x80000-0x85fff irq 21 on simplebus0
mvs0: Gen-IIe, 2 3Gbps ports, Port Multiplier supported with FBS
mvsch0: <Marvell SATA channel> at channel 0 on mvs0
mvsch1: <Marvell SATA channel> at channel 1 on mvs0
pcib0: <Marvell Integrated PCI/PCI-E Controller> mem 0xf1040000-0xf1041fff irq 44 on ofwbus0
pci0: <PCI bus> on pcib0
cryptosoft0: <software crypto>
Timecounters tick every 10.000 msec
usbus0: 480Mbps High Speed USB v2.0
```

Any ideas as to why it stops at this point?


----------



## Phishfry (Dec 3, 2017)

It looks like you got something booting. Maybe not completely.
Usually around this point you would see the 'mount root' stuff come up...
So either it didn't get that far or it's not seeing the usb drive right.

Is that what your booting from here right, USB with serial output?
Or is this tftp booting?

You did do that KERNCONF edit that was needed for 'IPSEC_NAT_T' right? From the mailing list.


----------



## Phishfry (Dec 3, 2017)

So I went through that mailing list post and noticed his dmesg:

```
usbus0: 480Mbps High Speed USB v2.0
Trying to mount root from ufs:/dev/da0s1a []...
Root mount waiting for: usbus0
ugen0.1: <Marvell EHCI root HUB> at usbus0
uhub0: <Marvell EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
uhub0: 1 port with 1 removable, self powered
mountroot: waiting for device /dev/da0s1a...
Mounting from ufs:/dev/da0s1a failed with error 19.
```
Seems like yours is indeed failing at the root mount as I thought.
Maybe this as a ubldr problem.


----------



## balanga (Dec 3, 2017)

Phishfry said:


> So I went through that mailing list post and noticed his dmesg:
> 
> ```
> usbus0: 480Mbps High Speed USB v2.0
> ...



That's his dmesg - I don't get as far as the mount root prompt.... I wondered if the root mount problem was due to fstab not being set correctly...


----------



## Phishfry (Dec 3, 2017)

fstab resides on root so your not getting that far.
Its probably either u-boot or ubldr.

Now I wonder about the u-boot API. That deals with Arm storage as well so it may be at that point.
Maybe you could ask bodhi on u-boot about compiling in the API.

Read here for some more details:
https://kernelnomicon.org/?p=351

"ubldr uses U-Boot API to enumerate devices that might be used as a boot source: block (e.g. SD card) or network."


----------



## balanga (Dec 3, 2017)

Phishfry said:


> It looks like you got something booting. Maybe not completely.
> Usually around this point you would see the 'mount root' stuff come up...
> So either it didn't get that far or it's not seeing the usb drive right.
> 
> ...



It is booting via the serial port using `kwboot` and then from USB.

I didn't edit anything...



> Taking the DOCKSTAR kernel, ripping out the IPSEC_NAT_T to
> make it compile at all (why this module does not compile on
> TARGET_ARCH=arm despite being enabled in this stock kernel I
> do not know)



For me, it built without any problem so didn't see any need.

Also, I'm loading FreeBSD completely differently using:-

```
usb start
fatload usb 0:1 0x02000000 ubldr
bootelf 0x02000000
```

...from the Autoboot prompt.

Maybe I should use ubldr.bin instead ofubldr. Both files are created by the build.


----------



## balanga (Dec 3, 2017)

Phishfry said:


> fstab resides on root so your not getting that far.
> Its probably either u-boot or ubldr.





> Booting from disk0s2:
> /boot/kernel/kernel data=0x443064+0x30f9c syms=[0x4+0x932d0+0x4+0x667c5]
> 
> Hit [Enter] to boot immediately, or any other key for command prompt.
> ...



Is there anything I can put in loader.conf to show additional boot msgs to verify that it is using the correct kernel?


----------



## aragats (Dec 3, 2017)

ubldr's purpose is to boot the kernel, and you see it boots.
I guess, the problem is in the kernel configuration, it should know where is the root filesystem.
Quoting the same source Phishfry mentioned above https://kernelnomicon.org/?p=306:
_FreeBSD kernel should be properly configured in order to be suitable for mounting root_.
I'm not sure whether it's possible to pass the root device as a parameter at boot time (as in Linux) or it has to be pre-configured.


----------



## balanga (Dec 3, 2017)

aragats said:


> ubldr's purpose is to boot the kernel, and you see it boots.



One of the problems for me is that my build creates ubldr ubldr.bin and ubldr.pie and I don't really know what the purpose of these three files is and which U-Boot should load. At the moment I have
`fatload usb 0:1 0x02000000 ubldr`
Maybe I'll change that to
`fatload usb 0:1 0x02000000 ubldr.bin`

and see what happens... Not very scientific.... Feels like clutching at straws.


----------



## balanga (Dec 3, 2017)

aragats said:


> ubldr's purpose is to boot the kernel, and you see it boots.
> I guess, the problem is in the kernel configuration, it should know where is the root filesystem.
> Quoting the same source Phishfry mentioned above https://kernelnomicon.org/?p=306:



That article is related to Netbooting whereas I'm using a USB stick so I don't know if it applies.


----------



## balanga (Dec 4, 2017)

Is there any option I can use in /boot/loader.rc which would allow single stepping during the boot process to see exactly how far I get?

I did change /boot/defaults/loader.conf to include

```
autoboot_delay=20
```
just to see if I was able to alter some aspect of the loader and that worked but setting on verbose loading had no effect.


----------



## SirDice (Dec 6, 2017)

Never edit /boot/defaults/loader.conf! Put your changes in /boot/loader.conf.


----------



## aragats (Dec 6, 2017)

balanga said:


> That article is related to Netbooting whereas I'm using a USB stick so I don't know if it applies.


The kernel must now the rootfs location anyway, have you tried _vfs.root.mountfrom_ option in /boot/loader.conf? From loader.conf(5)

```
....
     vfs.root.mountfrom
                   Specify the root partition to mount.  For example:

                         vfs.root.mountfrom="ufs:/dev/da0s1a"

                   loader(8) automatically calculates the value of this
                   tunable from /etc/fstab from the partition the kernel was
                   loaded from.  The calculated value [b]might be calculated
                   incorrectly[/b] when /etc/fstab is not available during
                   loader(8) startup (as [b]during diskless booting[/b] from NFS), or
                   if a different device is desired by the user.  The
                   preferred value can be set in /loader.conf.
....
```


----------



## balanga (Dec 6, 2017)

aragats said:


> The kernel must now the rootfs location anyway, have you tried _vfs.root.mountfrom_ option in /boot/loader.conf? From loader.conf(5)



Since I see this while booting it's clear to me that the kernel must have found rootfs which exists on /dev/da0s2. 


```
Consoles: U-Boot console  
Compatible U-Boot API signature found @0x7b12860

FreeBSD/arm U-Boot loader, Revision 1.2
(Thu Nov 30 14:06:37 GMT 2017 root@Test)

DRAM: 128MB
Number of U-Boot devices: 2
U-Boot env: loaderdev not set, will probe all devices.
Found U-Boot device: disk
  Probing all disk devices...
  Checking unit=0 slice=<auto> partition=<auto>... good.
Booting from disk0s2:
/boot/kernel/kernel data=0x443064+0x30f9c syms=[0x4+0x932d0+0x4+0x667c5]

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Using DTB compiled into kernel.
Kernel entry at 0x1200100...
Kernel args: (null)
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
   The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-RELEASE #0: Thu Nov 30 14:22:01 GMT 2017
    root@Test:/usr/obj/arm.arm/usr/src/sys/DB-88F6XXX arm
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
module mvs already present!
CPU: Feroceon 88FR131 rev 1 (**unknown 4** core)
  Little-endian DC enabled IC disabled WA disabled DC streaming enabled
  BTB disabled L2 enabled L2 prefetch enabled
  WB enabled LABT branch prediction disabled
  16KB/32B 4-way instruction cache
  16KB/32B 4-way write-back-locking-C data cache
real memory  = 134213632 (127 MB)
avail memory = 123895808 (118 MB)
SOC: Marvell 88F6281 rev A1, TClock 200MHz
```

I'm not sure that the option you suggest would get me past my hang but in the absence of any other suggestion I'll give it a try.


----------

