# Jetway NUC won't boot any FreeBSD > 11.1



## sremick (Jun 17, 2020)

Help! Currently my home network is hosed as a result of this, as the box ran pfSense. Thing never normally reboots, but had to be rebooted and now this bit me. Apparently a pfSense update that carried with it an underlying FreeBSD update triggered this (although how it survived that last reboot is a mystery).

Hardware: Jetway JBC313U591W-3160-B Intel Braswell Celeron N3160 Dual Intel LAN Fanless NUC Barebone PC

Will boot FreeBSD  =<11.1 but starting w/ 11.2 it hangs pretty early on at the "Booting..." line, which doesn't give much sense as to what is causing the problem.
I suspect maybe there's a tunable, but it's a bit of a chicken/egg problem until I can find out which. 

Confirmed it's _not _machdep.disable_msic_migration (didn't really align with my symptoms but it was the best a search could turn up so it was worth a shot).

Any ideas? I'm outside of my forte here.


----------



## sremick (Jun 18, 2020)

Urgency reduced: I located a pfSense 2.4.3 installer, and restored a config from March. I'm functional for now, but still need to figure out what FreeBSD broke about my hardware in v11.2 because until then I can't upgrade pfSense and I don't like being behind on security updates for my firewall.


----------



## rootbert (Jun 18, 2020)

first: pfsense questions belong somewhere else, this is a FreeBSD forum. Can you boot a stock FreeBSD 12.1?


----------



## sremick (Jun 18, 2020)

rootbert said:


> first: pfsense questions belong somewhere else, this is a FreeBSD forum. Can you boot a stock FreeBSD 12.1?


Sorry I did not make that clearer. Once I determined it was FreeBSD-related (by testing FreeBSD installation media) and not just a pfSense thing, I did my testing there. I tested all FreeBSD versions from 9.3-12.1 which is what lead to what I said about FreeBSD =<11.1 working but 11.2 onward not working.


----------



## rootbert (Jun 18, 2020)

ok, i see. Did you try i386 also?


----------



## sremick (Jun 18, 2020)

rootbert said:


> ok, i see. Did you try i386 also?


No, I only tried AMD64 as that's what I've been using all along and was working prior to FreeBSD 11.2 on this hardware. Would something have changed in 11.2 that would require changing from AMD64 to i386?


----------



## Emrion (Jun 18, 2020)

I don't understand your update story. For the best I know, pfSense does not update alone. You must do it yourself when you see there is a new version. Then, as part of the update process, the system reboots at the end. So you can see if there is a problem in the first place.

When you say FreeBSD > 11.1 hangs, is it with a USB stick? If yes, maybe your pfSense installation on disk is interfering with the FreeBSD boot process?


----------



## Jose (Jun 18, 2020)

UEFI or CSM(BIOS)? Can you try the other option? I.e., try ACPI if you're using BIOS and vice-versa.


----------



## sremick (Jun 18, 2020)

Emrion said:


> I don't understand your update story. For the best I know, pfSense does not update alone. You must do it yourself when you see there is a new version.


I didn't mean to imply that it updated on its own without my intervention.



> Then, as part of the update process, the system reboots at the end. So you can see if there is a problem in the first place.


I concur and am puzzled too by this. I don't have a good answer to give you. I'm left just to work with studying the symptoms of the current state.



> When you say FreeBSD > 11.1 hangs, is it with a USB stick? If yes, maybe your pfSense installation on disk is interfering with the FreeBSD boot process?


I tested this with the mSATA drive removed. All test installs were done from the same USB drive, with the img file written in the same manner. I tested: 9.3, 10.1, 10.2, 10.3, 10.4, 11.0., 11.1, 11.2, 11.3, 11.4, 12.0, 12.1

On all I can get to the boot options menu ("boot3"?). From this menu I have tried both a default multi-user boot as well as single user. I've also tried setting various hints based upon slightly related posts from Google searches. But it's after choosing an option here and proceeding that anything 11.2 or higher hangs. All I get for further output is the
	
	



```
/boot/kernel/kernel text=nnnn
```
 etc line, then the 
	
	



```
Booting...
```
 line, then the  spinner a line below that but the spinner instantly freezes.

Same hardware, same BIOS settings, mSATA drive removed. Only changed variable is the FreeBSD img (version) used each time. It's 100% reproducible. 

The _only _way I can get any variance (besides the FreeBSD version used) is if I choose to disable ACPI from the boot options menu, but then it fails with the "running without device atpic requires a local APIC" error. But I don't believe this is related as from what I read, most modern hardware requires ACPI enabled and this error can result from disabling it in those instances.


----------



## sremick (Jun 18, 2020)

Jose said:


> ACPI or CSM(BIOS)? Can you try the other option? I.e., try ACPI if you're using BIOS and vice-versa.


I believe the only ACPI option I saw was related to sleep/suspend behavior (C6, C7 etc). There were options related to UEFI vs Legacy which I did try switching around. I can check again tonight...  I just need to be at home and take my network down then move the box to my workbench. 

I also ordered a spare mSATA drive so that I could potentially do "destructive" testing without messing with my working install. But the issues are reproducible even with no mSATA drive installed.


----------



## Emrion (Jun 18, 2020)

It's weird. Your hardware is merely common at first glance. I agree with you about ACPI.

One thing you can try is FreeBSD 13-CURRENT because you may have more information on your screen on what is failing (its kernel is in debugging mode).


----------



## Jose (Jun 18, 2020)

I'm sorry, I meant UEFI, not ACPI. Edited original post. I'm asking because I had a Hell of a time with a Jetway mobo's UEFI.


----------



## sremick (Jun 18, 2020)

Emrion said:


> One thing you can try is FreeBSD 13-CURRENT because you may have more information on your screen on what is failing (its kernel is in debugging mode).


That is a _fantastic _suggestion. I'll do that next as well as revisit BIOS settings (however even if it's a BIOS setting, something changed in FreeBSD at some point to trigger the problem. But maybe a debugging kernel can shed some light on that).


----------



## rootbert (Jun 18, 2020)

you could also try to disable apic ... at the loader prompt, type:  
	
	



```
set hint.apic.0.disabled="1"
```


----------



## sremick (Jun 21, 2020)

Ok this has been solved. Apparently it wasn't really locked up. It was a known console bug, supposedly only affecting Atom-based systems but mine is Celeron-based.

The solution is to use: 
	
	



```
set kern.vty=sc
```

I've since added it to the appropriate place in pfSense to persist... a normal FreeBSD user would add it to loader.conf.local

I am now running just fine, but wanted to update this thread for posterity. Thanks everyone for your help.


----------

