# Onboard NIC in HPE ProLiant ML10 not recognized by FreeBSD



## rtwingfield (Mar 7, 2017)

I have attempted to install a boot'able FreeBSD v10.1 OS on a fresh new HPE ProLiant ML10 GEN9 enterprise server system.  This box is supplied with a UEFI, rather than a BIOS.   My existing OS (harddrive) boots from the BIOS, and I'm not ready (or willing . . .yet) to convert the system to UEFI.   The *AMI Aptio (UEFI) Setup Utility*, v2.17.1254, is configured as follows:

```
Advanced
     Network Stack Configuration (UEFI)
         disabled
         (not) enabled (Preboot Execution Environment  - following are not optioned if disabled)
             Ipv4 PXE support (enable/disable)
             Ipv6 PXE support (enable/disable)
             PXE boot wait time
             Media detect count

     CMS Configuration (Compatibility Support Module Configuration)
         CMS Support (enabled)
         CMS16 module version 07.79 (static)
         Boot option filter
             Legacy only (enabled)
             UEIF only (not enabled)

     WHEA (Windows Hardware Error Architecture Support) disabled
  
     CPU configuration (many options here)
         Hyper-threading disabled
```

OS (FreeBSD v10.1, and previous versions). . .has been working for 15 years+ on BIOS booting system.  OS does boot under CMS legacy mode.  Problem is:  ifconfig(8) (FreeBSD command line utility) does *NOT* recognize and/or report the onboard NIC.  All services that require ethernet interface will not/can not communicate via the NIC -- no DNS, NTP, SMTP/POP3,  SSH, etc.

I have read from other HP web pages that this is because a "driver" is not installed.   Well, drivers are installed in the FreeBSD kernel, so where else should such a driver be installed?  AMI is no help . . .their support tech, "not our problem, contact HP".  HP is virtually impossible to contact.

System is unusable.  What to do?  What would preclude rc.conf - `ifconfig` from recognizing the NIC?  How does the OS interact with the UEFI "_bios_"?

I haven't tried installing a separate PCIe NIC yet (can't find one in stock locally, but would if I could.)

Suggestions?


----------



## Phishfry (Mar 8, 2017)

Separate NIC would be a good test. On EFI bios I see a setting called Network Stack. This routes the ethernet packets through EFI stack. I would try and disable it for legacy use. Especially with your problem.
Is the ethernet interface showing up at all? What ethernet adapter does it use?

I also wonder if your install is using true legacy-only/csm mode. On a UEFI problem-board of mine I had to set the hard drive boot priority to USB PMAP Memstick not UEFI Memstick to get a true legacy install(So one memstick show two different modes in HD Boot Priority list). So check bios HD priority settings. Is your install picking GPT or MBR? That seems to be a distinction. EFI requires GPT scheme. CSM is just a compatibility layer. EFI is still lurking. So the FreeBSD memstick installer picks whatever mode the bios uses. Picking Legacy or PMAP USB HardDisk setting for boot helps the installer run in legacy mode and that in turn then preforms a legacy install. The EFI install uses a different bootloader.


----------



## rtwingfield (Mar 8, 2017)

Phishfry said:


> Separate NIC would be a good test.


. . .just ordered a  Rosewill RC-411v3 - Network Adapter 10/100/1000 Mbps PCI-Express card (none locally available).



Phishfry said:


> On EFI bios I see a setting called Network Stack. This routes the ethernet packets through EFI stack. I would try and disable it for legacy use. Especially with your problem.


Yes, I have already done that.  In my "shopping list" of the UEFI "_bios_", that is what I was trying to indicate in my previous post:

```
Network Stack Configuration (UEFI)
        disabled
```



Phishfry said:


> Is the ethernet interface showing up at all? What ethernet adapter does it use?


No, `ifconfig` only sees and reports the loopback device.  Apparently, HP product documentation does not identify the on-board NIC.  On another "_web-help_" site post, I have read, "_It's an embedded NC105i PCI Express Gigabit Ethernet Server Adapter as HP calls it. Around the internet people identify it as a Broadcom BCM5722 card_.", but I don't know for sure.  It is not an actual physical PCIe card, but rather an on-board embedded chip system.



Phishfry said:


> I also wonder if your install is using true legacy-only/csm mode. On a UEFI problem-board of mine I had to set the hard drive boot priority to USB PMAP Memstick not UEFI Memstick to get a true legacy install(So one memstick show two different modes in HD Boot Priority list). So check bios HD priority settings. Is your install picking GPT or MBR? That seems to be a distinction. EFI requires GPT scheme. CSM is just a compatibility layer. EFI is still lurking. So the FreeBSD memstick installer picks whatever mode the bios uses. Picking Legacy or PMAP USB HardDisk setting for boot helps the installer run in legacy mode and that in turn then preforms a legacy install. The EFI install uses a different bootloader.


Yes, again, in my previous post:

```
CMS Configuration (Compatibility Support Module Configuration)
         CMS Support (enabled)
         CMS16 module version 07.79 (static)
         Boot option filter
             Legacy only (enabled)
             UEIF only (not enabled)
```

. . .I tried to indicate this, too.  Actually the OS HDD was GPT partitioned but something like

```
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 da0
bootcode written to da0
```
 was used as I recall.  Not "EFI" _booted_.  (I'm avoiding use of the term BIOS to describe UEFI.)
and finally note that the OS does boot and seems fully operational other than any service dependent on the NIC.

So, given the curt list of options assoc/w the *Aptio Setup Utility v2.17.1254, copyright 2016 American Megatrends, Inc.* (and no help from AMI), I guess my first option is to try the PCIe NIC that I've ordered, and hope for success.


----------



## SirDice (Mar 8, 2017)

rtwingfield said:


> How does the OS interact with the UEFI "_bios_"?


It doesn't. UEFI (or the 'old' BIOS) is there so the machine boots. But that's about it as far as any 'interaction' goes. Good old MS-DOS on the other hand used BIOS INT calls for almost everything but most (if not all) modern operating systems don't use it at all.


----------



## rtwingfield (Mar 8, 2017)

Yeah, what I suspected. So if `ifconfig` can't _see_ the NIC, then a driver for the NIC is missing from the FreeBSD OS?


----------



## SirDice (Mar 8, 2017)

rtwingfield said:


> So if  ifconfig can't _see_ the NIC, then a driver for the NIC is missing from the FreeBSD OS?


Either that or the existing bge(4) driver doesn't recognize this particular variant of the chipset. The bge(4) driver does support the 'Dell PowerEdge R300 integrated BCM5722 NIC (10/100/1000baseTX)' which appears to be the same chipset. But perhaps HP uses slightly different identification IDs.


----------



## rtwingfield (Mar 15, 2017)

Received and installed a  Rosewill RC-411v3 - Network Adapter 10/100/1000 Mbps PCI-Express card   and disabled the on-board NIC (in the legacy _BIOS_).

I configured 
/etc/rc.conf to include

```
gateway_enable="YES"    # Set to YES if this host will be a gateway.
defaultrouter="162.202.233.80"

network_interfaces="re0"
ifconfig_re0="inet 162.202.233.81 netmask 255.255.255.248"
```

Module pci/em is found by `kldstat -v`; however, re0 is not in the list, yet `ifconfig` finds it.

`ifconfig`reveals that the device is active; however, it can not communicate with the WAN.


```
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,
WOL_MAGIC,LINKSTATE>
        ether 68:1c:a2:12:75:7d
        inet 162.202.233.81 netmask 0xfffffff8 broadcast 162.202.233.87
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
```

`netstat -rn` seems to report that things are in order, so what is wrong?


```
Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
default            162.202.233.80     UGS         re0
127.0.0.1          link#2             UH          lo0
162.202.233.0/24   162.202.233.81     UGS         re0
162.202.233.80/29  link#1             U           re0
162.202.233.81     link#1             UHS         lo0

Internet6:
Destination                       Gateway                       Flags      Netif
 Expire
::/96                             ::1                           UGRS        lo0
::1                               link#2                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff01::%lo0/32                     ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%lo0/32                     ::1                           U           lo0
```


----------



## Phishfry (Mar 16, 2017)

Several things in your rc.conf are not valid or needed.
https://www.freebsd.org/doc/handbook/config-network-setup.html
https://www.freebsd.org/doc/en/articles/linux-users/network.html

You can basically drop all but your last line:

```
ifconfig_re0="inet 162.202.233.81 netmask 255.255.255.248"
```
Gateway is for a network serving box. Get it working first then worry about these extraneous services.


```
network_interfaces="re0"
```
 This is not needed.

Make sure you have a hostname in there too.


----------



## rtwingfield (Mar 16, 2017)

Originally had 
	
	



```
network_interfaces="re0"
```
 . . .commented out.  (always thought it was unnecessary -- *gone* again!)

re:  





> netmask should be 255.255.255.0 for a typical network


Actually mine is a /29 network of 8 static IP's (*.80-*.87); therefore, has to be 255.255.255.248

I've always included

```
gateway_enable="YES"    # Set to YES if this host will be a gateway.
defaultrouter="162.202.233.80"
```
  . . .have always run host(s) as a gateway, always worked in the past . . .since 2001.

Also, "_Make sure you have a hostname in there too._"  (sorry) omitted from my original text post:  actually 
	
	



```
hostname="alpha.archaxis.net"
```
 . . .already included.

So you see, I just can't understand why this is not working.  All I've done is move the OS v10.1-RELEASE hdd from the old hardware box to the new *HPE ProLiant ML10* platform. Could not get the on-board HP NIC configured (missing kernel module problem?), but the add-on _Rosewill_ PCIe NIC "card" is recognized (by the kernel as *re0*; seems to actually be a Realtek chip); unfortunately, just not working 100%; can only ping LAN IP address and loopback (127.0.0.1)  Just doesn't make sense 

. . .and finally, THANKS! for your suggestions!


----------



## SirDice (Mar 16, 2017)

The gateway_enable is only needed if you route traffic from one interface to another. It's not required or needed for a single gateway.


----------



## rtwingfield (Mar 16, 2017)

SirDice said:


> The gateway_enable is only needed if you route traffic from one interface to another. It's not required or needed for a single gateway.


OK, I suppose this is/was my lack of understanding of routing (a black art) dating back to circa late 1990's-2001.  I've always included it, perhaps needlessly, but the system interface has always worked.  This morning, as an experiment (for chuckles and grins), I've commented it out of /etc/rc.conf.  Unfortunately, the results are the same  . . .can only ping _primary_ (NIC) LAN IP address and loopback (127.0.0.1) -- no WAN or additional LAN addresses; cannot ping gateway or broadcast addresses.

All server platforms including three FreeBSD, an IBM AS/400, a D-Link DP-301P+ printer server, and various Windoze Windows workstations, are all attached inside the LAN routed by a Cisco RV016 router.  The AT&T U-verse modem is a _bridged_ Motorola NVG598 dual line (2 twisted pair) modem.  All server platforms can communicate with the WAN; only this newly configured replacement _alpha_ platform (FreeBSD v10.1) cannot.  Very frustrating.  

Configuration of the Cisco router hasn't changed in over a year. Could there be _something_ in the router configuration that needs to be adjusted?


----------



## SirDice (Mar 16, 2017)

Your addresses don't add up. Assuming 162.202.233.81/29 is correct, 162.202.233.80 would be the network address (you can't use this) and 162.202.233.87 the broadcast address (you can't use this address either). 

Typically either the first or the last address of a subnet (not counting the network and broadcast addresses) is used as a gateway address. So your gateway is either 162.202.233.81 or 162.202.233.86. Your hosts can have addresses 162.202.233.82 up to 162.202.233.85 (or 86 if it's not used for the gateway).

Hosting providers usually have 2 routers using VRRP to provide fail-over redundancy. So the .81 is the VRRP address, .82 and .83 will be the addresses of the physical interfaces of the routers. That would mean the first address that's actually usable will be .84.


----------



## rtwingfield (Mar 16, 2017)

Address as assigned by AT&T are:

.80  network
.81  user available
.82  "
.83  "
.84  "
.85  " (apparently a 5th?)
.86  gateway
.87  broadcast

This /29 network block was assigned by AT&T U-verse.
I have confirmed that .86 is set as the default gateway in the Cisco router.  I suppose that any VRRP is handled by the ISP (if at all).  Although the Cisco RV016 is dual WAN, we only use one.

The four (or five) user addresses were _"offered and sold _(four were promised?)_"_ by AT&T and have used them for years, primarily .81. 

In /etc/rc.conf, I have changed the defaultrouter _pointer_ to .86 but still no joy . . .still can't reach the WAN and other LAN addresses on another local subnet.


----------



## Phishfry (Mar 16, 2017)

I hate to bring this up but Realtek does not have the greatest reviews around here. The newer RT8111/8162 chipset interfaces like 8111G and 8111H can be problematic. 

In this post people are so desperate they are trying to port a factory driver.
https://forums.freebsd.org/threads/55861/

There is a chance this is driver related issue.
Nothing else laying around?

I am not best to comment about direct connect internet. I still use my pfSense crutch.
I erased my netmask comment after researching it. I did an arin search and it shows /29 for your IP.


----------



## SirDice (Mar 16, 2017)

rtwingfield said:


> I have confirmed that .86 is set as the default gateway in the Cisco router.


Wait. There's a router between the FreeBSD host and the WAN? 


> Although the Cisco RV016 is dual WAN, we only use one.


Looked up the model, it has one "DMZ" port and a bunch of ethernet "LAN" ports. I'm trying to build a picture in my head how things are connected and how it's supposed to work. 


> In /etc/rc.conf, I have changed the defaultrouter _pointer_ to .86 but still no joy . . .still can't reach the WAN and other LAN addresses on another local subnet.


Judging by the external address used on the FreeBSD host, is it connected to the DMZ port? I'm assuming all the other equipment is connected to the LAN ports? I'm also guessing the Cisco only has a web interface to configure it. 

It's a bit of a long shot in the dark but can you try setting this:

```
defaultrouter="-iface re0"
```

A whole lot depends on how exactly the Cisco is configured. And how things are connected.


----------



## rtwingfield (Mar 16, 2017)

SirDice said:


> Judging by the external address used on the FreeBSD host, is it connected to the DMZ port? I'm assuming all the other equipment is connected to the LAN ports? I'm also guessing the Cisco only has a web interface to configure it.


Everything is connected to LAN ports.   DMZ is not used.  Yes, the Cisco router is configured via a web GUI.



SirDice said:


> It's a bit of a long shot in the dark but can you try setting this:
> defaultrouter="-iface re0"


. . .did try this but again, no joy.

I have _concerns_ regarding the "found or identified module", re0, which does not show up in the list revealed by `kldstat -v`, yet it seems to _automagically_ appear in the `ifconfig` display, regardless of whether I'm using re0, em, or _xyx_.  This is a GENERIC (default years old version) amd64 kernel; I don't know where it's getting the re0 information . . .where it's coming from?

I'm in the process of loading a vendor provided Linux driver for the Rosewill PCIe NIC  (transferring from CD via USB stick).  This will take a while (can't use the ssh interface because of this network problem).  Regardless, how would this legacy kernel find the re0 module if it apparently is not loaded in/by the kernel?


----------



## Phishfry (Mar 16, 2017)

Hey I found that the chip is actually RTL8111C on your Rosewill card. So that chipset version is from 2008 and should work OK.
Later versions are problematic.


----------



## rtwingfield (Mar 17, 2017)

rtwingfield said:


> I'm in the process of loading a vendor provided Linux driver for the Rosewill PCIe NIC (transferring from CD via USB stick). This will take a while (can't use the ssh interface because of this network problem).


Well, as suspected, this will be a problem.  There is a compile process involving a Makefile and  autorun.sh (bash shell script) that are Linux shell dependent.  What a POS!  Seems anymore that if it's not _Windoze_, then Linux is the other option.  What has made Linux so @#$! popular?      I don't want to rewrite this script, editing-out the Linux crap . . .even then, would it work?  

In the Makefile, this noted:  





> r8168 is the Linux device release for Realtek Ggabit Ethernet controllers with PCI-Express interfaces.  Copyright 2013 Realtek . . .


and as I suspected, it is a Realtek device rebranded as a Rosewill product.


----------



## rtwingfield (Mar 17, 2017)

Phishfry said:


> Hey I found that the chip is actually RTL8111C on your Rosewill card. So that chipset version is from 2008 and should work OK.
> Later versions are problematic.


Segue'ing ahead re:


rtwingfield said:


> I'm in the process of loading a vendor provided Linux driver for the Rosewill PCIe NIC


 Any thoughts regarding how the system is coming up with re0 when `ifconfig` is executed? (since it's not included in the `kldstat -v` list)

In other words, where did/does this re0 driver module come from?


----------



## SirDice (Mar 17, 2017)

rtwingfield said:


> Everything is connected to LAN ports.


That just doesn't make sense. It's really bad form to put internet accessible stuff on the LAN. That's what the DMZ is for. Your LAN will be configured differently, most likely using NAT and connected using private IP ranges (RFC-1918). 

Forget installing the Linux driver, even if you get it to compile. It's not going to work, ever. The way drivers work is completely different between FreeBSD and Linux. And there's no point, the interface is correctly recognized and there's no indication it doesn't work. We just haven't figured out how to configure it correctly for _your_ situation.


----------



## rtwingfield (Mar 17, 2017)

SirDice said:


> Forget installing the Linux driver, even if you get it to compile. It's not going to work, ever. The way drivers work is completely different between FreeBSD and Linux. And there's no point, the interface is correctly recognized and there's no indication it doesn't work. We just haven't figured out how to configure it correctly for _your_ situation.


Noted:  your advice regarding Linux, DMZ configuration, et al. (more perhaps for a future discussion).




rtwingfield said:


> Any thoughts regarding how the system is coming up with re0 when  `ifconfig` is executed? (since it's not included in the  `kldstat -v list`).
> In other words, where did/does this re0 driver module come from?


Regardless, I still suspect that something is broken with this _Realtek_ driver.  I'm breaking down the new HP box and swapping the Rosewill PCIe NIC for a new Intel NIC.

EDIT/UPDATE:  Done, but unfortunately, same results 

. . .I just can't believe that this is a problem with the Cisco router configuration (regardless of DMZ or not to DMZ); it has been working for two years+ in the current configuration.  The only changes were made in the upgrade to the _HPE ProLiant ML10 Gen9_ platform and (now) the _Intel PCIe_ NIC; . . .so frustrating.

I think I need to go trout fishing.


----------



## SirDice (Mar 17, 2017)

Just to rule it out, have you tried connecting the FreeBSD box to the DMZ port? It may have been accidentally connected to the wrong port when you swapped stuff out.


----------



## rtwingfield (Mar 17, 2017)

No, actually the DMZ port has never been enabled in the Cisco configuration.  (the Cisco port end of the CAT5 has never been unplugged from a LAN port on the switch.)

BTW, I've discovered that I can ping the gateway port (.86) in addition to user (.81), and also another FreeBSD http server on a LAN subnet 192.168.1.74 from a Windoze workstation, but not from the FreeBSD .81 interface.  It's almost as if there's something "_firewall like_" inside the HPE ProLiant box that is choking the system.

Regarding the Cisco router, I have unplugged from, and connected the FreeBSD's NIC directly to the Motorola NVG538 U-verse modem's 4-port switch;  still no joy.


----------



## rtwingfield (Mar 23, 2017)

It's time to put this dog down; I don't think it's the fault of the onboard NIC in HPE ProLiant ML10 box.  I've suffered with this problem for over three weeks.  I've resorted to breaking down the ProLiant box and restoring the original FreeBSD v10.1 OS HDD to the original _legacy_ platform (hardware restored to working condition) and yet the problem persists.  The system still cannot "_see_" the services.  BTW, after some struggling with ISO downloading and CD burning, I've been able to install v11.0 on a fresh HDD, but now removed from the ProLiant platform . . .and stored on the shelf (for future use).


----------

