# QLogic QLE8142-IBMX unstable on FreeBSD and Windows



## Nyakov (Jul 26, 2020)

Hi.

I have stability issues(to point of unusability) with QLogic QLE8142 on FreeBSD.
Before trying to open issue on bugtracker I need help to verify sanity of my actions.
Perhaps I am missing something?

I have two 10G Qlogic converged network cards.
QLogic QLE8142-IBMX (42C1802)
QLogic QLE8142-SR-IBM (46K8088)
They a basically the same(looks and works same), both flashed with latest firmware (from Qlogic support)

They a transceiver specific and I use them with Qlogic FLTX8571D3BCL-QL optical transceivers (it seams they support only fiber)

I am testing them with iperf3.

I setup direct fiber link between two NICs on different PCs with different OS
Like this (ifconfig inet 192.168.10.1/24) and (192.168.10.2/24) for second PC.

On FreeBSD this card got identified as:

```
ql1: <Qlogic ISP 8000 PCI CNA Adapter-Ethernet Function v2.0.0> port 0xe200-0xe2ff mem 0xfea08000-0xfea0bfff,0xfe700000-0xfe7fffff at device 0.1 on pci1
```

On Linux:

```
qlge 0000:01:00.0: QLogic 10 Gigabit PCI-E Ethernet Driver
qlge 0000:01:00.0: Driver name: qlge, Version: 1.00.00.35.
qlge 0000:01:00.0 eth1: Link is down.
qlge 0000:01:00.0 eth1: Clearing MAC address
qlge 0000:01:00.0 eth1: Function #0, Port 0, NIC Roll 0, NIC Rev = 1, XG Roll = 0, XG R>
qlge 0000:01:00.0 eth1: MAC address 00:c0:dd:26:25:80
```

Now, what works:
OS Linux (Arch and Debian tested) qlge kernel module.
QLE8142 -> fiber -> any transceiver(including Qlogic one) + any 10G card at any os(except QLE8142 card on other then linux OS)
mtu 9000 and mtu 1500
All great I see channel saturation and no errors. Different offload settings have small effect on anything.

What does not work:
QLE8142 card on other then Linux OS with any card in other end (including QLE8142 on Linux)
OS FreeBSD(qlxge driver) 10.2 and 12.1 tested.
And Windows Server 2012 (with official Qlogic drivers as well as with vendor specific(Lenovo))
On both OSes I get similar unstable behavior.
Low link speed, or dropout of link when test start.

It is unstable, so, it is hard to describe whats going on.
One example:

iperf3 connecting from Linux box to FreeBSD12.1 with QLE8142 - all seems fine.

iperf3 connecting from FreeBSD12.1 with QLE8142 to Linux box - low unstable speed (2Gb/s - 5Gb/s) then dropout to zero. Link seems to stay active but no new connection can pass at all(for some time)

This somehow gets affected by -tso setting, but unstable behavior just become different.
On FreeBSD10.2 with -tso setting and mtu 1500 - card can work stable for some time, but sometimes it is not.
With mtu 9000 - unstable, drop of connections.

Also FreeBSD show messages in log such as:

```
ql0: qls_mbx_get_link_status 0x00004000 0x10000051 0x00000000 0x00000037 0x000000f9 0x05050504
ql0: qls_hw_send: tx_free[0] = 2
ql0: qls_mbx_get_link_status 0x00004000 0x10000051 0x00000000 0x00000037 0x000000fd 0x05050504
ql0: qls_mbx_isr: AEN [0x00008012 0x00000060 0x00000037 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000]

ql0: qls_rx_comp: DS bit not set
ql0: qls_rx_comp: (rxb->paddr != cq_e->b_paddr)[0x55270000, 0x5527c800]
ql0: qls_rx_comp: (rxb->paddr != cq_e->b_paddr)[0x55270000, 0x35b3cc000]
ql0: qls_rx_comp: (rxb->paddr != cq_e->b_paddr)[0x55270000, 0x53fe4000]
```

From FreeBSD:
`ifconfig -m ql0`

```
ql0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c013b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4,VLAN_HWTSO,LINKSTATE>
capabilities=c013b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4,VLAN_HWTSO,LINKSTATE>
ether 00:c0:dd:26:25:80
inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
media: Ethernet autoselect (10Gbase-SR <full-duplex>)
status: active
supported media:
media autoselect
media autoselect mediaopt full-duplex
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
```

From Linux:
`sudo ethtool enp1s0f0`

```
netlink error: No such file or directory
Settings for enp1s0f0:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: d
Wake-on: d
Current message level: 0x000060f7 (24823)
drv probe link ifdown ifup rx_err tx_err hw wol
Link detected: yes
```

`sudo ethtool -k enp1s0f1`

```
Features for enp1s0f1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
```


----------



## thomas_gaberial (Sep 15, 2020)

Hi, I am a newbie here, 

I have the same card dual 2 X CNA cards. Tried to install one on windows 10 and one on FreeNAS with Freebsd based OS. Both of them not taking this cards. Could you please help me out how to make it work. Thanks


----------



## Nyakov (Sep 15, 2020)

thomas_gaberial said:


> Hi, I am a newbie here,
> 
> I have the same card dual 2 X CNA cards. Tried to install one on windows 10 and one on FreeNAS with Freebsd based OS. Both of them not taking this cards. Could you please help me out how to make it work. Thanks



Hi, on FreeBSD you need to load kernel module by add in to /boot/loader.conf  "if_qlxge_load="YES"" line.
I believe for FreeNAS this is the same.

On windows, do you actually installed drivers? From here?


----------

