# Packet loss when using FIB and multiple gateways and interfaces



## Jason_25 (Aug 8, 2022)

I have setup my FreeBSD workstation with 2 gigabit NICs for a resilient networking configuration.  I won't bore you with the details but one gateway is 192.168.1.1 connected to re0 and one gateway is 192.168.5.1 connected to ue0.  I am using 12.3-RELEASE-p5.

Interface re0 is setup by rc.conf.  These are the commands used:

```
sudo ifconfig ue0 up
sudo ifconfig ue0 192.168.5.4
sudo setfib 1 route add 192.168.5.0/24 -iface ue0
sudo setfib 1 route add default 192.168.5.1
```

I am experiencing packet loss when pinging 192.168.5.1 through fib 1 like this:

```
setfib 1 ping 192.168.5.1
```
About half the packets are lost.  It will stop and start.  Also strangely I cannot ping 192.168.5.2 for instance with ping saying "sendto: Invalid argument".

These are the routing tables:

```
(fib 0)
Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS         re0
127.0.0.1          lo0                UHS         lo0
192.168.1.0/24     link#1             U           re0
192.168.1.2        link#1             UHS         lo0

Routing tables (fib: 1)
Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.5.1        UGS         ue0
127.0.0.1          lo0                UHS         lo0
192.168.5.0/24     00:05:1b:b0:63:aa  US          ue0
```

It works the same way with packet loss to 192.168.5.1 and no ping at all to 192.168.5.2 whether I use net.add_addr_allfibs=0 or not.  What could I be missing?


----------



## VladiBG (Aug 8, 2022)

> ifconfig ue0 192.168.5.4/24 fib 1


----------



## Jason_25 (Aug 8, 2022)

The problem was with the ue0 driver and the Asix AX88179 gigabit ethernet device.  Like Icarus I was flying so high with a twin high speed ethernet connection only to come crashing down with the realization of USB device bugs.  No enterprise network connection for me.  This is not the first time I have run into USB ethernet failures with FreeBSD.  I am only suprised I did not expect it sooner.

I thought someone was telling FIBs about how this all works but the ue0 driver works ok with my older USB 2.0 fast ethernet device.  I got FIB to work with three gateways actually.  I am not sure about the implementation though.  I understand that you can start servers that listen on different FIBs which is fine.  But this is a workstation and I would prefer that user connections have an automatic fall back as well.  I will just have to remember to start a web browser for instance from the command line with a different FIB when the primary gateway is down.

I will be hanging out in the slow lane if you need me.

edit:
I think I spoke too soon on the ue driver working for the older insignia fast ethernet USB adapter.  Eventually it developed the same problems as before.

The problems are also described here:





						244845 – ue0 realtek interface stops working after some time connection stalls problem down->up helps unitek Y-3083B
					






					bugs.freebsd.org
				








						255251 – The ue0 device has an unstable network connection
					






					bugs.freebsd.org
				








						252165 – usb network and mii bus media status race condition
					






					bugs.freebsd.org
				




I am going to try another fast ethernet device that I have that is a different brand and I think uses a different Asix chipset but still uses the ue driver.  That is the last different kind of USB ethernet device I have.  They all use the ue driver.  If this fails then that will be 4 different types and brands of USB ethernet devices all not working with FreeBSD.  The j5create USB 3.0 gigabit ethernet hub, the j5create USB gigabit standalone adapter, the Insignia fast ethernet adapter, and now I wil test the Trendnet TU2-ET100 which another of has successfully ran on PFSense for 10 years or so.

 I need this to work as I can not physically or electrically build a low power system that will support PCI express cards.  I could not see replacing this SBC with another with dual ethernet built in either because any replacement needs the ability to support 16GB RAM or more.

As I mentioned I am testing the final available USB ethernet device now.

edit 2:

The Trendnet TU2-ET100 seems to be working even though there is an up/down event every 15 minutes or so.  It has certainly survived my gradual ping tests for much longer than the other devices.  I suppose I will order another one of those.  If someone wants to provide a gigabit alternative that would be great.


----------

