# Multiple network cards/interfaces



## usersend (Dec 16, 2009)

Hi!
Situation:
Have web server with 2 NIC
1 gateway and each interface have 1 IP from same /24 subnet
Each NIC have dedicated 100mbit connection.
BUT it is not used as i thought it would:

```
systat -ifstat
     Load Average

      Interface           Traffic               Peak                Total
            lo0  in     14.632 KB/s        153.466 KB/s           29.601 MB
                 out    14.632 KB/s        153.466 KB/s           29.597 MB

           bge0  in    335.088 KB/s        480.631 KB/s          230.989 MB
                 out     0.000 KB/s          0.008 KB/s            0.574 KB

           bce0  in     25.521 KB/s         39.490 KB/s           15.318 MB
                 out     7.697 MB/s         10.161 MB/s            3.729 GB
```

incoming traffic comes in appropriately to IP address, websites use both bge0 and bce0, works as supposed.
But all outgoing traffic goes only through 1 at boot selected interface in this case bce0, but i want it going out appropriately to request e.g. SITE1 using bge0 so all outgoing traffic for this SITE1 uses bge0, SITE2 using bce0 so all outgoing traffic for this SITE2 uses bce0

(in described situation loosing 100mbit...)
Thanks in advance!


----------



## phoenix (Dec 16, 2009)

Have a read through lagg(4), which covers Link Aggregation (combining multiple NICs into one logical connection).  Works best if you have a switch that support LACP (Link Aggregation Control Protocol).

Alternatively, you'll want to look at setfib() and policy routing in pf() and/or ipfw().


----------



## usersend (Dec 16, 2009)

before creating this thread i read similar post at this forum about lagg

and tried from docs:

```
ifconfig lagg0 create
ifconfig lagg0 up laggproto loadbalance laggport bce0 laggport bge0
```
which completely hanged server networking, i did something wrong or it needs additional configuration before executing these commands?


----------



## SirDice (Dec 16, 2009)

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-aggregation.html


----------



## usersend (Dec 16, 2009)

```
ifconfig lagg0 create
ifconfig lagg0 up laggproto lacp laggport bce0 laggport bge0
```
hanged networking too, server is remote, can't see what error is there

maybe required module isn't loaded or anything else? how can i watch which modules are loaded right now and how to load additional module?


----------



## DutchDaemon (Dec 16, 2009)

kldstat(8), kldload(8).


----------



## phoenix (Dec 16, 2009)

Don't do it remotely on both of the NICs at once.    It resets the NICs to add them to the lagg0 device.  You'll need to do it in small steps:

Connect to the server using the IP on one NIC (say the one on b*c*e0)
Create the lagg0 device:  `ifconfig lagg0 create`
Remove the IP from one NIC:  `ifconfig b[b]g[/b]e0 <IP> delete`
Add that NIC to the lagg0 device, with the old IP:  `ifconfig lagg0 laggproto loadbalance laggpot b[b]g[/b]e0 <IP>`
Connect again to the server, but using the IP on the lagg0 device
If that succeeds, then remove the IP from the other NIC (bce0):  `ifconfig bce0 <IP> delete`
You may need to recreate the default route at this point:  `route add default <IP of router>`
Add the other NIC to the lagg0 device:  `ifconfig lagg0 loadbalance laggport bge0 laggport bce0 <IP>[/list]`

I think you may be able to just do:  `ifconfig lagg0 laggport bce0`
as the last step.

If you are still connected, then you can add the second IP as an alias to the lagg0 device:  `ifconfig lagg0 <IP2>/32 alias`

Finally, you can edit /etc/rc.conf to make it permanent across reboots:

```
cloned_interfaces="lagg0"
ifconfig_bce0="up"
ifconfig_bge0="up"
ifconfig_lagg0="laggproto loadbalance laggport bge0 laggport bce0 <IP1>"
ifconfig_lagg0_alias0="<IP2>/32"
```


----------



## usersend (Dec 17, 2009)

stuck at 5th step

```
5. Connect again to the server, but using the IP on the lagg0 device
```
can't login via that IP via ssh, can't even ping that IP neither remotely neither locally, strangely because if freebsd have ip configured on lagg0 i should at least be able to ping it from inside...


----------



## phoenix (Dec 17, 2009)

Double-check the routing, to make sure that packets destined for that IP are actually making it to the box.  You may need to run tcpdump on both NICs to see where traffic is coming in and where it's going out.

(The above steps were done from memory, I've never actually tried to do it.  It's just a theory.)


----------

