# IP Multipathing support for FreeBSD



## AndyUKG (May 11, 2010)

Hi,

  does anything exist that provides redundant network connections in FreeBSD? Ie to avoid single points of failure on the NIC, cabling, and Network switch? For example something like Solaris IPMP? 
If not is anyone developing this? Presumably it would be possible and fairly easy to port IPMP from opensolaris? As an idea 

thanks Andy.


----------



## SirDice (May 11, 2010)

How about CARP and Lagg?

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-aggregation.html
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/carp.html


----------



## AndyUKG (May 11, 2010)

Hi,

  yes these can both be useful, but donÂ´t provide the same functionality as, for example, Solaris IPMP. If we just compare link aggregation for example (speaking just of redundancy with a single host) typically the network hardware requires that you connect all links to the same physical switch, which means you loose out on the redundancy network side (ie what happens if the network switch breaks),

thanks Andy.


----------



## SirDice (May 11, 2010)

AndyUKG said:
			
		

> If we just compare link aggregation for example (speaking just of redundancy with a single host) typically the network hardware requires that you connect all links to the same physical switch,


As far as I know it doesn't.


----------



## AndyUKG (May 11, 2010)

It doesnt? Are you referring to "network hardware requires that you connect all links to the same physical switch"?
Having working with serveral different Cisco network guys in several different companies they always tell me that, with Cisco kit, port aggregation must be to the same switch. I dont know for non-cisco kit but I imagine the same limitation applies to most vendors.


----------



## SirDice (May 11, 2010)

AndyUKG said:
			
		

> It doesnt? Are you referring to "network hardware requires that you connect all links to the same physical switch"?


Yes because you can use lagg to switch between wired and wireless. See example 31-3 in the handbook.



> Having working with serveral different Cisco network guys in several different companies they always tell me that, with Cisco kit, port aggregation must be to the same switch. I dont know for non-cisco kit but I imagine the same limitation applies to most vendors.


You are referring to FEC and/or LACP. In those cases lagg is used to increase available bandwidth. Not as a fail-over.


----------



## AndyUKG (May 11, 2010)

Ah, ok. So the failover functionality of lagg this is completely hardware independent and is basing "up status" on link status provided by each NIC? This should be a satisfactory solution, thanks 

If you arent aware, and in case you are interested. I believe Solaris IPMP provides a superior soltion in a couple of ways.
1) Link status is not only checked via physical link status of the NIC, but also whether some defined IPs (usually the default gateway) are reachable. This can detect some network problems that would go unnoticed with just checking physical link status.
2) Outbound IP traffic is load balanced across all available links (inbound traffic is not load balanced).
3) Physical links may be LACP port aggregated links.

Downside to IPMP is you need an extra IP address for each physical link over and above those that you wish to configure for services etc.

thanks for the information.


----------



## SirDice (May 11, 2010)

AndyUKG said:
			
		

> If you arent aware, and in case you are interested.


I am aware, I admin about 75 Solaris boxes 

I'm not sure what the license is on that bit of Solaris code. I do know Sun donated quite a lot to FreeBSD. Not sure what will happen now Oracle took over though :\


----------



## phoenix (May 11, 2010)

AndyUKG said:
			
		

> yes these can both be useful, but donÂ´t provide the same functionality as, for example, Solaris IPMP. If we just compare link aggregation for example (speaking just of redundancy with a single host) typically the network hardware requires that you connect all links to the same physical switch, which means you loose out on the redundancy network side (ie what happens if the network switch breaks)



Nope.  Link Aggregation Control Protocol (LACP) works across switches.  You just need to have switches that support LACP and Spanning Tree.


----------



## phoenix (May 11, 2010)

AndyUKG said:
			
		

> It doesnt? Are you referring to "network hardware requires that you connect all links to the same physical switch"?
> Having working with serveral different Cisco network guys in several different companies they always tell me that, with Cisco kit, port aggregation must be to the same switch. I dont know for non-cisco kit but I imagine the same limitation applies to most vendors.



Then they either don't know what they are talking about, or they want you to buy more expensive Cisco crap that supports it.

This has been available on low-end HP ProCurve switches for years.


----------



## SirDice (May 11, 2010)

phoenix said:
			
		

> You just need to have switches that support LACP and Spanning Tree.


STP might be the reason why the netadmins told the OP to use the same switch. If not configured properly  a recalculation of the tree can take down the entire network for several minutes. Not what you want if only one switch dies.


----------



## phoenix (May 11, 2010)

AndyUKG said:
			
		

> Ah, ok. So the failover functionality of lagg this is completely hardware independent and is basing "up status" on link status provided by each NIC? This should be a satisfactory solution, thanks
> 
> If you arent aware, and in case you are interested. I believe Solaris IPMP provides a superior soltion in a couple of ways.
> 1) Link status is not only checked via physical link status of the NIC, but also whether some defined IPs (usually the default gateway) are reachable. This can detect some network problems that would go unnoticed with just checking physical link status.
> ...



Sounds like carp(4), but internal to a single system.

On FreeBSD, lagg(4) is used to create redundant links on a single system.  And carp(4) is used to create redundant links between identical systems (like clustering).


----------



## phoenix (May 11, 2010)

AndyUKG said:
			
		

> Ah, ok. So the failover functionality of lagg this is completely hardware independent and is basing "up status" on link status provided by each NIC?



Depends on how you configure the lagg(4) device.  If you use one of the "dumb" settings like fail-over or round-robin, then yes, it just looks at the "up" status of the link.  If you select one of the "smart" settings like LACP, then it actually sends control packets out to the switch to make sure everything is still working and to actively load-balance and fail-over correctly.  The member devices of an LACP connection all work together to make sure things are working.


----------



## AndyUKG (May 11, 2010)

phoenix said:
			
		

> Then they either don't know what they are talking about, or they want you to buy more expensive Cisco crap that supports it.
> 
> This has been available on low-end HP ProCurve switches for years.



Its quite hard to google info on this, but from an HP forum:

http://forums11.itrc.hp.com/service...47626+1273594729727+28353475&threadId=1421720

Which is what Ive been previously been told re LACP and Cisco kit...


----------



## phoenix (May 11, 2010)

Hrm, weird, we've been doing redundant links to multiple switches for over a year now at one school.  Maybe it's just via STP, then?  Whatever it is, it works.  And we're an all-ProCurve site.


----------

