[Cialug] link aggregation help requested
Dave Weis
djweis at sjdjweis.com
Mon Mar 25 16:44:21 UTC 2019
I'm not sure on what's causing the LACP to not come up but I can answer on
the speed question.
Say you have two links in a group between two computers, The hashing
function looks at a specific packet and decides which of the two links that
given packet will traverse. At the lowest level the driver only knows for
certain the MAC address of the receiver. The hashing function then does
some math to determine that the packet to 00:11:22:33:44:55 goes to link 1.
The same thing happens on the computer on the other end of the LAG.
This makes a transfer between two computers always stick to a single member
of the LAG. It would be different if there was a switch at each end and
multiple computers hooked to each switch all communicating to different
computers. The MAC addresses would hash differently to pick either link 1
or link 2 so traffic would balance a bit better.
Newer/more expensive gear can look further inside the packet and find the
IP addresses and TCP port numbers to help encourage the hash function to
spread the traffic more evenly on point to point style circuits. This is
more important if you had two devices with a LAG between them such as two
computers or two routers. The MAC address will never change on each end so
traffic between them will always hash to the same individual member with
the basic hash. You need to incorporate IP's or something else more dynamic
into the mix.
dave
On Sun, Mar 24, 2019 at 11:44 PM E. Hakan Duran <ehakanduran at gmail.com>
wrote:
> Thank you Dave for your reply. I actually changed the LACP rate from the
> default slow to fast today manually while troubleshooting this. I can
> easily go back to default settings since I saved the config file; however I
> know it will not be a solution to my problem. The other parameter I
> manually modified from the automatically chosen Network Manager settings
> was the transmit hash policy, which was changed to 2+3 from 0 if I recall
> correctly.
>
> Perhaps I get this wrong. I thought that by making a bond, one would
> effectively increase the bandwidth of the connection, which would translate
> to higher data transfer such as faster file transfers, etc. Is this
> perception accurate? Does layer 3 hashing enable faster data transfer
> then? How can I figure whether my devices offer layer 3 hashing or not? The
> switch has a setting to modify load balance algorithm which is set at MAC
> address at the moment, but can be changed to IP/MAC address if desired.
> Does this have the potential to achieve the faster data transfer goal?
>
> I am in the process of upgrading the firmware and will report if it causes
> any noticable change in behavior.
>
> Thanks very much in advance and apologies for some probably basic/silly
> questions. I don't have a technical background and so computing more as a
> hobbyist.
>
> Hakan
>
> On Sunday, March 24, 2019 9:03:42 PM CDT Dave Weis wrote:
> > Hello
> >
> > I looked at https://cstan.io/?p=8876&lang=en and there's a setting in
> the
> > BONDING_OPTS that might be affecting it. His/her output shows an LACP
> rate
> > of slow and yours shows an LACP rate of fast.
> >
> > As far as speed, a single stream connection between two hosts will always
> > stay hashed to a single member of the LAG unless your devices offer
> layer 3
> > hashing such as port numbers. The usual method is mac address hashing and
> > that's going to be the same for every connection between the same two
> hosts.
> >
> > Also make sure you have the newest firmware on your switch.
> >
> > dave
> >
> >
> >
> > On Sun, Mar 24, 2019 at 8:37 PM Hakan E. Duran <ehakanduran at gmail.com>
> > wrote:
> >
> > > Dear all,
> > >
> > > This may be a stupid oversight as far as I know, but I could not solve
> > > it myself without some help for several months now and decided to ask
> > > here.
> > >
> > > I have a workstation that I am trying to set up link aggregation using
> > > the two available on-board ethernet interfaces (motherboard is Asus
> > > Z10PE-D8WS if matters). I was able to set up the bond using the
> > > nm-connection-editor in linux (manjaro flavor) and also set up the LAG
> > > on the Cisco managed switch I have (SG300-28, please see attached). The
> > > workstation is the LAG2 whereas LAG1 and 2 are 2 NAS units. As you may
> > > see on the attached image, LAG2 seems to have a stand by member on port
> > > 24, instead of having 2 active members like the NASes. I cannot figure
> > > out why one of the ethernet connections is assigned a stanby member
> > > status. I pasted the output of the command "sudo cat /proc/net/bonding
> > > /bond0" below, I am unable to recognize a major problem there. The
> Cisco
> > > switch documentation states that this would happen when there is a
> > > hardware limitation in the connected device for building the aggregare
> > > (also see attached), but I am not sure if there would be any such
> > > limitation since this is a server main board, and also I don't want to
> > > reach that conclusion before ruling out the possibility that I am
> > > missing something. I can confirm that the current bond benchmarks are
> > > not superior to a single ethernet connection in terms of speed at this
> > > point of time (about 40-60 MBps for file transfer over Gigabit ethernet
> > > connection).
> > >
> > > Your pointers will be very much appreciated.
> > >
> > > Hakan
> > >
> > > sudo cat /proc/net/bonding/bond0
> > > ...
> > > Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
> > >
> > > Bonding Mode: IEEE 802.3ad Dynamic link aggregation
> > > Transmit Hash Policy: layer2+3 (2)
> > > MII Status: up
> > > MII Polling Interval (ms): 100
> > > Up Delay (ms): 0
> > > Down Delay (ms): 0
> > >
> > > 802.3ad info
> > > LACP rate: fast
> > > Min links: 0
> > > Aggregator selection policy (ad_select): stable
> > > System priority: 65535
> > > System MAC address: fa:aa:91:6b:bc:d2
> > > Active Aggregator Info:
> > > Aggregator ID: 1
> > > Number of ports: 2
> > > Actor Key: 9
> > > Partner Key: 1001
> > > Partner Mac Address: 00:38:df:d0:ee:49
> > >
> > > Slave Interface: enp6s0
> > > MII Status: up
> > > Speed: 1000 Mbps
> > > Duplex: full
> > > Link Failure Count: 1
> > > Permanent HW addr: 2c:fd:a1:c6:1f:19
> > > Slave queue ID: 0
> > > Aggregator ID: 1
> > > Actor Churn State: monitoring
> > > Partner Churn State: monitoring
> > > Actor Churned Count: 0
> > > Partner Churned Count: 0
> > > details actor lacp pdu:
> > > system priority: 65535
> > > system mac address: fa:aa:91:6b:bc:d2
> > > port key: 9
> > > port priority: 255
> > > port number: 1
> > > port state: 7
> > > details partner lacp pdu:
> > > system priority: 1
> > > system mac address: 00:38:df:d0:ee:49
> > > oper key: 1001
> > > port priority: 1
> > > port number: 71
> > > port state: 117
> > >
> > > Slave Interface: enp5s0
> > > MII Status: up
> > > Speed: 1000 Mbps
> > > Duplex: full
> > > Link Failure Count: 1
> > > Permanent HW addr: 2c:fd:a1:c6:1f:18
> > > Slave queue ID: 0
> > > Aggregator ID: 1
> > > Actor Churn State: monitoring
> > > Partner Churn State: monitoring
> > > Actor Churned Count: 0
> > > Partner Churned Count: 0
> > > details actor lacp pdu:
> > > system priority: 65535
> > > system mac address: fa:aa:91:6b:bc:d2
> > > port key: 9
> > > port priority: 255
> > > port number: 2
> > > port state: 7
> > > details partner lacp pdu:
> > > system priority: 1
> > > system mac address: 00:38:df:d0:ee:49
> > > oper key: 1001
> > > port priority: 1
> > > port number: 72
> > > port state: 69
> > > _______________________________________________
> > > Cialug mailing list
> > > Cialug at cialug.org
> > > https://www.cialug.org/cgi-bin/mailman/listinfo/cialug
> > >
> > _______________________________________________
> > Cialug mailing list
> > Cialug at cialug.org
> > https://www.cialug.org/cgi-bin/mailman/listinfo/cialug
> >
>
> _______________________________________________
> Cialug mailing list
> Cialug at cialug.org
> https://www.cialug.org/cgi-bin/mailman/listinfo/cialug
>
More information about the Cialug
mailing list