[Chilli] Too sharp upload bandwidth limit on 1gbps wired network

Richard REY richard at rexy.fr
Mon Mar 2 21:12:37 UTC 2015 

I've just tested to disable "GRO - Generic Receive Offload" (ethtool -K 
gro off) on the coova Gb NIC, and it fixes the issue.
The result for the upload bandwidth :
  - W/O coova & GRO on = 800Mb/s (jumbo frames)
  - W/O coova & GRO off = 700Mb/s (jumbo frames)
  - W coova & GRO on = 70Kb/s (lot of ICMP packets "fragmentation needed")
  - W coova & GRO off = 250Mb/s (no ICMP - no jumbo frame - coova uses 
100% of CPU)

As i understand the meaning of GRO, can somebody explain me the 
technical problem with coova & "GRO on"?

Rexy

Le 28/02/2015 03:37, kl a écrit :
>
>
> Based on my testing, turn off generic receive offload fixes my issue of too sharp upload bandwidth limit on 1 Gb wired network.
>
> But the strange thing is older kernel has the same setting 'on' and without problem, while newer kernel has to turn it 'off'.
>
>
> On Saturday, February 28, 2015 6:01 AM, Richard REY <richard at rexy.fr> wrote:
> Yes, but this doesn't explain why there isn't any problem when the flows
> pass through the appliance. this issue is seen only when the TCP flows
> is sent to an internal processes
>
> I'll try to disable packet off-loading next monday.
> I'll also try to forbid the "jumbo frame" on NIC
>
> Rexy
>
> Le 27/02/2015 20:50, Alexandru Gheorghe a écrit :
>> Usually fragmentation needed means you need to adjust the MTU of your nic.
>>
>> Alex
>>
>>
>> On 02/26/2015 02:30 PM, Claus Stjernoe wrote:
>>> Hi Richard,
>>>
>>> Try to disable packet off-loading on the Gigabit NIC.
>>>
>>> Kind regards,
>>> Claus
>>>
>>> -----Oprindelig meddelelse-----
>>> Fra: chilli-bounces at coova.org [mailto:chilli-bounces at coova.org] På vegne af chilli-request at coova.org
>>> Sendt: 26. februar 2015 13:00
>>> Til: chilli at coova.org
>>> Emne: Chilli Digest, Vol 63, Issue 14
>>>
>>> Send Chilli mailing list submissions to
>>>      chilli at coova.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>      http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>> or, via email, send a message with subject or body 'help' to
>>>      chilli-request at coova.org
>>>
>>> You can reach the person managing the list at
>>>      chilli-owner at coova.org
>>>
>>> When replying, please edit your Subject line so it is more specific than "Re: Contents of Chilli digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>>      1. Re: Chilli Digest, Vol 63, Issue 12 (Richard REY)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Wed, 25 Feb 2015 22:48:56 +0100
>>> From: Richard REY <richard at rexy.fr>
>>> To: chilli at coova.org
>>> Subject: Re: [Chilli] Chilli Digest, Vol 63, Issue 12
>>> Message-ID: <54EE4348.2090503 at rexy.fr>
>>> Content-Type: text/plain; charset=windows-1252; format=flowed
>>>
>>> Hi,
>>> I use COOVA in its last version (SVN revision 492) running on an ALCASAR appliance (mageia 4.1 - kernel 3.14.32), all is OK excepted a flow issue.
>>> When a TCP stream goes across the appliance there isn't any flow issue (forward mode). BUT when a TCP stream is sent inside the appliance (to a HTTP proxy for example) the observed flow is very weak (Input mode). The problem is maximal when the card connected to COOVA is in the Gigabit mode.
>>>
>>> Our investigation :
>>>    From the appliance, we try to download a file located on a FTP server connected on the network controlled by COOVA. Without COOVA the observed flow in gigabit mode is about 888Mb/s whereas with COOVA we can see just
>>> 68 Kb/s (YES 68 Kb/s). Regarding the packets exchanged we can see ICMP packets sent by the appliance (Destination unreachable (Fragmentation needed).
>>> We found a by-pass ?solution? by fixing the speed of the card to 100Mb/s. At this speed the phenomenon is practically invisible (just one ICMP packet sent) and the flow approaches the maximum supported by the network card. On a virtual machine where we can't fix Ethernet speed, we must choose a 100Mb/s virtual Ethernet card when building the VM.
>>> We have the Pcap file of our investigation. I say it again, this issue is seen only when the TCP flows is sent to an internal processes (in 'forward' mode all is ok).
>>>
>>> Regards
>>> Rexy & ALCASAR team
>>>
>>> Le 23/02/2015 08:59, kl a ?crit :
>>>> I see problem on kernel version > 3.7 right up to 3.10.  Above 3.10 not tested.
>>>> However, in the other quoted post, it says kernel 2.6.39 upto 3.3.4. Above that not tested.
>>>> Something not consist here.
>>>>
>>>> So which kernel version you guys are using which are seeing this problem ?
>>>>
>>>>
>>>> On Monday, February 23, 2015 3:13 PM, Danny Magat <danny at sb-ps.ae> wrote:
>>>> This problem is still exist on my setup.
>>>> http://lists.coova.org/pipermail/chilli/2013-November/002437.html
>>>>
>>>> I hope that somebody will give us a hint on how to solve this.
>>>>
>>>> Regards,
>>>> Danny
>>>>
>>>> -----Original Message-----
>>>> From: chilli-bounces at coova.org [mailto:chilli-bounces at coova.org] On
>>>> Behalf Of chilli-request at coova.org
>>>> Sent: Sunday, February 22, 2015 4:00 PM
>>>> To: chilli at coova.org
>>>> Subject: Chilli Digest, Vol 63, Issue 12
>>>>
>>>> Send Chilli mailing list submissions to
>>>>       chilli at coova.org
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>       http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>>> or, via email, send a message with subject or body 'help' to
>>>>       chilli-request at coova.org
>>>>
>>>> You can reach the person managing the list at
>>>>       chilli-owner at coova.org
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than
>>>> "Re: Contents of Chilli digest..."
>>>>
>>>>
>>>> Today's Topics:
>>>>
>>>>       1. Too sharp upload bandwidth limit on 1gbps wired network (kl)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Sun, 22 Feb 2015 02:21:53 +0000 (UTC)
>>>> From: kl <kl522 at yahoo.com>
>>>> To: Chilli List <chilli at coova.org>
>>>> Subject: [Chilli] Too sharp upload bandwidth limit on 1gbps wired
>>>>        network
>>>> Message-ID:
>>>>        <122480453.6349415.1424571713923.JavaMail.yahoo at mail.yahoo.com>
>>>> Content-Type: text/plain; charset=UTF-8
>>>>
>>>>
>>>>
>>>> Nobody is facing this issue with coova chilli ?
>>>>
>>>> http://www.coova.org/node/5171
>>>>
>>>> https://github.com/coova/coova-chilli/issues/32
>>>>
>>>> Happens only on Gigabit wired network and certain kernel verson.
>>>> Applies to 1.2.9, 1.3.0 and other versions ....
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> _______________________________________________
>>>> Chilli mailing list
>>>> Chilli at coova.org
>>>> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>>>
>>>>
>>>> End of Chilli Digest, Vol 63, Issue 12
>>>> **************************************
>>>>
>>>>
>>>> _______________________________________________
>>>> Chilli mailing list
>>>> Chilli at coova.org
>>>> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>>> _______________________________________________
>>>> Chilli mailing list
>>>> Chilli at coova.org
>>>> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>>>
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> Chilli mailing list
>>> Chilli at coova.org
>>> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>>
>>>
>>> End of Chilli Digest, Vol 63, Issue 14
>
>>> **************************************
>>> _______________________________________________
>>> Chilli mailing list
>>> Chilli at coova.org
>>> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>
>>
>>
>> _______________________________________________
>> Chilli mailing list
>> Chilli at coova.org
>> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>>
> _______________________________________________
> Chilli mailing list
> Chilli at coova.org
> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
> _______________________________________________
> Chilli mailing list
> Chilli at coova.org
> http://lists.coova.org/cgi-bin/mailman/listinfo/chilli
>

More information about the Chilli mailing list