[netsa-tools-discuss] Reflecting ipfix flow to another collector

[Poole, Ruth J.]

Iptables would have been a good solution, but, alas, we are stuck with an older Linux that doesn't have the TEE option support.  I did find a workaround, though it still has the disadvantage of a user-space reflector.  I modified the reflector code so that it uses the SO_REUSEADDR option to share the port.  Now I run the reflector to redirect the netflow V5 and V9 packets, with the ipfix flowcap listening directly to the original feed, along with the reflector.  This works, but now the reflector and ipfix flowcap are competing for resources on the same server.  Luckily, ipfix is a lower percentage of our netflow, at least for now.

Ruth Poole
Phone: 507-284-0456
Email: poole.ruth at mayo.edu<mailto:poole.ruth at mayo.edu>

From: B Galliart <bgallia at gmail.com<mailto:bgallia at gmail.com>>
Date: 2015, Tuesday, August 4, at 17:55
To: "Poole, Ruth J." <poole.ruth at mayo.edu<mailto:poole.ruth at mayo.edu>>
Cc: "netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>" <netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>>
Subject: Re: [netsa-tools-discuss] Reflecting ipfix flow to another collector

It sounds like your reflector is operating in user-space which not only has the disadvantage of changing the packet header but also is more likely to drop packets under heavy traffic/load.

To try to get maximum throughput, I used the linux iptables to act as the reflector.  The IP address of the reflector is 192.168.1.5 and while the destination MAC address gets changed by the kernel, the UDP/IP headers are kept the same.

Because a newer kernel is required than provided with CentOS 6, I use CentOS 7.  However, I disable the firewalld service and enable iptables-services.  Then in iptables I have the following rules:

-A PREROUTING -i bond0 -p udp -m udp --dport 4739 -j TEE --gateway 192.168.1.6
-A PREROUTING -i bond0 -p udp -m udp --dport 4739 -j TEE --gateway 192.168.1.7
-A INPUT -p udp -m udp --dport 4739 -j DROP

The last rule is to keep the reflector from producing port unreachable errors.

On 192.168.1.6 and 192.168.1.7, I have a loopback alias of lo:1 with the IP address 192.168.1.5 and the following sysctl/kernel parameters:
net.ipv4.conf.enp3s0.arp_ignore = 1
net.ipv4.conf.enp3s0.arp_announce = 2
net.ipv4.conf.enp4s0.arp_ignore = 1
net.ipv4.conf.enp4s0.arp_announce = 2

The only problem with the setup that I am using is that the reflected packets can not be routed to another VLAN.  The reflector and destination systems must all be on the same VLAN so that the reflector can deliver the packets strictly based on MAC address destination.

Then the sensor.conf has:
  probe ipfix1 ipfix
    listen-on-port 4739
    listen-as-host 192.168.1.5
    protocol udp
  end probe

I am not sure if this would help in your specific situation.  Hopefully you are able to find a solution.


On Tue, Aug 4, 2015 at 8:07 AM, Poole, Ruth J. <Poole.Ruth at mayo.edu<mailto:Poole.Ruth at mayo.edu>> wrote:
Hello all,

In our setup, we have a udp packet reflector which receives feeds from multiple sources.  Our feed consists of netflow v5, v9, and IPFIX, so we split the feed by netflow version, so that we can run separate instances of flowcap for each version.  We were having issues with the v9 and IPFIX not getting read correctly because by going through the reflector, the packets all looked like they were coming from the same source, but were actually from different routers, so the templates got mixed up between the different sources.  We modified the reflector so that now it retains the original source address in the ip and udp headers.  The v5 and v9 flowcop collectors now process the incoming packets correctly.  The IPFIX collector, however, appears to be rejecting the incoming packets from the reflector.  The reflector sets the ip checksum to zero and it gets recalculated.  It calculates and sets the udp header.  I have checked the packets in Wireshark, and the ip and udp checksums are correct.  I'm wondering is flowcap doing validation on the md5 checksum in the IPFIX records and rejecting them because it no longer matches after modifying the destination address in the original packet? I don't get any error messages even with log-level=debug  Is there a way to turn off this validation in flowcap?  Or do I have to also calculate a new md5 checksum in the reflector?

Ruth Poole
Phone: 507-284-0456<tel:507-284-0456>
Email: poole.ruth at mayo.edu<mailto:poole.ruth at mayo.edu>

-------------- next part --------------
HTML attachment scrubbed and removed