[netsa-tools-discuss] Help troubleshooting rwflowpack not recording flows

[Junk Mail]

Hello:

I got some guidance out of band that will hopefully resolve the issue related to UDP receive queue exhaustion.  We are going to work on making some changes to alleviate that.

Thanks,
Eric

On Nov 02, 2017, at 01:39 PM, Junk Mail <inetjunkmail at icloud.com> wrote:

Hello:

We've been using Silk for a few years.  Recently, we upgraded our hardware and performed a fresh install with a more current version.   As we began building the new server, we noticed on the old server that traffic from one particular interface on one of our routers was not being written to disk.  We did PCAP's on the collector of the data being sent to the rwflowpack port and could see the interface data in question in Wireshark but, for some reason, it wouldn't show up on disk after being processed by rwflowpack (it was working for about a year up until this point and the router's flow configuration and software version were unchanged).  Lots of service/server restarts and other steps later, we were at a loss.

We use UDP for flow data and use samplicator to send the flow data to multiple processes so we tried sending a copy of the flow data from the broken router to the new server we were building and, low and behold, the data was being written to disk.  Thinking we had found some strange bug in the old version and knowing that we were moving to the new server, we gave up on diagnosis and escalated the move to the new hardware.

Now, after a few months on the new server, we are seeing something similar.  It's the same router but this time, we not writing _ANY_ flows for that device.   Again, PCAP's confirm the data is reaching the rwflowpack listener.  We've tried adding --log-level=debug to the process but didn't see anything interesting in the logs.  I get that it being the same router suggests that the router may be the issue...and it may be, but the PCAP data suggests otherwise.  Since Wireshark can use the template and parse the data I'm inclined to think the data is good.

I don't have any info on the versions installed on the original server but the new server has SiLK version 3.12.2 and libfixbuf version 1.7.1.  The old server was probably a couple years old version so, if it is a bug, it's been around a while. This is IPFIX data from a JUNOS version 13.3 router.  

Are there any ideas of things we can look at next?

Thanks for any help,
Eric
-------------- next part --------------
HTML attachment scrubbed and removed