[netsa-tools-discuss] SiLK/libfixbuf ignoring sFlow records

[Kent Kuriyama]

This is a slightly different question on this same thread.

As you state there are multiple sflow streams in the data I am collecting.
There is only a single Foundry switch that I am collecting from, isn't it
odd that the switch generates multiple streams?  Thanks.

Kent

On Tue, Jun 16, 2015 at 6:24 AM, Kent Kuriyama <kent.kuriyama at gmail.com>
wrote:

> Mark,
>
> Thank you for your reply.  The problem of not collecting data is related
> to my rwflowpack startup file.  When I start up use the command line:
>
> /usr/local/sbin/rwflowpack --sensor-configuration=/usr/
> local/share/silk/etc/sensor.conf --root-directory=/data/silk
> --log-destination=syslog
>
> data is indeed being collected.  I suspect that I was distracted by the
> "sequence number" messages and did not bother to check the data directories.
>
> Again, thank you very much for your assistance.
>
> Kent
>
> On Tue, Jun 16, 2015 at 3:41 AM, Mark Thomas <mthomas at cert.org> wrote:
>
>>
>> Thank you for your query, and we especially thank you for including
>> the sample pcap file and your sensor.conf.
>>
>> The "sequence number mismatch" messages are not an indication of an
>> error.  Typically the message means that a UDP packet has arrived
>> out of sequence or gotten lost, or that data is arriving too quickly
>> for rwflowpack to process.  They are meant to be informative
>> warnings and do not prevent the records from being written to disk.
>>
>> Given the sequence numbers that I am seeing in both the log messages
>> and the pcap file, it appears that there at least are three
>> "streams" of data being intermingled.  Looking at the log messages,
>> I see a
>>
>> * 0x11f8b4 stream (decimal 1177780)
>>
>> * 0x219a9 stream (decimal 137641)
>>
>> * 0xd9c88f3, 0xd9c8900 stream (decimal 228362483)
>>
>> The streams are more pronounced in when you examine the sequence
>> numbers in pcap file:
>>
>>   $ tshark -V -n -r file.tcpdump  | grep 'sample, seq' | sort
>>       Flow sample, seq 1179103
>>       Flow sample, seq 1179104
>>       Flow sample, seq 1179105
>>       Flow sample, seq 1179106
>>       Flow sample, seq 1432328
>>       Flow sample, seq 228375879
>>       ...
>>       Flow sample, seq 228375905
>>       Flow sample, seq 614783
>>       Flow sample, seq 614784
>>
>> The "sequence number mismatch" message is going to appear whenever
>> rwflowpack/libfixbuf is expecting a packet from one stream but
>> receives a packet from a different stream.
>>
>> A solution to the sequence number mismatch issue is to configure
>> each of those streams to use a different agent number.  Alternately,
>> you may disable those warnings by setting the
>> SILK_LIBFIXBUF_SUPPRESS_WARNINGS environment variable to 1 before
>> starting rwflowpack.
>>
>> However, it is unusual that you are not seeing any data files.
>>
>> My colleague and I independently examined your pcap file and we were
>> both able to store the sFlow data in that file using rwflowpack.
>>
>> Is it possible that other syslog messages from rwflowpack are being
>> hidden by the "sequence number mismatch" messages?
>>
>> Every two minutes (the --flush-timeout default), you should see a
>> syslog message that looks like:
>>
>>   rwflowpack[45266]: 'IPA-Core': forward 31, reverse 0, ignored 0, sflow:
>> missing-pkts 0
>>
>> showing that the 'IPA-Core' probe generated 31 flow records from the
>> sFlow packets it received.  rwflowpack also flushes any in-memory
>> records to disk during at the flush-timeout, and this generates
>> messages such as:
>>
>>   rwflowpack[45266]:
>> /data/silk/int2int/2015/06/15/int2int-S0_20150615.19: 4 recs
>>
>> showing the full path to the file that was written to and the number
>> of records written.
>>
>> Are you seeing either of these sorts of message in syslog?
>>
>> Thanks again.
>>
>> -Mark
>>
>>
>> On Sun, 14 Jun 2015 10:39:34 -1000, Kent Kuriyama wrote:
>>
>> > I too am trying to collect sFlow records using Silk 3.10 and
>> > libfixbuf version 1.6.2.  I am running rwflowpack using the
>> > following command line:
>> >
>> >
>> > /usr/local/sbin/rwflowpack
>> > --sensor-configuration=/usr/local/share/silk/etc/sensor.conf
>> > --root-directory=/data/silk --log-destination=syslog
>> >
>> > Looking at the syslog I get a bunch of entries
>> >
>> > Jun 14 10:12:43 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
>> number mismatch for agent 0x0001, expecting 0x11f8b4 received 0xd9c88f3
>> > Jun 14 10:12:44 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
>> number mismatch for agent 0x0001, expecting 0xd9c8900 received 0x219a9
>> > Jun 14 10:12:44 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
>> number mismatch for agent 0x0001, expecting 0x219aa received 0xd9c8900
>> > Jun 14 10:12:44 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
>> number mismatch for agent 0x0001, expecting 0xd9c890a received 0x11f8b4
>> >
>> > I have not encountered any of the "Ignoring sFlow record: sFlow Record
>> > Length Mismatch" messages mentioned in this thread, applying the patch
>> to
>> > fbsflow.c did not make any difference.
>> >
>> > No data is being collected in /data/silk. I have attached a 10 packet
>> > capture of the sFlow traffic. Has anyone seen this problem? Thanks.
>> >
>> > Kent Kuriyama
>> >
>> > -------------- sensor.conf ---------------------
>> > probe IPA-Core sflow listen-on-port 6343 protocol udp end probe sensor
>> S0
>> > sflow-probes IPA-Core external-interface 216 internal-interface
>> remainder
>> > end sensor
>>
>
>
-------------- next part --------------
HTML attachment scrubbed and removed