[netsa-tools-discuss] SiLK/libfixbuf ignoring sFlow records

[Kent Kuriyama]

Chris,

Yes, the switch is a Foundry multi-bladed chassis.  Thank you for the
explanation.

Kent

On Wed, Jun 17, 2015 at 2:06 AM, Chris Inacio <inacio at cert.org> wrote:

>
>
> > On Jun 16, 2015, at 12:27 PM, Kent Kuriyama <kent.kuriyama at gmail.com>
> wrote:
> >
> > This is a slightly different question on this same thread.
> >
> > As you state there are multiple sflow streams in the data I am
> collecting.  There is only a single Foundry switch that I am collecting
> from, isn't it odd that the switch generates multiple streams?  Thanks.
> >
> > Kent
> >
>
> Kent,
>
> Is the Foundry switch in question a blade chassis system with multiple
> switch blades?
>
> Often on hardware with that type of design, the flow generation is done by
> data plane components, and therefore every blade has its own flow
> generation.  On various vendors of gear, those blades then get combined
> into a single flow data stream our of the networking gear by a “mediator”
> process on the control processor blade.  I am assuming the Foundry doesn’t
> combine those streams.
>
> So in short, a lot odd, but understanding how the process of flow
> generation is done inside the networking gear - not shocking that the
> networking gear might operate this way.
>
> Regards,
> --
> Chris Inacio
> inacio at cert.org
>
>
> > On Tue, Jun 16, 2015 at 6:24 AM, Kent Kuriyama <kent.kuriyama at gmail.com>
> wrote:
> > Mark,
> >
> > Thank you for your reply.  The problem of not collecting data is related
> to my rwflowpack startup file.  When I start up use the command line:
> >
> > /usr/local/sbin/rwflowpack
> --sensor-configuration=/usr/local/share/silk/etc/sensor.conf
> --root-directory=/data/silk --log-destination=syslog
> >
> > data is indeed being collected.  I suspect that I was distracted by the
> "sequence number" messages and did not bother to check the data directories.
> >
> > Again, thank you very much for your assistance.
> >
> > Kent
> >
> > On Tue, Jun 16, 2015 at 3:41 AM, Mark Thomas <mthomas at cert.org> wrote:
> >
> > Thank you for your query, and we especially thank you for including
> > the sample pcap file and your sensor.conf.
> >
> > The "sequence number mismatch" messages are not an indication of an
> > error.  Typically the message means that a UDP packet has arrived
> > out of sequence or gotten lost, or that data is arriving too quickly
> > for rwflowpack to process.  They are meant to be informative
> > warnings and do not prevent the records from being written to disk.
> >
> > Given the sequence numbers that I am seeing in both the log messages
> > and the pcap file, it appears that there at least are three
> > "streams" of data being intermingled.  Looking at the log messages,
> > I see a
> >
> > * 0x11f8b4 stream (decimal 1177780)
> >
> > * 0x219a9 stream (decimal 137641)
> >
> > * 0xd9c88f3, 0xd9c8900 stream (decimal 228362483)
> >
> > The streams are more pronounced in when you examine the sequence
> > numbers in pcap file:
> >
> >   $ tshark -V -n -r file.tcpdump  | grep 'sample, seq' | sort
> >       Flow sample, seq 1179103
> >       Flow sample, seq 1179104
> >       Flow sample, seq 1179105
> >       Flow sample, seq 1179106
> >       Flow sample, seq 1432328
> >       Flow sample, seq 228375879
> >       ...
> >       Flow sample, seq 228375905
> >       Flow sample, seq 614783
> >       Flow sample, seq 614784
> >
> > The "sequence number mismatch" message is going to appear whenever
> > rwflowpack/libfixbuf is expecting a packet from one stream but
> > receives a packet from a different stream.
> >
> > A solution to the sequence number mismatch issue is to configure
> > each of those streams to use a different agent number.  Alternately,
> > you may disable those warnings by setting the
> > SILK_LIBFIXBUF_SUPPRESS_WARNINGS environment variable to 1 before
> > starting rwflowpack.
> >
> > However, it is unusual that you are not seeing any data files.
> >
> > My colleague and I independently examined your pcap file and we were
> > both able to store the sFlow data in that file using rwflowpack.
> >
> > Is it possible that other syslog messages from rwflowpack are being
> > hidden by the "sequence number mismatch" messages?
> >
> > Every two minutes (the --flush-timeout default), you should see a
> > syslog message that looks like:
> >
> >   rwflowpack[45266]: 'IPA-Core': forward 31, reverse 0, ignored 0,
> sflow: missing-pkts 0
> >
> > showing that the 'IPA-Core' probe generated 31 flow records from the
> > sFlow packets it received.  rwflowpack also flushes any in-memory
> > records to disk during at the flush-timeout, and this generates
> > messages such as:
> >
> >   rwflowpack[45266]:
> /data/silk/int2int/2015/06/15/int2int-S0_20150615.19: 4 recs
> >
> > showing the full path to the file that was written to and the number
> > of records written.
> >
> > Are you seeing either of these sorts of message in syslog?
> >
> > Thanks again.
> >
> > -Mark
> >
> >
> > On Sun, 14 Jun 2015 10:39:34 -1000, Kent Kuriyama wrote:
> >
> > > I too am trying to collect sFlow records using Silk 3.10 and
> > > libfixbuf version 1.6.2.  I am running rwflowpack using the
> > > following command line:
> > >
> > >
> > > /usr/local/sbin/rwflowpack
> > > --sensor-configuration=/usr/local/share/silk/etc/sensor.conf
> > > --root-directory=/data/silk --log-destination=syslog
> > >
> > > Looking at the syslog I get a bunch of entries
> > >
> > > Jun 14 10:12:43 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
> number mismatch for agent 0x0001, expecting 0x11f8b4 received 0xd9c88f3
> > > Jun 14 10:12:44 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
> number mismatch for agent 0x0001, expecting 0xd9c8900 received 0x219a9
> > > Jun 14 10:12:44 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
> number mismatch for agent 0x0001, expecting 0x219aa received 0xd9c8900
> > > Jun 14 10:12:44 IPAmonitor rwflowpack[40932]: sFlow Sample sequence
> number mismatch for agent 0x0001, expecting 0xd9c890a received 0x11f8b4
> > >
> > > I have not encountered any of the "Ignoring sFlow record: sFlow Record
> > > Length Mismatch" messages mentioned in this thread, applying the patch
> to
> > > fbsflow.c did not make any difference.
> > >
> > > No data is being collected in /data/silk. I have attached a 10 packet
> > > capture of the sFlow traffic. Has anyone seen this problem? Thanks.
> > >
> > > Kent Kuriyama
> > >
> > > -------------- sensor.conf ---------------------
> > > probe IPA-Core sflow listen-on-port 6343 protocol udp end probe sensor
> S0
> > > sflow-probes IPA-Core external-interface 216 internal-interface
> remainder
> > > end sensor
> >
> >
>
>
-------------- next part --------------
HTML attachment scrubbed and removed