[netsa-tools-discuss] Question about handling multiple streams of flow data.

[Mark Thomas]

Matt-

flowcap (via libfixbuf) is designed to support multiple IPFIX or
NetFlow v9 UDP streams arriving on a single port as long as flowcap
is able to distinguish the packets as distinct streams.  flowcap
makes this distinction by looking at

  * the source address of the sender

  * the source port of the sender (unless the
    SK_IPFIX_UDP_IGNORE_SOURCE_PORT environment variable is set and
    has a value other than 0 or the empty string)

  * the observation domain specified in the NetFlow/IPFIX header

If this distinction cannot be made, flowcap _may_ still work
correctly as long as all streams are either using the same set of
NetFlow templates or each stream is using a distinct set of
templates identifiers.  If the same template ID has a different
structure depending on the sender, flowcap will interpret at least
some of the data incorrectly.

Assuming either that flowcap can distinguish the streams or that
there are no template conflicts between the streams, flowcap should
be working correctly.  It may drop packets, but I certainly do not
expect it to write incorrect data.

You may have discovered a bug in SiLK or libfixbuf, and we would
certainly like to fix it.

Are you only seeing the flow record with bad timestamps and counts
when flowcap is running under high load?  Or do these bad flow
records appear even when flowcap is not stressed?

Let me note that problems with timestamps in NetFlow are not
uncommon.  Timestamps are often given as an offset from the device
initialization time, and sometimes this causes strange behavior---
particularly after the device has been up for 49 days and these
offset fields roll-over.  (There are also at least a half a dozen
other ways that timestamps can be given.)  If you add

  log-flags default record-timestamps

to the "probe" block in the "sensor.conf" file, flowcap records the
timestamps on every record it processes.  This produces a lot of log
messages, but it may help to debug the issue.

Setting flowcap's log-level to 'debug' may also help in determining
the source of these bad flow records (at the cost of more log
messages and more dropped packets).

I do not think the following applies when all packets arrive on the
same port, but I will mention it for completeness.  When flowcap
receives a template that uses an information element that is unknown
to libfixbuf, the libfixbuf code updates its information model with
the "alien" information element; however, this change to the
information model is not thread-safe.  When all streams are arriving
on a single port, there is only one thread processing the templates,
and this should not be a problem.

Any information you can provide to use to help us find, debug, and
fix this issue would certainly be welcome.

Thanks,

-Mark


On Thu, 30 Jul 2015 10:57:42 -0500, Matthew Markland wrote:

> All:
>
> I'm continuing to try to evaluate the SiLK toolchain but have run
> into a situation I cannot explain.
>
> In the environment I am working in the NetFlow from a large network is
> being reflected to a single port on my processing box. In other words,
> I am seeing flow information from multiple routers interleaved on a
> single port. When I run flowcap, it appears to capture data correctly,
> although it does issue many messages about sequence numbers (which I
> would expect). However, when attempting to do other processing on the
> files, many fields in the generated information appear to be incorrect
> (dates like 1776, or 21345 in flows, bad counts, etc). When I capture
> the same stream with a different collector (flowd in this case), we
> appear to get clean data without these incorrect values. Clearly this
> could be a result of flowd throwing something away, but it makes us
> suspicious that the SiLK system doesn't like having all this
> interleaved traffic coming in for processing.
>
> My question to the SiLK tools maintainers is whether their tools are
> designed to work in the environment I describe or whether they expect
> a one-to-one match between a flow generator (i.e. YAF or a router) and
> an instance of flowcap.
>
> Thanks for your time!
>
> Matt
> ----
> Matthew Markland
> mwmarkland at outlook.com
>
>