[netsa-tools-discuss] Maximum duration

[inetjunkmail]

Thanks for the guidance.  We have confirmed that SiLK is parsing the
timestamps properly so we'll look at some of the options you suggested.  I
appreciate your help.

On Wed, Jan 28, 2015 at 12:43 PM, Mark Thomas <mthomas at cert.org> wrote:

> My reply is below.
>
> On Wed, 28 Jan 2015 08:58:07 -0500, inetjunkmail at gmail.com wrote:
>
> > I just got SiLK installed and am seeing 100's of thee per second
> > in my logs:
> >
> > Jan 28 08:53:04 silk rwflowpack[13441]: Record's duration greater than
> that allowed in file
> '/data/silk/ext2ext/2588/07/15/ext2ext-S1_25880715.02': 1474356 > 4095
> >
> > This is data from several IPFIX Sensors.  Is the 4095 limit
> > tunable or does this indicate a problem with the data that's being
> > received?
> >
> > Thanks,
> > E
>
> Before we solve the long duration issue we should determine why SiLK
> thinks your flow records have a start date of July 15, 2588.
>
> When a flow exporter writes IPFIX data, there are several different
> information elements that it may use to express the timestamps on a
> record.  Some of these are absolute times, some are offsets from the
> initialization time, and some are offsets from the packet's export
> time that is contained in the IPFIX header.
>
> The IPFIX reading code in SiLK has code to handle these many types
> of time input.  SiLK makes a good-faith effort in the cases where
> the IPFIX record is under-specified---for example, the record
> specifies the timestamps as offsets from an initialization time but
> that initialization time is not provided.
>
> The odd times you see could be a result of
>
> * SiLK's good-faith effort being incorrect,
>
> * the flow record using a combination of information elements that
>   SiLK does not expect, or
>
> * a bug in one of the time-handling code-paths that has not been
>   exercised often.
>
> If you are using SiLK 3.10.0, you can tell SiLK to log how it is
> determining the times by modifying the "probe" block in the
> "sensor.conf" file to contain the line
>
>   log-flags default record-timestamps
>
>
> The maximum duration of 4095 is due to the file format that SiLK
> uses to store IPv4 flow records.  There are a couple of ways to tell
> SiLK to use a file format that does not have that limitation:
>
> * you can add the following to the probe block
>
>   quirks zero-packets
>
> * you can reconfigure and rebuild SiLK to support IPv6 flow records
>
> Either of those approaches cause SiLK to use a file format that
> supports a duration of 49 days.
>
> -Mark
>
-------------- next part --------------
HTML attachment scrubbed and removed