[netsa-tools-discuss] Problem with IN_BYTES and OUT_BYTES introduced in SiLK 3.16.0

[Bo Bayles]

After some more digging, I think the problem is with libfixbuf 1.8.0 (and
presumably 2.0 as well).

I compiled SiLK 3.17.1 with libfixbuf 1.7.1 and got the expected output
(i.e. that which I labeled as SiLK 3.14 / 3.15).

So I think the issue was that I was using the earlier libfixbuf for the
earlier SiLK, but not the later.

My overall complaint is the same, I suppose. But I think this is closer to
the right diagnosis? The fixbuf 1.7.1 release notes say:
> Bug Fix for NetFlow v9 devices that implement Reverse Information Elements

I wonder if that didn't make it to 1.8 / 2.0?

Many thanks,
-Bo

On Wed, May 2, 2018 at 12:51 PM, Bo Bayles <bbayles at obsrvbl.com> wrote:

> I found I started having an issue with processing data from some NetFlow v9
> exporters after upgrading SiLK.
>
> Specifically, rwcut and rwuniq used to properly reflect the bidirectional
> traffic being reported by these exporters, but they don't anymore.
>
> After doing some digging, I found that things changed for the worse in
> version
> 3.16.0. I suspect the relevant Release Note is:
>
> > Change processing of NetFlow v9 records so that, when SiLK is compiled
> > against libfixbuf 1.8.0, the OUT_BYTES and OUT_PKTS values are used when
> the
> > IN_BYTES and IN_PKTS values are 0.
>
> The specific problem occurs when an exporter specifies both traffic
> directions
> in its template. That is, both IN_BYTES and OUT_BYTES (RFC 3954 fields 1
> and
> 23, respectively) / both IN_PKTS and OUT_PKTS (fields 2 and 24).
>
> Given a flow like this, the SiLK tools used to report two rows - one for
> the
> bytes and packets from 192.168.12.205 -> 15.73.97.78, and one for the
> bytes and
> packets from 15.73.97.78 -> 192.168.12.205.
>
> * IP_SRC_ADDR: 192.168.12.205
> * IP_DST_ADDR: 15.73.97.78
> * IN_PKTS: 3987
> * OUT_PKTS: 1807
> * IN_BYTES: 634500
> * OUT_BYTES: 9580
>
> Output when captured by flowcap 3.14 (3.15 is the same):
>
> $ rwcut --fields 1,2,6,7 --no-columns "version_3.14.silk"
> sIP|dIP|packets|bytes|
> 192.168.12.205|15.73.97.78|3987|634500|
> 15.73.97.78|192.168.12.205|1807|9580|
>
> Output when captured by flowcap 3.16.1:
>
> $ rwcut --fields 1,2,6,7 --no-columns "version_3.16.1.silk"
> sIP|dIP|packets|bytes|
> 192.168.12.205|15.73.97.78|3987|634500|
>
> Clearly useful information is being lots in the second version - I hope
> this
> can be fixed in the next release?
>
> I've attached a PCAP with the originating v9 packets, plus the two SiLK
> files.
>
> Many thanks,
> -Bo
>
-------------- next part --------------
HTML attachment scrubbed and removed