[netsa-tools-discuss] Fwd: silk

Daniel Hermans daniel.hermans at gmail.com
Wed Dec 13 16:26:11 EST 2017


Also - am curious why the flow is being processed by ski_yafrec_next rather
than ski_nf9rec_next (which has the quirks logic)?
or is that the normal flow?

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101 [0x7f98ec0014c0] skiTemplateCallback()

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE sourceIPv4Address (0/8)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE destinationIPv4Address (0/12)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE ingressInterface (0/10)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE egressInterface (0/14)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE sourceTransportPort (0/7)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE destinationTransportPort (0/11

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0000008000, IE protocolIdentifier (0/4)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101, bmap 0x0100008000, IE octetDeltaCount (0/1)

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Will process template
0x0101 with the YAF template

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101 [0x7f98ec0014c0], bmap 0x0000712c, written

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: 'ME1A': conn =
0x7f98ec002a00, source = 0x2502080, fbuf = 0x2561f50

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101 [0x7f98ec0014c0], bmap 0x0000712c, read by ski_yafrec

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Found zero bytes or
packets; byte=344, pkt=0, rev_byte=0, rev_pkt=0

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]:
IGNORED|10.39.252.91|10.39.132.10|51702|25|6|0|344|byte or packet count is
zero|

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Domain 0X0001,
TemplateID 0X0101 [0x7f98ec0014c0], bmap 0x0000712c, read by ski_yafrec

Dec 14 08:17:08 ip-10-39-128-243 rwflowpack[9199]: Found zero bytes or
packets; byte=7173, pkt=0, rev_byte=0, rev_pkt=0



Thanks!

On Thu, Dec 14, 2017 at 8:14 AM, Daniel Hermans <daniel.hermans at gmail.com>
wrote:

> Hi,
> Thanks for prompt reply.
> Apologies for not including my config files but yes I am using the
> zero-packets quirk:
>
> probe ME1A netflow-v9
>   listen-on-port 2055
>   listen-as-host 10.39.32.50
>   protocol udp
>   quirks zero-packets
> end probe
>
> group local-networks
>   ipblocks 10.0.0.0/8
> end group
>
> sensor ME1A
>   netflow-v9-probes ME1A
>   internal-ipblocks @local-networks
>   external-ipblocks remainder
> end sensor
>
> i recompiled with TRACE active - i just get a new message:
> Found zero bytes or packets; byte=344, pkt=0, rev_byte=0, rev_pkt=0
>
> inexpertly reading the code - i see the quirk is processed in this if
> block:
> } else if (!(record->bmap & NF9REC_INITIATOR)) }
>
> which it never enters for my data
>
>
> So as you suggest i'm doing something wrong in my config for it not to
> process the quirk
> Any ideas appreciated!
>
> On Thu, Dec 14, 2017 at 6:50 AM, Mark Thomas <mthomas at cert.org> wrote:
>
>> Daniel-
>>
>> Thank you for using the NetSA tools.
>>
>> SiLK is ignoring your data because the template used by your router
>> does not include the number of packets in the flow.  The template
>> include the octetDeltaCount but not a corresponding
>> packetDeltaCount.
>>
>> There is a work-around for this issue in SiLK, which requires that
>> you edit the sensor.conf file that rwflowpack is using.  In the
>> probe block for your router, add the following:
>>
>>   quirks zero-packets
>>
>> This causes rwflowpack to use a packet count of 1 for all incoming
>> flow records that do not have their own packet count, and that
>> should allow rwflowpack to store the records.
>>
>> I hope that solves your issue.
>>
>> -Mark
>>
>>
>> -----Original Message-----
>> From: Daniel Hermans <daniel.hermans at gmail.com>
>> Date: Wed, 13 Dec 2017 09:51:05 +1100
>> To: <netsa-tools-discuss at cert.org>
>> Subject: [netsa-tools-discuss] Fwd: silk
>>
>> Hi SiLK team,
>>
>> I'm trying to get SILK to ingest netflow v9 flows and having some issues
>> as
>> can be seen by the logs below (with the template shown):
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Will process template
>> 0x0101 with the YAF template
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Contains 8 Elements, Enabled by
>> SILK_IPFIX_PRINT_TEMPLATES
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   0, Length     4, IE           8, Name
>> sourceIPv4Address
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   1, Length     4, IE          12, Name
>> destinationIPv4Address
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   2, Length     4, IE          10, Name
>> ingressInterface
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   3, Length     4, IE          14, Name
>> egressInterface
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   4, Length     2, IE           7, Name
>> sourceTransportPort
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   5, Length     2, IE          11, Name
>> destinationTransportPort
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   6, Length     1, IE           4, Name
>> protocolIdentifier
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]: Domain 0X0001,
>> TemplateID 0X0101, Position   7, Length     4, IE           1, Name
>> octetDeltaCount
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.252.91|10.39.132.10|51702|25|6|0|344|byte or packet count
>> is
>> zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.39.221|10.11.40.143|8089|26286|6|0|7173|byte or packet
>> count
>> is zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.133.138|10.39.248.145|25|16976|6|0|566|byte or packet count
>> is zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.181.67|10.39.145.255|31055|443|6|0|2107|byte or packet
>> count
>> is zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.39.221|10.40.29.69|8089|62069|6|0|6970|byte or packet count
>> is zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.38.4|10.11.40.233|8089|29179|6|0|6021|byte or packet count
>> is
>> zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.39.46|10.108.10.84|58492|80|6|0|551|byte or packet count is
>> zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.39.120|10.40.28.248|9997|50238|6|0|12292|byte or packet
>> count
>> is zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.38.4|10.11.40.40|8089|33242|6|0|6125|byte or packet count
>> is
>> zero|
>>
>> Dec 12 14:47:27 ip-10-39-128-243 rwflowpack[4370]:
>> IGNORED|10.39.132.10|10.39.252.91|25|51702|6|0|566|byte or packet count
>> is
>> zero|
>>
>>
>>
>>
>>
>> So all messages are ignored. A typical flow record as seen in wireshark
>> looks like:
>>
>> Flow 1
>>
>>     SrcAddr: 10.39.154.46
>>
>>     DstAddr: 10.108.162.144
>>
>>     InputInt: 133
>>
>>     OutputInt: 0
>>
>>     SrcPort: 443
>>
>>     DstPort: 51769
>>
>>     Protocol: TCP (6)
>>
>>     Octets: 83026
>>
>>
>>
>>
>> Any thoughts as to what I'm doing wrong?
>>
>> I don't actually know the model number of Cisco device(s) sending the flow
>> ( the flows are sent from a service provider )  but i believe it's a Nexus
>> 9K
>>
>>
>> My sensor and silk configs are small and simple - 1 sensor, 1 v9 flow
>> etc..
>>
>>
>> I'm rusty on C but tried to enable TRACE mode based on a post I saw on the
>> archive list.
>>
>>
>> Didn't get too far with that either, as when I compile libfixbuf-1.8.0 it
>> installs the shared object /usr/local/lib/libfixbuf.so.3.2.0 rather than
>> /usr/local/lib/libfixbuf.so.1.8.0
>>
>>
>> The  silk-3.16.0 configure then does not find the shared object and
>> doesn't
>> enable netflow support ( fails the version check )
>>
>>
>>
>>
>> Thanks for any help!
>>
>
>
-------------- next part --------------
HTML attachment scrubbed and removed


More information about the netsa-tools-discuss mailing list