[netsa-tools-discuss] Using rwfilter to search a generated record file

[Mark Thomas]

Currently there is no way to process a PCAP file directly in
rwflowpack.  You must use YAF to convert the file to IPFIX, and then
use rwflowpack to process the IPFIX file.  Configure a probe in the
sensor.conf file to to poll a directory for IPFIX files, such as:

 probe S0 ipfix
     poll-directory /tmp/rwflowpack/incoming
 end probe

Put your file in that directory.  Configure rwflowpack to use that
probe:

 sensor S0
     ipfix-probes S0
     internal-ipblocks  192.0.2.0/24
     external-ipblocks  remainder
 end sensor

YAF will generate flow records from the packets in the file.  YAF
treats all captured data the same---that is, it cannot pull NetFlow
records from the packet data.

-Mark


On Wed, 2 Sep 2015 23:00:44 +0400, Hosam Hittini wrote:

> Dear Mark,
>
> I actually need to confirm that rwflowpack is detecting the ext2ext
> traffic correctly (and storing it as ext2ext)
> I don¹t mind processing the file in rwflowpack
> Is there a way to process a PCAP file in rwflowpack?
> And can the PCAP contents be packets or they need to be packet logs (e.g.
> NetFlow)?
>
> Regards,
> Hosam Hittini
> System Security, Security Operations Centre
>
>
>
>
> On 9/2/15, 6:58 PM, "Mark Thomas" <mthomas at cert.org> wrote:
>
>>The assignment of the flowtype field (that is, class/type pair, such
>>as "all/inweb" and "all/ext2ext") of a SiLK record occurs when the
>>record passes through the packing logic that is loaded into
>>rwflowpack.
>>
>>A SiLK Flow file created with rwp2yaf2silk has the flowtype of its
>>records set to 0, which is "all/in".
>>
>>Unfortunately, there not a way to assign a flowtype to the records
>>in a file without processing the file with rwflowpack, which will
>>split in incoming file into several output files.
>>
>>The easiest way to find the flow records in your input file that
>>would be categorized as "ext2ext" would be create an IPset that
>>contains the internal IP space of your organization, and then run
>>
>> rwfilter --not-sipset=internal.set --not-dipset=internal.set \
>>     pass=ext2ext.rw input.rw
>>
>>where internal.set is the IPset of internal IPs, input.rw is the
>>file created by rwp2yaf2silk, and ext2ext.rw is the output file that
>>contains flow records that would be categorized as external to
>>external.
>>
>>I hope that helps.
>>
>>-Mark
>>
>>
>>-----Original Message-----
>>From: Hosam Hittini <hosam.hittini at ies.etisalat.ae>
>>Date: Wed, 2 Sep 2015 08:31:47 +0400
>>To: <netsa-tools-discuss at cert.org>
>>Cc: 'Majid Qureshi' <mmajid at ies.etisalat.ae>
>>Subject: [netsa-tools-discuss] Using rwfilter to search a generated record
>>	file
>>
>>Hi,
>>
>> 
>>
>>If I generated a record file from a PCAP file using rwp2yaf2silk
>>
>>Would rwfilter be able to determine if the traffic in that file is
>>external
>>to external according to its configuration?
>>
>>Thanks in advance
>>
>> 
>>
>>Regards,
>>
>>Hosam Hittini
>>
>>M: +971 50 3343 585
>>
>>System Security Maintenance & Support
>>
>>Etisalat
>>
>>