[netsa-tools-discuss] results with rwfilter

Tue Apr 14 14:39:44 EDT 2020

Try: rwcut –all-fields directly on some of your hourly files to see some of the data you are capturing.  Then you can fine tune the rwfilter.  VR, N

From: netsa-tools-discuss-bounces+nathan.rutland.ctr=us-cert.gov at cert.org <netsa-tools-discuss-bounces+nathan.rutland.ctr=us-cert.gov at cert.org> On Behalf Of Angela Horneman
Sent: Tuesday, April 14, 2020 1:34 PM
To: Kirk Olson <Kirk_Olson at secura.net>; Timur David Snoke <tdsnoke at cert.org>; netsa-tools-discuss at cert.org
Subject: Re: [netsa-tools-discuss] results with rwfilter

Hi Kirk,

Try

rwfilter --start=2020/04/10 --sensor=ACIDVS --type=in --protocol=0- --pass=stdout --site-config-file=/var/silk/data/silk.conf | rwcut --fields=proto,sip,dip --no-col | head

When I try to troubleshoot issues like this, my process is generally:

  1.  Verify I have data in my repository for the date and time I’m using.
  2.  Check my --type value. I got in the habit of explicitly typing the types I want. When I personally don’t get data, or the data I expect, it is almost always due the --type parameter.
  3.  Verify that the basic query for a full day is working (as above).
  4.  Once I get a basic query that works, add other limits (e.g. more types, hour specification) and parameters one at a time.

Angela Horneman

From: Kirk Olson <Kirk_Olson at secura.net<mailto:Kirk_Olson at secura.net>>
Date: Tuesday, April 14, 2020 at 2:16 PM
To: Timur Snoke <tdsnoke at cert.org<mailto:tdsnoke at cert.org>>, Angela Horneman <ahorneman at cert.org<mailto:ahorneman at cert.org>>, "netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>" <netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>>
Subject: RE: [netsa-tools-discuss] results with rwfilter

The ‘in’ and ‘int2int’ directories also contain files for that date and time:

root at ho-nflo-p01:/var/silk/data/ACIDVS
# ls -l
total 0
drwxr-xr-x. 3 root root 18 Apr 10 12:41 in
drwxr-xr-x. 3 root root 18 Apr  9 14:42 int2int
drwxr-xr-x. 3 root root 18 Apr  9 14:42 other

From: Timur David Snoke <tdsnoke at cert.org<mailto:tdsnoke at cert.org>>
Sent: Tuesday, April 14, 2020 12:24 PM
To: Kirk Olson <Kirk_Olson at secura.net<mailto:Kirk_Olson at secura.net>>; Angela Horneman <ahorneman at cert.org<mailto:ahorneman at cert.org>>; netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>
Subject: Re: [netsa-tools-discuss] results with rwfilter

Hi Kirk,

Your selection criteria might be too limiting, in the example that you presented you are looking for any UDP traffic to or from 172.18.18.151 between April 10 at 09:00 and April 12 at 11:00.

Try something that you know will get a result for a shorter time period to get a quicker validation that you have content.

rwfilter --start=2020/04/10T09 --sensor=ACIDVS --type=all  --protocol=0- --pass=/tmp/netstat/ACIDVS-2020041009.rw --site-config-file=/var/silk/data/silk.conf

This will query for all traffic seen by the sensor during the hour of April 10 at 09:00 and write it to a file.

You can also look at your data repository for the date/time in question and confirm that you have content.

-Timur Snoke

From: <netsa-tools-discuss-bounces+tdsnoke=cert.org at cert.org<mailto:netsa-tools-discuss-bounces+tdsnoke=cert.org at cert.org>> on behalf of Kirk Olson <Kirk_Olson at secura.net<mailto:Kirk_Olson at secura.net>>
Date: Tuesday, April 14, 2020 at 1:07 PM
To: Angela Horneman <ahorneman at cert.org<mailto:ahorneman at cert.org>>, "netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>" <netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>>
Subject: Re: [netsa-tools-discuss] results with rwfilter

My apologies Angela, this is the command which returns no data:

rwfilter --start=2020/04/10T09 --end=2020/04/12T11 --sensor=ACIDVS --type=all --any-address=172.18.18.151 --protocol=17 --pass=/tmp/netstat/HOCA01udp.rw --site-config-file=/var/silk/data/silk.conf

From: Angela Horneman <ahorneman at cert.org<mailto:ahorneman at cert.org>>
Sent: Tuesday, April 14, 2020 11:52 AM
To: Kirk Olson <Kirk_Olson at secura.net<mailto:Kirk_Olson at secura.net>>; netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>
Subject: Re: [netsa-tools-discuss] results with rwfilter

Hi Kirk,

In the command below you have a start year of 2020 and an end of 2015.

Angela Horneman
Situational Awareness Analysis Team Lead
CMU/SEI/CERT

From: <netsa-tools-discuss-bounces+ahorneman=cert.org at cert.org<mailto:netsa-tools-discuss-bounces+ahorneman=cert.org at cert.org>> on behalf of Kirk Olson <Kirk_Olson at secura.net<mailto:Kirk_Olson at secura.net>>
Date: Tuesday, April 14, 2020 at 12:46 PM
To: "netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>" <netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>>
Subject: [netsa-tools-discuss] results with rwfilter

I have been using the following rwfilter command to pull data from a sensor named ACIDVS:

rwfilter --start=2020/04/10T09 --end=2015/06/17T11 --sensor=ACIDVS --type=all --any-address=172.18.18.151 --protocol=17 --pass=HOCA01udp.rw

rwfilter does build a resultant file with headers in the top row but there is no data from the sensor in the file. Is there something simple I am missing here? I have read the docs and it is not obvious to me where I might be going wrong.

Thank you for your time.
-Kirk

Kirk Olson
Information Security Engineer
Direct: 920-224-7426
Toll Free: 800-558-3405 ext. 7426
[cid:16b8a6c7a1e6d227b41]<https://www.secura.net/>
website | blog | Facebook | Twitter | LinkedIn<https://www.secura.net/>
 <https://www.secura.net/>
Recognized among Ward’s Top 50 and rated A Excellent by A.M. Best. <https://www.secura.net/>
Confidentiality Note: This email may contain confidential and/or private information. If you received this email in error, please delete and notify sender.<https://www.secura.net/>
-------------- next part --------------
HTML attachment scrubbed and removed
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 3706 bytes
Desc: image001.jpg
URL: <http://lists.sei.cmu.edu/pipermail/netsa-tools-discuss/attachments/20200414/db89ec54/attachment.jpg>