[netsa-tools-discuss] Rwflowpack config questions

[Chacko P]

Hi Mark,
Thank you very much, the settings changes you recommended worked right off the bat. For the outcome of changing the LOG_POST_ROTATE, I’ll have to wait until tomorrow, but I expect it should work.
Thanks again,
Chacko 
PS. I might have a follow-on question once I understand the consequences of the increased number of flow records produced by YAF as a result of making these changes.


________________________________________
From: Mark Thomas <mthomas at cert.org>
Sent: Monday, December 7, 2015 9:04 PM
To: Chacko P
Cc: netsa-tools-discuss at cert.org
Subject: Re: [netsa-tools-discuss] Rwflowpack config questions

YAF is the tool that reads packet data and converts the packets into
flow records.  By default, YAF uses a 30 minute active timeout (for
long-lived flow records such as ssh sessions) and a 5 minute idle
timeout.

To get data closer to "real time", configure YAF to use 30 second
timeouts by either specifying these switches on the command line:

 --active-timeout=30 --idle-timeout=30

or adding those switches to the YAF_EXTRAFLAGS variable in the
yaf.conf file:

 YAF_EXTRAFLAGS="--silk --active-timeout=30 --idle-timeout=30"

Reducing these timeouts will increase the number of flow records
that YAF produces and the number of records that rwflowpack must
process.

The rwflowpack tool reads flow records and writes them to disk.  The
FLUSH_TIMEOUT setting (which sets the --flush-timeout switch)
determines how often the flow records in rwflowpack's memory are
written to disk.  When rwflowpack is processing flow records from a
busy link, rwflowpack is regularly copying records from its memory
to disk.


Setting LOG_POST_ROTATE to 'rm %s' should remove the previous day's
log file.  Since the post-rotate process runs with the same
privileges as rwflowpack, the leading "sudo" is not necessary.


I hope that helps.

-Mark


-----Original Message-----
From: Chacko P <chacko.p at incism.com>
Date: Sun, 6 Dec 2015 12:57:52 +0000
To: "netsa-tools-discuss at cert.org" <netsa-tools-discuss at cert.org>
Subject: [netsa-tools-discuss] Rwflowpack config questions

Hello,

I have a couple of questions regarding a single machine install of
rwflowpack and yaf.

My questions are as follows:

  1.  Our application that uses SiLK has some measure of real time
flow visualization.  Toward this end the config file has the entry
FLUSH_TIMEOUT=30 with the intent to get recent flow records. This
setting does not have an apparent effect as traffic that that flowed 5
minutes prior does not register with the rwfilter command with a
relevant time filter. It does, though, if the rwflowpack process is
cycled, which I expect cause a flush to disk. Is there some other
setting that needs to be configured as well to achieve this? Is there
an alternate way to get flow details up to, say, the most recent
minute?

  2.  The log directory is being filled with rwflowpack....gz
files. So I introduced the following line in the conf
file. LOG_POST_ROTATE='sudo rm %'. It's a guess at best since I was
not able to locate a working example. Will this work? Or does the
statement '(Old log files are not removed by rwflowpack; the
administrator should use another tool to remove them.)' supersede even
this and should look for another tool.

Any help would be greatly appreciated. If there is a document that has
the answers to these questions, then please point me to it.


 Thanks,



Chacko P., CISSP

Principal Consultant, Information Security.

Incism

email<mailto:chacko.p at incism.com>| web<http://www.incism.com/> |
profile<https://in.linkedin.com/in/chackopallathucheril> | social

Incisive views. Truly.