[netsa-tools-discuss] Flags type count in rwstats

[Angela Horneman]

Hello Hossam,

I'm not quite sure of your end goal. In your current rwstats command, the output is not showing the count of TCP flags. The "24" is the integer value of AP (ACK, PSH). The values for each flag are:
FIN = 1
SYN = 2
RST = 4
ACK = 16
URG = 32
If you add up the flags that occur in one flow, you get the integer flag value. Network flow only tells the distinct flags that occurred, not the number times they each occurred.

Before I try to help with your output, let's clarify your set-up. First, as I'm sure you know, network flow summarizes connections over a period of time. For TCP, the summaries will cover all packets of a TCP session and will not be exported as a flow record until the session is terminated (e.g Rst or Fin flags occur) or an active timeout occurs. For many products the active timeout is 30 minutes. Therefore a single flow record may span traffic for several minutes.

If you want to find your most active talkers by minute intervals, the files do not need to be set to be one minute intervals-the SiLK summary commands (rwstats, rwuniq, rwcount) have time bin options. However, you need to think about if you are measuring traffic per minute or counting the completed flows in a minute. If the first, you would need to write a script to process the longer flows into one-minute time lengths, or set the active timeout to be one minute (which might cause performance issues on the appliance that is generating the IPFIX).

Now, the further question is, given the following example data, what do you want the output to show? (This sample is very fake for illustration only.)

sTime    |eTime |sIP        |sPort   |dIP       |dPort  |proto  |flags    |bytes
:01:00    |:01:00  |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |PA        |10
:01:00    |:01:00  |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |R           |10
:01:00    |:01:03  |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |R           |10
:01:00    |:01:00  |x.x.x.1 |33333  |x.x.x.2 |76543  |6           |PA        |10
:01:02    |:01:00  |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |PA        |10


rwstats --fields=sTime,sIP,sPort,dIP,dPort,proto,flags --values=bytes -bin-time=60
will give
sTime    |sIP        |sPort   |dIP       |dPort  |proto  |flags    |bytes
:01:00    |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |PA        |10
:01:00    |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |R           |20
:01:00    |x.x.x.1 |33333  |x.x.x.2 |76543  |6           |PA        |10
:01:02    |x.x.x.1 |12345  |x.x.x.2 |98765  |6           |PA        |10

What values would you expect to see in your table:

sIP|sPort|dIP|dPort|pro|ACK|SYN|FIN|Bytes|


Angela Horneman
Analysis Team Lead
Member of the Technical Staff
CERT Software Engineering Institute
Carnegie Mellon University
ahorneman at cert.org<mailto:ahorneman at cert.org>



From: netsa-tools-discuss-bounces+ahorneman=cert.org at cert.org [mailto:netsa-tools-discuss-bounces+ahorneman=cert.org at cert.org] On Behalf Of Hossam Zalabany
Sent: Thursday, July 18, 2019 8:44 AM
To: netsa-tools-discuss at cert.org
Subject: [netsa-tools-discuss] Flags type count in rwstats

Dear Cert.

I am trying to run SilK to parse statistics of my IPFIX top talkers per minute, I configured the sensor to have a new file for each minute, and keeping the in and out pairing out of the scoop at the moment, all fine so far except that I am only able to count total number of TCP flags, I need to count each flag time separately like, ACK, SYN, and so.

the current command I use is :

rwstats --fields=sip,sport,dip,dport,protocol,flags --integer-tcp-flags --values=byte --count=10 ext2ext-sens1_20190718.09
INPUT: 41300277 Records for 25847944 Bins and 816419146922 Total Bytes
OUTPUT: Top 10 Bins by Bytes
                                    sIP|sPort|                                    dIP|dPort|pro|fla|               Bytes|    %Bytes|   cumul_%|
                          10.21.64.133 |60870|                             10.5.54.69| 2051|  6| 24|          7257019952|  0.888884|  0.888884|
                             10.5.63.12| 2049|                            10.1.223.88|  958|  6| 24|          7136227712|  0.874089|  1.762973|
                          10.21.64.133 |34073|                             10.5.54.69| 2051|  6| 24|          5803794764|  0.710884|  2.473857|
                             10.5.63.12| 2049|                              10.5.4.31| 1020|  6| 24|          3883330408|  0.475654|  2.949511|
                          10.21.64.133 |34595|                             10.5.54.69| 2051|  6| 24|          3857964856|  0.472547|  3.422058|
                             10.5.204.4| 5247|                          10.120.30.236|53789| 17|  0|          3674064752|  0.450022|  3.872080|
                          10.21.64.133 |37529|                             10.5.54.69| 2051|  6| 24|          3262020960|  0.399552|  4.271632|
                             10.6.8.250|    0|                             10.5.204.4|    0| 97|  0|          2994127260|  0.366739|  4.638371|
                           10.134.26.21|    0|                             10.3.250.6|    0| 97|  0|          2893997631|  0.354474|  4.992846|
                             10.5.63.12| 2049|                          10.134.144.21|  747|  6| 24|          2853011756|  0.349454|  5.342300|


what I desire to have is  sIP|sPort|dIP|dPort|pro|ACK|SYN|FIN|Bytes|


is there is any suggested steps ?


regards


Hossam
-------------- next part --------------
HTML attachment scrubbed and removed