[netsa-tools-discuss] analysis pipeline alerting issues with EVAL block
asad
a.alii85 at gmail.com
Thu Nov 12 01:58:16 EST 2015
Good news to all, its working i tested Angela i saw STATISTICS block
working, compared it with EVAL block I see that I was looking for
events in 5 minutes window not 6 as in STATISTICS i corrected that and
its working now:) Thanks Timur and Angela you people are good.
On 11/12/15, asad <a.alii85 at gmail.com> wrote:
> Timur,
>
> This is so strange, consider this
>
> farhan at netflow:~/silkydata$ rwfilter --sensor=S0 --type=int2int
> --saddress=10.10.81.74 --start-date=2015/11/06T09:15
> --end-date=2015/11/06T09:20 --dport=137 --pass=stdout | rwuniq
> --field=sip --dip-distinct
> rwfilter: Warning: start-date precision greater than hours ignored
> rwfilter: Warning: end-date precision greater than hours ignored
> sIP|dIP-Distin|
> 10.10.81.74| 2928|
>
>
> I'm getting 2928 unique destination ip matches and analysis pipeline
> won't even detect 5?
> Why?
>
> On 11/11/15, Timur D. Snoke <tdsnoke at cert.org> wrote:
>> Asad, over what interval are those connections?
>>
>> You can try this:
>>
>> rwfilter \
>> --sensor=S0 \
>> --type=int2int \
>> --saddress=10.10.81.74 \
>> --start-date=2015/11/6 \
>> --dport=137
>> --pass=stdout \
>> | rwcount \
>> --bin-size=300 \
>> --skip-zeroes
>>
>>
>> Your evaluation is that an inside host talks to more than 5 outside hosts
>> during a five minute window. The previous rwfilter query will help us
>> find
>> the most active hour and then determine you can try the following
>> commands
>> to determine if the conditions are met. Assuming the busiest time slice
>> is
>> 12:30-12:40.
>>
>> rwfilter \
>> --sensor=S0 \
>> --type=int2int \
>> --saddress=10.10.81.74 \
>> --start-date=2015/11/6T12:30 \
>> --end-date=2015/11/16T12:40 \
>> --dport=137
>> --pass=stdout \
>> | rwuniq --field=sip —dip-distinct
>>
>> And to see what the flows look like for that time period ordered by time
>> try
>> this:
>>
>> rwfilter \
>> --sensor=S0 \
>> --type=int2int \
>> --saddress=10.10.81.74 \
>> --start-date=2015/11/6T12:30 \
>> --end-date=2015/11/16T12:40 \
>> --dport=137
>> --pass=stdout \
>> | rwsort --field=stime \
>> | rwcut
>>
>> I think you are seeing the traffic but not the 5 dips in 5 minutes. If
>> you
>> get results that are different please include them in your response.
>>
>>
>>
>> I hope this helps,
>>
>> --
>> Timur Snoke
>> Network Defense Analyst
>> CERT/CC - Network Situational Awareness
>> Software Engineering Institute (SEI)
>> O: (412) 268-7806
>>
>>
>>
>>
>>
>>
>>
>>
>> On 11/11/15, 5:34 AM, "asad" <a.alii85 at gmail.com> wrote:
>>
>>>Here is the result of the filter
>>>
>>>farhan at netflow:~$ rwfilter --sensor=S0 --type=int2int
>>>--saddress=10.10.81.74 --start-date=2015/11/6 --dport=137
>>>--pass=stdout | rwstats --fields=sip,dip --values=records --top
>>>--count=5
>>>INPUT: 142555 Records for 2944 Bins and 142555 Total Records
>>>OUTPUT: Top 5 Bins by Records
>>> sIP| dIP| Records| %Records| cumul_%|
>>> 10.10.81.74| 192.168.33.74| 96| 0.067342| 0.067342|
>>> 10.10.81.74| 192.168.172.1| 96| 0.067342| 0.134685|
>>> 10.10.81.74| 192.168.181.45| 96| 0.067342| 0.202027|
>>> 10.10.81.74| 192.168.172.17| 96| 0.067342| 0.269370|
>>> 10.10.81.74| 10.10.232.21| 96| 0.067342| 0.336712|
>>>
>>>
>>>Now, I'm not sure why the ALERT will not still be seen.
>>>
>>>The auxLog.log shows
>>>
>>>"
>>>2015-11-10
>>> 05:22:08|Memory_Reset|5|Systems_using_many_different_protocols|130396|Common-worm-portS|0|Excessive-firwall-accepts-From-Multiple-Sources-to-a-Single-Destination|0|"
>>>
>>>
>>>My update pipeline.conf says
>>>
>>>FILTER non-local-to-remote
>>>TYPENAME IN_LIST [int2int]
>>>SIP NOT_IN_LIST "/root/silkydata/DMZ.set"
>>>SIP NOT_IN_LIST "/root/silkydata/voip.set"
>>>SIP NOT_IN_LIST "/root/silkydata/rns.set"
>>>DPORT IN_LIST [135, 136, 137, 138, 139, 444, 445, 995, 996, 997, 998,
>>>999, 8998, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135, 3136,
>>>3137, 3138, 3139, 3140, 3141, 3142, 3143,3146, 3147, 31$
>>>END FILTER
>>>
>>>
>>>EVALUATION Systems_using_many_different_protocols
>>>FILTER outgoing-flows
>>>FOREACH SIP
>>>CHECK THRESHOLD
>>>DISTINCT DPORT > 25
>>>TIME_WINDOW 3600 SECONDS
>>>END CHECK
>>>SEVERITY 7
>>>ALERT JUST_NEW_THIS_TIME
>>>ALERT ALWAYS
>>>CLEAR NEVER
>>>END EVALUATION
>>>
>>>
>>>EVALUATION Common-worm-ports
>>>FILTER non-local-to-remote
>>>FOREACH SIP
>>>CHECK THRESHOLD
>>>DISTINCT DIP > 5
>>>TIME_WINDOW 300 SECONDS
>>>END CHECK
>>>SEVERITY 7
>>>ALERT ALWAYS
>>>CLEAR NEVER
>>>END EVALUATION
>>>
>>>On 11/10/15, Timur D. Snoke <tdsnoke at cert.org> wrote:
>>>> Asad,
>>>>
>>>> The more information you provide the better our ability to help you
>>>> work
>>>> through what your configuration will need to be.
>>>>
>>>> It is good to be using TYPENAME as a limiting factor at the start of
>>>> your
>>>> FILTER, often we see that at least half of the traffic by volume is web
>>>> traffic so excluding that from your EVALUATION will provide a
>>>> performance
>>>> improvement.
>>>>
>>>> The INT2INT traffic usually reflects an incomplete site definition, it
>>>> would
>>>> be good to fix that because you might find that you have to make
>>>> special
>>>> accommodations in your FILTER composition.
>>>>
>>>> I hope this helps,
>>>>
>>>> --
>>>> Timur Snoke
>>>> Network Defense Analyst
>>>> CERT/CC - Network Situational Awareness
>>>> Software Engineering Institute (SEI)
>>>> O: (412) 268-7806
>>>>
>>>> From: asad <a.alii85 at gmail.com<mailto:a.alii85 at gmail.com>>
>>>> Date: Tuesday, November 10, 2015 at 9:32 AM
>>>> To: timur snoke <tdsnoke at cert.org<mailto:tdsnoke at cert.org>>
>>>> Subject: Re: [netsa-tools-discuss] analysis pipeline alerting issues
>>>> with
>>>> EVAL block
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 10, 2015 at 6:56 PM, Timur D. Snoke
>>>> <tdsnoke at cert.org<mailto:tdsnoke at cert.org>> wrote:
>>>> Hello Asad,
>>>>
>>>> This is an interesting question but I am not sure I understand from
>>>> your
>>>> description what you are trying to capture.
>>>>
>>>> Thanks Timur,
>>>>
>>>> Let me re-explain it in a clear way.
>>>>
>>>>
>>>>
>>>> You are using type and defining SIP in your filters but do not really
>>>> explain it in your use case.
>>>>
>>>> In my case I want the source IP which is involved in communicating with
>>>> common worm ports to at least x5 different destinations IP. Further I
>>>> want
>>>> this to match as much as 5 times. I think I need RECORD COUNT >5?
>>>>
>>>>
>>>> Are you looking for outside hosts that are trying to scan these ports
>>>> on
>>>> multiple hosts inside your network?
>>>> If this is the case you should just use IN for your TYPENAME, there are
>>>> no
>>>> web ports or icmp traffic that you are concerned with in your port
>>>> list.
>>>> If
>>>> the initiating host is outside then you wouldn’t want INT2INT, OUT or
>>>> OUTWEB. This change will potentially limit the total number of flows
>>>> being
>>>> evaluated.
>>>>
>>>> In my current system which is SIEM the rules are firing for traffic
>>>> direction which is int2int.
>>>>
>>>>
>>>> Can you show example flows that should match but doesn’t?
>>>>
>>>> I will prepare an rwfilter results for you and get back to you. I have
>>>> evidence of its using traffic logs from cisco asa I can show that If
>>>> you
>>>> want.
>>>>
>>>> --
>>>> Timur Snoke
>>>> Network Defense Analyst
>>>> CERT/CC - Network Situational Awareness
>>>> Software Engineering Institute (SEI)
>>>> O: (412) 268-7806
>>>>
>>>> From:
>>>> <netsa-tools-discuss-bounces+tdsnoke=cert.org at cert.org<mailto:netsa-tools-discuss-bounces+tdsnoke=cert.org at cert.org>>
>>>> on behalf of asad <a.alii85 at gmail.com<mailto:a.alii85 at gmail.com>>
>>>> Date: Tuesday, November 10, 2015 at 8:30 AM
>>>> To: "netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>"
>>>> <netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>>
>>>> Subject: [netsa-tools-discuss] analysis pipeline alerting issues with
>>>> EVAL
>>>> block
>>>>
>>>> Hello,
>>>>
>>>> I have a very simple alerting requirement
>>>>
>>>> "
>>>> when a destination ports matches ports which are considered as 'worm
>>>> ports'
>>>> traffic is send to 5 different unique ips in 5 minutes time.
>>>>
>>>> "
>>>>
>>>> I know on traffic level i'm getting required data since using traffic
>>>> logs
>>>> from the cisco asa (same device is also sending netflows) and it works
>>>> as
>>>> expected. I'm suppose to see an ip address but on alert.log I see
>>>> nothing.
>>>> Below is the logic.
>>>>
>>>> Any help?
>>>>
>>>>
>>>>
>>>> FILTER outgoing-flows
>>>> TYPENAME IN_LIST [in,int2int,out,outweb,outicmp]
>>>> SIP NOT_IN_LIST "/root/silkydata/DMZ.set"
>>>> SIP NOT_IN_LIST "/root/silkydata/voip.set"
>>>> SIP NOT_IN_LIST "/root/silkydata/rns.set"
>>>> END FILTER
>>>>
>>>> FILTER non-local-to-remote
>>>> TYPENAME IN_LIST [in,int2int,out,outweb,outicmp]
>>>> SIP NOT_IN_LIST "/root/silkydata/DMZ.set"
>>>> SIP NOT_IN_LIST "/root/silkydata/voip.set"
>>>> SIP NOT_IN_LIST "/root/silkydata/rns.set"
>>>> DPORT IN_LIST [135, 136, 137, 138, 139, 444, 445, 995, 996, 997, 998,
>>>> 999,
>>>> 8998, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135, 3136, 3137,
>>>> 3138, 3139, 3140, 3141, 3142, 3143,3146, 3147, 31$
>>>> END FILTER
>>>>
>>>>
>>>> EVALUATION Systems_using_many_different_protocols
>>>> FILTER non-local-to-remote
>>>> FOREACH SIP
>>>> CHECK THRESHOLD
>>>> DISTINCT DIP > 25
>>>> TIME_WINDOW 3600 SECONDS
>>>> END CHECK
>>>> SEVERITY 7
>>>> ALERT JUST_NEW_THIS_TIME
>>>> ALERT ALWAYS
>>>> CLEAR NEVER
>>>> END EVALUATION
>>>>
>>>>
>>>
>>
>
More information about the netsa-tools-discuss
mailing list