[netsa-tools-discuss] analysis pipeline alerting issues with EVAL block
asad
a.alii85 at gmail.com
Thu Nov 12 01:41:57 EST 2015
Timur,
This is so strange, consider this
farhan at netflow:~/silkydata$ rwfilter --sensor=S0 --type=int2int
--saddress=10.10.81.74 --start-date=2015/11/06T09:15
--end-date=2015/11/06T09:20 --dport=137 --pass=stdout | rwuniq
--field=sip --dip-distinct
rwfilter: Warning: start-date precision greater than hours ignored
rwfilter: Warning: end-date precision greater than hours ignored
sIP|dIP-Distin|
10.10.81.74| 2928|
I'm getting 2928 unique destination ip matches and analysis pipeline
won't even detect 5?
Why?
On 11/11/15, Timur D. Snoke <tdsnoke at cert.org> wrote:
> Asad, over what interval are those connections?
>
> You can try this:
>
> rwfilter \
> --sensor=S0 \
> --type=int2int \
> --saddress=10.10.81.74 \
> --start-date=2015/11/6 \
> --dport=137
> --pass=stdout \
> | rwcount \
> --bin-size=300 \
> --skip-zeroes
>
>
> Your evaluation is that an inside host talks to more than 5 outside hosts
> during a five minute window. The previous rwfilter query will help us find
> the most active hour and then determine you can try the following commands
> to determine if the conditions are met. Assuming the busiest time slice is
> 12:30-12:40.
>
> rwfilter \
> --sensor=S0 \
> --type=int2int \
> --saddress=10.10.81.74 \
> --start-date=2015/11/6T12:30 \
> --end-date=2015/11/16T12:40 \
> --dport=137
> --pass=stdout \
> | rwuniq --field=sip —dip-distinct
>
> And to see what the flows look like for that time period ordered by time try
> this:
>
> rwfilter \
> --sensor=S0 \
> --type=int2int \
> --saddress=10.10.81.74 \
> --start-date=2015/11/6T12:30 \
> --end-date=2015/11/16T12:40 \
> --dport=137
> --pass=stdout \
> | rwsort --field=stime \
> | rwcut
>
> I think you are seeing the traffic but not the 5 dips in 5 minutes. If you
> get results that are different please include them in your response.
>
>
>
> I hope this helps,
>
> --
> Timur Snoke
> Network Defense Analyst
> CERT/CC - Network Situational Awareness
> Software Engineering Institute (SEI)
> O: (412) 268-7806
>
>
>
>
>
>
>
>
> On 11/11/15, 5:34 AM, "asad" <a.alii85 at gmail.com> wrote:
>
>>Here is the result of the filter
>>
>>farhan at netflow:~$ rwfilter --sensor=S0 --type=int2int
>>--saddress=10.10.81.74 --start-date=2015/11/6 --dport=137
>>--pass=stdout | rwstats --fields=sip,dip --values=records --top
>>--count=5
>>INPUT: 142555 Records for 2944 Bins and 142555 Total Records
>>OUTPUT: Top 5 Bins by Records
>> sIP| dIP| Records| %Records| cumul_%|
>> 10.10.81.74| 192.168.33.74| 96| 0.067342| 0.067342|
>> 10.10.81.74| 192.168.172.1| 96| 0.067342| 0.134685|
>> 10.10.81.74| 192.168.181.45| 96| 0.067342| 0.202027|
>> 10.10.81.74| 192.168.172.17| 96| 0.067342| 0.269370|
>> 10.10.81.74| 10.10.232.21| 96| 0.067342| 0.336712|
>>
>>
>>Now, I'm not sure why the ALERT will not still be seen.
>>
>>The auxLog.log shows
>>
>>"
>>2015-11-10
>> 05:22:08|Memory_Reset|5|Systems_using_many_different_protocols|130396|Common-worm-portS|0|Excessive-firwall-accepts-From-Multiple-Sources-to-a-Single-Destination|0|"
>>
>>
>>My update pipeline.conf says
>>
>>FILTER non-local-to-remote
>>TYPENAME IN_LIST [int2int]
>>SIP NOT_IN_LIST "/root/silkydata/DMZ.set"
>>SIP NOT_IN_LIST "/root/silkydata/voip.set"
>>SIP NOT_IN_LIST "/root/silkydata/rns.set"
>>DPORT IN_LIST [135, 136, 137, 138, 139, 444, 445, 995, 996, 997, 998,
>>999, 8998, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135, 3136,
>>3137, 3138, 3139, 3140, 3141, 3142, 3143,3146, 3147, 31$
>>END FILTER
>>
>>
>>EVALUATION Systems_using_many_different_protocols
>>FILTER outgoing-flows
>>FOREACH SIP
>>CHECK THRESHOLD
>>DISTINCT DPORT > 25
>>TIME_WINDOW 3600 SECONDS
>>END CHECK
>>SEVERITY 7
>>ALERT JUST_NEW_THIS_TIME
>>ALERT ALWAYS
>>CLEAR NEVER
>>END EVALUATION
>>
>>
>>EVALUATION Common-worm-ports
>>FILTER non-local-to-remote
>>FOREACH SIP
>>CHECK THRESHOLD
>>DISTINCT DIP > 5
>>TIME_WINDOW 300 SECONDS
>>END CHECK
>>SEVERITY 7
>>ALERT ALWAYS
>>CLEAR NEVER
>>END EVALUATION
>>
>>On 11/10/15, Timur D. Snoke <tdsnoke at cert.org> wrote:
>>> Asad,
>>>
>>> The more information you provide the better our ability to help you work
>>> through what your configuration will need to be.
>>>
>>> It is good to be using TYPENAME as a limiting factor at the start of
>>> your
>>> FILTER, often we see that at least half of the traffic by volume is web
>>> traffic so excluding that from your EVALUATION will provide a
>>> performance
>>> improvement.
>>>
>>> The INT2INT traffic usually reflects an incomplete site definition, it
>>> would
>>> be good to fix that because you might find that you have to make special
>>> accommodations in your FILTER composition.
>>>
>>> I hope this helps,
>>>
>>> --
>>> Timur Snoke
>>> Network Defense Analyst
>>> CERT/CC - Network Situational Awareness
>>> Software Engineering Institute (SEI)
>>> O: (412) 268-7806
>>>
>>> From: asad <a.alii85 at gmail.com<mailto:a.alii85 at gmail.com>>
>>> Date: Tuesday, November 10, 2015 at 9:32 AM
>>> To: timur snoke <tdsnoke at cert.org<mailto:tdsnoke at cert.org>>
>>> Subject: Re: [netsa-tools-discuss] analysis pipeline alerting issues
>>> with
>>> EVAL block
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Nov 10, 2015 at 6:56 PM, Timur D. Snoke
>>> <tdsnoke at cert.org<mailto:tdsnoke at cert.org>> wrote:
>>> Hello Asad,
>>>
>>> This is an interesting question but I am not sure I understand from your
>>> description what you are trying to capture.
>>>
>>> Thanks Timur,
>>>
>>> Let me re-explain it in a clear way.
>>>
>>>
>>>
>>> You are using type and defining SIP in your filters but do not really
>>> explain it in your use case.
>>>
>>> In my case I want the source IP which is involved in communicating with
>>> common worm ports to at least x5 different destinations IP. Further I
>>> want
>>> this to match as much as 5 times. I think I need RECORD COUNT >5?
>>>
>>>
>>> Are you looking for outside hosts that are trying to scan these ports on
>>> multiple hosts inside your network?
>>> If this is the case you should just use IN for your TYPENAME, there are
>>> no
>>> web ports or icmp traffic that you are concerned with in your port list.
>>> If
>>> the initiating host is outside then you wouldn’t want INT2INT, OUT or
>>> OUTWEB. This change will potentially limit the total number of flows
>>> being
>>> evaluated.
>>>
>>> In my current system which is SIEM the rules are firing for traffic
>>> direction which is int2int.
>>>
>>>
>>> Can you show example flows that should match but doesn’t?
>>>
>>> I will prepare an rwfilter results for you and get back to you. I have
>>> evidence of its using traffic logs from cisco asa I can show that If you
>>> want.
>>>
>>> --
>>> Timur Snoke
>>> Network Defense Analyst
>>> CERT/CC - Network Situational Awareness
>>> Software Engineering Institute (SEI)
>>> O: (412) 268-7806
>>>
>>> From:
>>> <netsa-tools-discuss-bounces+tdsnoke=cert.org at cert.org<mailto:netsa-tools-discuss-bounces+tdsnoke=cert.org at cert.org>>
>>> on behalf of asad <a.alii85 at gmail.com<mailto:a.alii85 at gmail.com>>
>>> Date: Tuesday, November 10, 2015 at 8:30 AM
>>> To: "netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>"
>>> <netsa-tools-discuss at cert.org<mailto:netsa-tools-discuss at cert.org>>
>>> Subject: [netsa-tools-discuss] analysis pipeline alerting issues with
>>> EVAL
>>> block
>>>
>>> Hello,
>>>
>>> I have a very simple alerting requirement
>>>
>>> "
>>> when a destination ports matches ports which are considered as 'worm
>>> ports'
>>> traffic is send to 5 different unique ips in 5 minutes time.
>>>
>>> "
>>>
>>> I know on traffic level i'm getting required data since using traffic
>>> logs
>>> from the cisco asa (same device is also sending netflows) and it works
>>> as
>>> expected. I'm suppose to see an ip address but on alert.log I see
>>> nothing.
>>> Below is the logic.
>>>
>>> Any help?
>>>
>>>
>>>
>>> FILTER outgoing-flows
>>> TYPENAME IN_LIST [in,int2int,out,outweb,outicmp]
>>> SIP NOT_IN_LIST "/root/silkydata/DMZ.set"
>>> SIP NOT_IN_LIST "/root/silkydata/voip.set"
>>> SIP NOT_IN_LIST "/root/silkydata/rns.set"
>>> END FILTER
>>>
>>> FILTER non-local-to-remote
>>> TYPENAME IN_LIST [in,int2int,out,outweb,outicmp]
>>> SIP NOT_IN_LIST "/root/silkydata/DMZ.set"
>>> SIP NOT_IN_LIST "/root/silkydata/voip.set"
>>> SIP NOT_IN_LIST "/root/silkydata/rns.set"
>>> DPORT IN_LIST [135, 136, 137, 138, 139, 444, 445, 995, 996, 997, 998,
>>> 999,
>>> 8998, 3127, 3128, 3129, 3130, 3131, 3132, 3133, 3134, 3135, 3136, 3137,
>>> 3138, 3139, 3140, 3141, 3142, 3143,3146, 3147, 31$
>>> END FILTER
>>>
>>>
>>> EVALUATION Systems_using_many_different_protocols
>>> FILTER non-local-to-remote
>>> FOREACH SIP
>>> CHECK THRESHOLD
>>> DISTINCT DIP > 25
>>> TIME_WINDOW 3600 SECONDS
>>> END CHECK
>>> SEVERITY 7
>>> ALERT JUST_NEW_THIS_TIME
>>> ALERT ALWAYS
>>> CLEAR NEVER
>>> END EVALUATION
>>>
>>>
>>
>
More information about the netsa-tools-discuss
mailing list