[netsa-tools-discuss] Support for PaloAlto NFv9 fields?

[Mark Thomas]

Mike-

On Thu, 5 Feb 2015 15:13:26 +0000, Mike Donovan wrote:

> Hello, I'm just setting up my first SiLK/FlowViewer system, to
> monitor NetFlow v9 data from a new Palo Alto firewall, which
> offers an option to export two extra fields containing the
> username and application:
...
> For now though, would it make sense to try to extend the packlogic
> flowtype code to at least hash those strings into a class/type
> pair or yaf-style applabel or in/out index value or bogus MAC
> address data I could access later on?  Can a packlogic module do
> all of the work, or would I need to customize rwflowpack's stream
> parsing or other parts just to make the strings available for
> hashing? Can PySiLK help?

In the SiLK 3.x series, the only file that has access to
IPFIX/NetFlow v9 information elements is
silk/src/libflowsource/skipfix.c.  This file is the interface
between SiLK and the information elements portions of the libfixbuf
library.

To teach libfixbuf about the information elements in the Palo Alto
Private Enterprise, either define the fields in the
ski_info_elements[] array or define your own array along those
lines.

The libfixbuf library maps ("trans-codes") each incoming IPFIX
record into the array of information elements specified by
"ski_extrwrec_spec[]".  To have access to the Palo Alto IEs, add the
name of elements that you defined in the ski_info_elements[] array.
The corresponding "ski_extrwrec_t" C structure must be kept in sync
with the ski_info_elements[] array, and this structure gives you
access to the data using C types.

The skiRwNextRecord() function examines the ski_extrwrec_t structure
and figures out how to fill in a SiLK Flow record.  That function
will have access to the App-ID and User-ID fields, and you may
manipulate/hash them as you wish and populate the field(s) of the
SiLK record that you think makes the most sense.  Note that
rwflowpack normally overwrites the values in the sensor and flowtype
(aka class/type) fields.

I believe skipfix.c is the only file you would need to change.  If
you put the data into the NextHop IP and/or input/output fields, no
changes to the packing logic should be required, though do specify
--pack-interfaces to rwflowpack since those fields are discarded by
default.

Good luck.

-Mark