[netsa-tools-discuss] SiLK: Extending rwRec stucture from extended template

[Mark Thomas]

My reply is in-line.

On Tue, 28 Apr 2015 21:51:21 -0500, B. Galliart wrote:

> When I enable SILK_IPFIX_PRINT_TEMPLATES, it shows that my IPFIX
> exporter is providing bgpSourceAsNumber and bgpDestinationAsNumber
> but it appears these values are then discarded.

Correct.  The set of IPFIX information elements captured by SiLK is
fixed to the set listed here.
http://tools.netsa.cert.org/silk/faq.html#ipfix-fields

> Anyone have any tips, advice or guide of what needs to be done to
> extend SiLK to support these?

> From what I can tell, I will need to extend the rwRec structure to
> include these?

Yes.

> Based on how I read the code, at the very least, it looks like I
> need to modify libflowsource/skipfix.c and libsilk/rwrec.[ch].

Yep.  You also need to modify all the flow record file formats that
SiLK uses to properly save/retrieve those fields to/from disk and do
so in a way that does not break backward compatibility.

  src/libsilk/flowcapio.c
  src/libsilk/rwaugmentedio.c
  src/libsilk/rwaugroutingio.c
  src/libsilk/rwaugsnmpoutio.c
  src/libsilk/rwaugwebio.c
  src/libsilk/rwfilterio.c
  src/libsilk/rwgenericio.c
  src/libsilk/rwipv6io.c
  src/libsilk/rwipv6routingio.c
  src/libsilk/rwnotroutedio.c
  src/libsilk/rwroutedio.c
  src/libsilk/rwsplitio.c
  src/libsilk/rwwwwio.c

> Then, it seems, I need to update the utilities that I want to act
> on the new rwRec methods.

Correct.  Some of that can be handled by updating
src/libsilk/rwascii.[ch], but in general you need to visit each
utility.

I have handled this in the past by using Perl to look for every use
of some other macro from rwrec.h, for example, 'rwRec\w+SPort\(',
and then adding whatever is necessary to handle the fields I am
adding.

> But before
> I get entrenched in doing all of that, I want to make sure I
> wasn't missing a simpler solution.

As Chris mentioned, a hackish approach is to commandeer existing
field(s) and use it(them) for the field(s) you are adding.
I described how to store the MAC addresses in the NextHopIp field in
this posting.
https://lists.sei.cmu.edu/pipermail/netsa-tools-discuss/2014-November/000037.html

-Mark