[netsa-tools-discuss] Analysis Pipeline

[Daniel J Ruef]

Chris,
Thank you for your interest in Analysis Pipeline.

It sounds like you want to have it ingest silk data, so you'll need to be sure and specify --silk and --incoming-directory on the command line, or use a data source configuration file with --data-source-configuration-file with the contents being similar to:
PRIMARY DATA SOURCE silkPolling
    SILK BUILDER
    INCOMING DIRECTORY "/data/pipelineIncoming"
    ERROR DIRECTORY "/data/pipelineError"
END DATA SOURCE

In general...it sounds like from your errors, you're not specifying where pipeline gets its data from properly. You can either use command line settings to specify everything (if there is only one data source), or specify a data source configuration file using a command line switch (for any number of data sources).

When specifying the data source, you have to tell if what type of data (silk, yaf, or ipfix), and how it will get it (socket, single file, poll a directory).

This data source configuration file is different than the one used to specify filters, evaluations, statistic, etc. This part of the documentation isn't that clear, sorry about that. 

If you let me know what you're trying to do, and what the explicit error you're getting is, I can help you further.

Dan

-----Original Message-----
From: netsa-tools-discuss-bounces+druef=cert.org at cert.org <netsa-tools-discuss-bounces+druef=cert.org at cert.org> On Behalf Of Chris Frazier
Sent: Tuesday, June 23, 2020 1:15 AM
To: netsa-tools-discuss at cert.org
Subject: [netsa-tools-discuss] Analysis Pipeline

Using rwflowpack only option where rwflowpack is sending to rwflowappend and creates the pipeline incoming directory for pipelines data source

When i try to verify-config, I get the error data source file variable not set

In the conf file I am providing absolute paths to the variable in question

Sent from my iPhone