[netsa-tools-discuss] super_mediator outputs broken JSON

[Porter, Leigh]

Hey All,

It seems that super_mediator is breaking some JSON output, here is an example:

This is a single flow record from yaf into super_mediator:

{"flows":{"flowStartMilliseconds":"2019-10-14 09:49:16.230","flowEndMilliseconds":"2019-10-14 09:50:43.138","flowDurationMilliseconds":86.908,"reverseFlowDeltaMilliseconds":0.031,"protocolIdentifier":6,"sourceIPv4Address":"10.137.137.132","sourceTransportPort":55912,"packetTotalCount":10,"octetTotalCount":756,"flowAttributes":"00","sourceMacAddress":"00:00:00:00:00:00","destinationIPv4Address":"13.225.84.145","destinationTransportPort":443,"reversePacketTotalCount":11,"reverseOctetTotalCount":5682,"reverseFlowAttributes":"00","destinationMacAddress":"00:00:00:00:00:00","initialTCPFlags":"S","unionTCPFlags":"APRF","reverseInitialTCPFlags":"AS","reverseUnionTCPFlags":"APF","tcpSequenceNumber":"0x459881a3","reverseTcpSequenceNumber":"0xe6764318","ingressInterface":0,"egressInterface":0,"vlanId":"0x000","silkAppLabel":443,"ipClassOfService":"0x00","flowEndReason":"","collectorName":"C1","sslCertList":[{"sslCertIssuer":{"sslCertIssuerCountryName":"US","sslCertIssuerOrgName":"DigiCert Inc","sslCertIssuerCommonName":"DigiCert Global CA G2","sslCertIssuerOrgUnitName":[]},"sslCertSubject":{"sslCertSubCountryName":"US","sslCertSubState":"Washington","sslCertSubLocalityName":"Seattle","sslCertSubOrgName":"Amazon.com, Inc.","sslCertSubCommonName":"*.cloudfront.net","sslCertSubjectOrgUnitName":[]},"sslCertVersion":2,"sslCertSerialNumber":"09 f4 da 45 55 23 f0 34 18 b0 55 7e 6d ce c0 11","sslCertValidityNotBefore":"190717000000Z","sslCertValidityNotAfter":"200705120000Z","sslPublicKeyLength":271,"sslExtensions":{}},{"sslCertIssuer":{"sslCertIssuerCountryName":"US","sslCertIssuerOrgName":"DigiCert Inc","sslCertIssuerCommonName":"DigiCert Global Root G2","sslCertIssuerOrgUnitName":["www.digicert.com"]},"sslCertSubject":{"sslCertSubCountryName":"US","sslCertSubOrgName":"DigiCert Inc","sslCertSubCommonName":"DigiCert Global CA G2","sslCertSubjectOrgUnitName":[]},"sslCertVersion":2,"sslCertSerialNumber":"0c 8e e0 c9 0d 6a 89 15 88 04 06 1e e2 41 f9 af","sslCertValidityNotBefore":"130801120000Z","sslCertValidityNotAfter":"280801120000Z","sslPublicKeyLength":271,"sslExtensions":{}},"sslCertVersion":2,"sslCertSerialNumber":"63 18 0d 38 fb 80 97 78 a9 d0 35 a3 16 18 f8 40"}],"sslServerCipher":49199,"sslClientVersion":3,"sslRecordVersion":771,"sslServerName":"d3c3cq33003psk.cloudfront.net"}}


However, pipe that into something like 'jq' and you will see this:

parse error: ':' not as part of an object at line 1, column 2115

If you remove the last occurrence of ","sslExtensions":{}}" then all is fine.

This feature seems to pop up fairly regularly on lots of flows..




Leigh Porter
Managing Consultant
Roke Manor Research Limited
leigh.porter at roke.co.uk

________________________________________
Roke Manor Research Limited, Romsey, Hampshire, SO51 0ZN, United Kingdom.Part of the Chemring Group. 
Registered in England & Wales. Registered No: 00267550
http://www.roke.co.uk
_______________________________________
The information contained in this e-mail and any attachments is proprietary to Roke Manor Research Limited and 
must not be passed to any third party without permission. This communication is for information only and shall 
not create or change any contractual relationship.
________________________________________
-------------- next part --------------
HTML attachment scrubbed and removed