[netsa-tools-discuss] segfaults in rwflowpack/rwflowappend

[John Green]

Hi,
Before I dig any deeper, does anyone else get very occasional segfaults
in the redblack tree routines of rwflowpack and rwflowappend (silk-3.9.0
and libfixbuf-1.6.1 compiled as per the spec file)?

Both look to be corrupted data structures - perhaps a thread
synchronization issue?  Alternatively it could be an issue with my
server.

rwflowpack occurs more frequently (once every few days)

/usr/sbin/rwflowpack --input-mode=fcfiles
--sensor-configuration=/etc/silk/sensor.conf --pack-interfaces
--flush-timeout=360 --file-cache-size=4096 --polling-interval=30
--site-config-file=/etc/silk/silk.conf
--incoming-directory=/netflow/silk/rwflowpack/incoming
--output-mode=incremental-files
--incremental-directory=/netflow/silk/rwflowpack/sender-incoming
--pidfile=/netflow/silk/rwflowpack/log/rwflowpack.pid --log-level=debug
--log-directory=/netflow/silk/rwflowpack/log --log-basename=rwflowpack

(gdb) bt
#0  __strcmp_sse2_unaligned ()
    at ../sysdeps/x86_64/multiarch/strcmp-sse2-unaligned.S:30
#1  0x00007fd023ac8bee in rb_traverse (insert=insert at entry=0,
key=0x3c84e00, 
    rbinfo=rbinfo at entry=0x2526510) at redblack/redblack.c:264
#2  0x00007fd023ac9017 in rbdelete (key=<optimized out>,
rbinfo=0x2526510)
    at redblack/redblack.c:177
#3  0x00007fd023d1d850 in remove_unseen (pd=0x25266a0, pd=0x25266a0)
    at skpolldir.c:294
#4  pollDir (vpd=0x25266a0) at skpolldir.c:528
#5  0x00007fd023d1e3be in sk_timer_thread (v_timer=0x3a80110) at
sktimer.c:193
#6  0x00007fd0227e40a4 in start_thread (arg=0x7fd021db8700)
    at pthread_create.c:309
#7  0x00007fd022518ccd in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

rwflowappend has occurred once
/usr/sbin/rwflowappend
--incoming-directory=/netflow/silk/rwflowappend/incoming
--error-directory=/netflow/silk/rwflowappend/error
--root-directory=/netflow/silk/data
--site-config-file=/etc/silk/silk.conf --compression-method=best
--pidfile=/netflow/silk/rwflowappend/log/rwflowappend.pid
--log-level=warning --log-destination=syslog --threads=16

(gdb) bt
#0  0x00007f426905f0ea in rb_delete_fix (x=0x7f4269298ac0 <rb_null>, 
    rootp=<optimized out>) at redblack/redblack.c:704
#1  rb_delete (z=<optimized out>, rootp=0x19d1090) at
redblack/redblack.c:655
#2  rbdelete (key=key at entry=0x19e1da0, rbinfo=0x19d1080)
    at redblack/redblack.c:183
#3  0x0000000000403146 in destroyOutputStream
(state=state at entry=0x19e1da0)
    at rwflowappend.c:778
#4  0x0000000000403ccc in appender_main (vstate=vstate at entry=0x19e1da0)
    at rwflowappend.c:1207
#5  0x000000000040408a in appender_main (vstate=0x19e1da0)
    at rwflowappend.c:1007
#6  0x00007f42684990a4 in start_thread (arg=0x7f42650e1700)
    at pthread_create.c:309
#7  0x00007f42681cdccd in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

I'll try rebuilding without optimization to enable better debugging.

John

Jisc is a registered charity (number 1149740) and a company limited by guarantee which is registered in England under Company No. 5747339, VAT No. GB 882 5529 90. Jisc’s registered office is: One Castlepark, Tower Hill, Bristol, BS2 0JA. T 0203 697 5800.
 
Jisc Collections and Janet Ltd. is a wholly owned Jisc subsidiary and a company limited by guarantee which is registered in England under Company No. number 2881024, VAT No. GB 614 9442 38. The registered office is: Lumen House, Library Avenue, Harwell, Didcot, Oxfordshire, OX11 0SG. T 01235 822200.