Skip to content

[BUG] SIGABRT in SRT:SndQ thread #2836

Description

@jeandube

Describe the bug
SRT group sender/listener asserts in SRT:SndQ thread

To Reproduce
System under test: SRT group sender/listener asserts when breaking backup link
eth0: main link (weight=1), eth1: backup link (default weight)
test cycle: network connector of backup link unplug/wait-1-2 secs/replug,
May takes many cycles depending on setup and peer.
MX4E -> SRT GW: 1-2 cycles (100% reproducible)
MX4e -> mxphub: 20-30 cycles (hard to reproduce)
libSRT 1.5.3, no encryption, 2 distinct network adapters used on each peer.

App running in gdb with a modified libsrt.so.1.5.3 with SRT_ASSERT remapped in our own _DoAssert() function (using SIGSEGV)
In gdb session below, assert (len <=cap) cap=1332 and len=18446744073551542776 (-158008840).

[15/12/2023` 10:36:41.730] ASSERT occurred in setLength(/home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp:261).
Expression = "len <= cap"
Process = "sessmgrd"
Thread ID = 19106 "SRT:SndQ:w2"
[Fri Dec 15 10:36:41 2023,sessmgrd:19061] threadcheck.c(__ShowThreadsInfo:1088): THREADCHECK: *THREAD "SRT:SndQ:w2"[19106] at worker(queue.cpp:526), Changes=6, Iterations=6, IsPaused=0, Changed=0s ago.
Thread 32 "SRT:SndQ:w2" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 19106]
0x0000007ff7ebd430 in _DoAssert (szExpression=0x7ff7d34258 "len <= cap",
szFunction=0x7ff7d343b8 <srt::CPacket::setLength(unsigned long, unsigned long)::FUNCTION> "setLength",
szFileName=0x7ff7d341f8 "/home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp", iLineNumber=261) at dbgtrace.c:292
292 dbgtrace.c: No such file or directory.
(gdb) [ 849.409384] axienet_phy_ethtool_get_link_ksettings:2657 ior off:180 1 b
bt
#0 0x0000007ff7ebd430 in _DoAssert (szExpression=0x7ff7d34258 "len <= cap",
szFunction=0x7ff7d343b8 <srt::CPacket::setLength(unsigned long, unsigned long)::FUNCTION> "setLength",
szFileName=0x7ff7d341f8 "/home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp", iLineNumber=261) at dbgtrace.c:292
#1 0x0000007ff7cecda8 in srt::CPacket::setLength (this=0x7f1fffe8c8,
len=18446744073551542776, cap=1332)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp:261
#2 0x0000007ff7c827cc in srt::CSndBuffer::readData (this=0x63e7b0,
w_packet=..., w_srctime=..., kflgs=0, w_seqnoinc=@0x7f1fffe514: 0)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/buffer_snd.cpp:318
#3 0x0000007ff7caec54 in srt::CUDT::packUniqueData (this=0x638e38,
w_packet=...)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/core.cpp:9627
#4 0x0000007ff7cae7ec in srt::CUDT::packData (this=0x638e38, w_packet=...,
w_nexttime=..., w_src_addr=...)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/core.cpp:9514
#5 0x0000007ff7cf4790 in srt::CSndQueue::worker (param=0x7f7807f5f0)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/sr---Type to continue, or q to quit---up
tcore/queue.cpp:585
#6 0x0000007ff7facfd8 in ?? () from /lib/libpthread.so.0
#7 0x0000007f9ac827c8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) up
#1 0x0000007ff7cecda8 in srt::CPacket::setLength (this=0x7f1fffe8c8,
len=18446744073551542776, cap=1332)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp:261
261 /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp: No such file or directory.
(gdb) p cap
$1 = 1332
(gdb) p len
$2 = 18446744073551542776
(gdb) up
#2 0x0000007ff7c827cc in srt::CSndBuffer::readData (this=0x63e7b0,
w_packet=..., w_srctime=..., kflgs=0, w_seqnoinc=@0x7f1fffe514: 0)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/buffer_snd.cpp:318
318 /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/buffer_snd.cpp: No such file or directory.
(gdb) [ 933.451703] axienet_phy_ethtool_get_link_ksettings:2657 ior off:180 1 3
[ 933.458395] axienet_phy_ethtool_get_link_ksettings:2677 ior off:1004 b b
p w_packet
$3 = (srt::CPacket &) @0x7f1fffe8c8: {static MAX_TIMESTAMP = 4294967295,
static TIMESTAMP_MASK = 4294967295, m_nHeader = {inarray = {4294967295, 0,
0, 0}}, m_PacketVector = {{ = {iov_base = 0x7f1fffe8c8,
iov_len = 16}, }, { = {iov_base = 0x641c98,
iov_len = 0}, }}, m_extra_pad = 0,
m_data_owned = false, m_DestAddr = {{sin = {sin_family = 0, sin_port = 0,
sin_addr = {s_addr = 0}, sin_zero = "\000\000\000\000\000\000\000"},
sin6 = {sin6_family = 0, sin6_port = 0, sin6_flowinfo = 0, sin6_addr = {
__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {
0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}},
sin6_scope_id = 0}, sa = {sa_family = 0,
sa_data = '\000' <repeats 13 times>}}, len = 28}, m_zCapacity = 0,
m_iSeqNo = @0x7f1fffe8c8, m_iMsgNo = @0x7f1fffe8cc,
m_iTimeStamp = @0x7f1fffe8d0, m_iID = @0x7f1fffe8d4,
m_pcData = @0x7f1fffe8e8, static HDR_SIZE = 16, static UDP_HDR_SIZE = 28,
static SRT_DATA_HDR_SIZE = 44, static ETH_MAX_MTU_SIZE = 1500,
static SRT_MAX_PAYLOAD_SIZE = 1456}
(gdb)

***later session to grab m_pCurrBlock
bt
#0 0x0000007ff7ebd430 in _DoAssert (szExpression=0x7ff7d34258 "len <= cap",
szFunction=0x7ff7d343b8 <srt::CPacket::setLength(unsigned long, unsigned long)::FUNCTION> "setLength",
szFileName=0x7ff7d341f8 "/home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp", iLineNumber=261) at dbgtrace.c:292
#1 0x0000007ff7cecda8 in srt::CPacket::setLength (this=0x7f1fffe8c8,
len=18446744073551542776, cap=1332)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/packet.cpp:261
#2 0x0000007ff7c827cc in srt::CSndBuffer::readData (this=0x63e7b0,
w_packet=..., w_srctime=..., kflgs=0, w_seqnoinc=@0x7f1fffe514: 0)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/buffer_snd.cpp:318
#3 0x0000007ff7caec54 in srt::CUDT::packUniqueData (this=0x638e38,
w_packet=...)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/core.cpp:9627
#4 0x0000007ff7cae7ec in srt::CUDT::packData (this=0x638e38, w_packet=...,
w_nexttime=..., w_src_addr=...)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/srtcore/core.cpp:9514
#5 0x0000007ff7cf4790 in srt::CSndQueue::worker (param=0x7f7807f5f0)
at /home/jdube/sandbox/makito2_project/components/vendors/haisrt/srt.git/sr---Type to continue, or q to quit---
tcore/queue.cpp:585
#6 0x0000007ff7facfd8 in ?? () from /lib/libpthread.so.0
#7 0x0000007f9ac827c8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) p m_pCurrBlock
$4 = (srt::CSndBuffer::Block *) 0x6491a0
(gdb) p *m_pCurrBlock
$5 = {m_pcData = 0x641c98 "", m_iLength = -158008840, m_iMsgNoBitset = 0,
m_iSeqNo = 0, m_tsOriginTime = {m_timestamp = 0}, m_tsRexmitTime = {
m_timestamp = 0}, m_iTTL = 64, m_pNext = 0x6491e0}
(gdb)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: BugIndicates an unexpected problem or unintended behavior[core]Area: Changes in SRT library core

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions