First, let me apologize as this is likely to be lengthy. I learned many
things from posts on this newsgroup and this is the first time I have been a
poster as opposed to a reader, and really hope to solve this problem ASAP as
it is hanging up my development timeline.

I am developing an application which requires the reliable, peforming use of
UDP. As many of you are already aware, the UdpClient class is inadequate for
functional UDP communications for a number of reasons. As such, my
implementation is based on the Socket class configured for UDP configuration.
I am moving files between peers, and I need to be able to guarantee each
packet makes it to the remote host by way of an ack.

My issue is this: I am testing on two computers on the same LAN, and am
seeing high and erratic packet loss. When I first started, before the ACKing,
I could move thousands of 16K packets across the network without dropping a
single one. After I built in ACKing and a UI, performance went down the tubes
and I started forking a new thread for each UI update that was necessary
because the redrawing was causing an issue. This thread updates a ProgressBar
and text listview item with progress every 2%. Once I did that, I was getting
RTT of <10ms, which is what was expected. However, I'm still seeing lots of
packet loss. The packet loss usually occurs about every 5%, and usually
stacks up fast. I have it configured not to send another packet until an ack
is received for the last one sent. I have a AutoResetEvent that blocks the
sending thread until it is signalled that an ack was received. The way the
ACKs are managed, the first 4 bytes are the byte position of the packet being
sent, and the receiver checks this number against its bytes received count,
and acks if it is greater, or resends the ack if it has already seen, written
and acked this packet. So basically, the sender is responsible for resending
and the receiver just acks as necessary as the packets come in.

As soon as the sender sends a packet, it uses a timer to set the timeout on
that packet to 3x the last RTT time, or 1000ms for the first packet before
the client knows what expected RTT is. After the first timeout, it doubles
the timeout -- i.e. if the RTT is 7, the first timeout would be 21, and the
second 42, and so on until it can get a packet across the network or passes a
threshold of resends that allows it to determine the connection is broken.
Usually what happens is I will see 5 or 6 timeouts in a row before I can get
a packet across. The receiver is not getting any of these 5 or 6 packets --
they never make it to the receiving network, as far as I can tell.

I have configured the application for asynchronous communication in an
attempt to solve the problem, but that is a difficult road to travel due to
the threading and synchronization required to support it. Ideally, I'd prefer
synchronous communication, but I have async working now so if I need to use
it I can.

My UDP packet size is dynamically adjusted to best suit the packet loss
ratio on the link, but PL behavior is consistent even at a constant packet
size of 1024.

Here is my theory:

I have seen lots of posts on "send" and "receive" buffers and the
possibility that they may be overflowing. All the buffers I'm aware of are
the byte arrays I allocate on the fly. This seems to make the most sense
given that, from what I've read already, if these buffers overflow packets
headed to them simply are dropped. This further makes sense because as the
resend timeout increases, it gives these buffers more time to clear out,
which would explain why 4-5 packets make it across with 7ms RTT each, and
then another 5 don't. Attempts to use SetSocketOption to set receive buffer
generate invalid operation exceptions. Please see the calls to
BeginReceiveFrom (in sub Listen) and the corresponding receive callback (sub
ReceiveCallback). If you think it is a buffer issue, please make
recommendations on how I might solve it. I am ignorant of send/receive buffer
logic or implementation aside from what's in my code below.

Here is the code for just the UDP class. The acking, timing out, and
resending is handled at the application layer by invoking the callback
delegate on the class that coordinates the tranfer.

Thanks to anyone who can shed some light on this in advance! This has been
plaguing me for weeks!

Josh


Imports System.net
Imports System.Net.Sockets
Imports System.Threading

Public Delegate Sub ExceptionHandler(ByRef ex As Exception)

Public Class Connection
    Private m_Socket As Socket
    Public m_RemoteIP As String
    Private m_LocalPort As Integer
    Private m_RemotePort As Integer
    Private m_Callback As ReceiveData       
    Private m_ManuallyClosed As Boolean = False
    Private m_CatchException10054 = False   
    Private m_LocalEP As IPEndPoint
    Private m_RemoteEP As IPEndPoint
    Private m_ExceptionHandler As ExceptionHandler
    Private m_SyncCloseObject As New Object

    Sub New(ByVal remoteIP As String, ByVal exception_handler As
ExceptionHandler)
        m_Socket = New Socket(AddressFamily.InterNetwork, SocketType.Dgram,
ProtocolType.Udp)
        m_Socket.SetSocketOption(SocketOptionLevel.Udp,
SocketOptionName.NoDelay, 1)
        m_RemoteIP = remoteIP
        m_ExceptionHandler = exception_handler
    End Sub
    'Establishes the local end point
    Sub Bind(ByVal port As Integer)
        Try
            Dim hostEntry As IPHostEntry = Dns.Resolve(Dns.GetHostName())
            m_LocalEP = New IPEndPoint(hostEntry.AddressList(0), port)
            m_LocalPort = port
            m_Socket.Bind(m_LocalEP)
        Catch ex As Exception
            DebugPrint("Bind Exception: " & ex.ToString())
            m_ExceptionHandler.Invoke(ex)
        End Try

    End Sub
    Sub SetCatchForciblyClosedException(ByVal bValue As Boolean)
        m_CatchException10054 = bValue
    End Sub

    'Sends data to a UDP iff connected
    Sub Send(ByVal bytes() As Byte, ByVal remotePort As Integer)

        Try
            m_Socket.BeginSendTo(bytes, 0, bytes.Length, SocketFlags.None,
New IPEndPoint(IPAddress.Parse(m_RemoteIP), remotePort), AddressOf
SendCallBack, Nothing)
        Catch ex As Exception
            DebugPrint("SendTo exception: " & ex.ToString())
            m_ExceptionHandler.Invoke(ex)
        End Try
    End Sub
    Sub SendCallBack(ByVal ar As IAsyncResult)       
        Dim numBytes As Integer
        Try
            numBytes = m_Socket.EndSendTo(ar)           
        Catch ex As Exception
            DebugPrint("EndSendTo exception: " & ex.ToString())
            m_ExceptionHandler.Invoke(ex)

        End Try

    End Sub
    Public Sub Close()

        'set flag indicating a manual close request has occurred, so when
ReceiveCallback returns with an exception,
        'it is properly caught
        SyncLock m_SyncCloseObject
            m_ManuallyClosed = True

            Try
                m_Socket.Close()
                m_Socket = Nothing
            Catch ex As Exception
                'do nothing, we don't care if it errors on close
            End Try
        End SyncLock
    End Sub
    Private Sub ReceiveCallBack(ByVal ar As IAsyncResult)
        Dim bytes() As Byte = ar.AsyncState
        Dim result As IAsyncResult
        Dim rEP As IPEndPoint
        Dim numBytes As Int32 = -1
        Dim tempEP As New IPEndPoint(IPAddress.Any, 0)

        SyncLock m_SyncCloseObject
            If m_ManuallyClosed Then
                DebugPrint("Socket cloed, bailing out of receive call back")
                Exit Sub
            End If
        End SyncLock

        Try
            numBytes = m_Socket.EndReceiveFrom(ar, tempEP)
            m_RemoteEP = CType(tempEP, IPEndPoint)

            If numBytes > 0 Then
                ReDim Preserve bytes(numBytes - 1)
                m_Callback.Invoke(bytes, numBytes)
            End If

        Catch SocketEx As SocketException
            'if it matches the specific Forcibly closed by remote host error
code
            If SocketEx.ErrorCode = 10054 Then 'Forcibly closed by remote host
                'and we're looking for it...
                If m_CatchException10054 Then
                    m_CatchException10054 = False 'reset the flag indicating
we caught it
                    DebugPrint("Caught 10054 forcibly closed exception as
expected")
                    Exit Try
                End If
            End If


            DebugPrint("Receive callback (1) Socket Exception: " &
SocketEx.ToString())
            m_ExceptionHandler.Invoke(SocketEx)
            Exit Sub
        End Try


        Try
            ReDim bytes(Constants.BufferSize)
            tempEP = New IPEndPoint(IPAddress.Any, 0)
            result = m_Socket.BeginReceiveFrom(bytes, 0, bytes.Length,
SocketFlags.None, tempEP, AddressOf ReceiveCallBack, bytes)
        Catch ex As Exception

            DebugPrint("Receive callback (1) Socket Exception: " &
ex.ToString())
            m_ExceptionHandler.Invoke(ex)
        End Try

    End Sub

    Sub Listen(ByVal callback As ReceiveData)
        Dim bytes = New [Byte](Constants.BufferSize) {}
        Dim tempEP As New IPEndPoint(IPAddress.Any, 0)
        m_Callback = callback
        m_Socket.BeginReceiveFrom(bytes, 0, bytes.Length, SocketFlags.None,
tempEP, AddressOf ReceiveCallBack, bytes)
    End Sub
    Public ReadOnly Property RemoteIPE() As IPEndPoint
        Get
            Return m_RemoteEP
        End Get
    End Property
    Public ReadOnly Property LocalIPE() As IPEndPoint
        Get
            Return m_LocalEP
        End Get
    End Property

End Class

Josh Bigelow wrote:
Why UDP requirement?  TCP is out?

I won't go into the details but UDP is absolutely necessary.

Show quoteHide quote

Incrementing the "Build"
XmlDocument.Load using MemoryStream...
Exception thrown during install
AMD/Dual-Core/64 bit: influence on compiled files? - Clients?
running app as a service - logoff user shuts down application