Announcement

Collapse
No announcement yet.

OVERLAPPED SOCKETS - The saga continues...

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • OVERLAPPED SOCKETS - The saga continues...

    The overlapped I/O model in Winsock offers applications better super performance without having to create a thread for each connection.
    In essence it is asynchronous socket communications with event signaling when there is data to be read or any of the other reasons a normal socket might block.
    These sockets do not block.

    All of my code uses TCP NOTIFY but I need to improve performance, actually the number of devices I can chat with on a single comm thread. I am told Overlapped I/O is the way to go forward.

    First I am trying to get one socket working. The code below has some of the parts working.

    Take a look and see if you can make some suggestions that might get this working. I will keep going on it too.

    The idea is to have a LISTENING socket that then hands off to the ACCEPTING socket.
    Then you do a WSAWaitForMultipleEvents until you are signaled then you do a WSARecv.

    I am still figuring this out so I may have some things out of order.

    When it is totally right it should be able to accept a thousand+ connections and then trigger an event when one of them wants to talk. All asynchronous.
    You can Telnet to the port to mess with it. I am going to write a device simulator to stress test it when I get it working. I only have 10 devices in my office but want to see how many It can handle.

    SO FAR:

    Code:
    #COMPILE EXE
    #DIM ALL
    
    #INCLUDE "Win32API.inc"
    
    
    GLOBAL GL_hDlg             AS DWORD
    GLOBAL GL_LOG_FILE_NUMBER  AS LONG
    
    
    ENUM CONTROLS SINGULAR
     ENUM_STARTING_NUMBER = 100
     TIMER_1
     LISTBOX_VIEWER
     TCP_EVENT
    END ENUM
    
    ENUM MESSAGES SINGULAR
     ENUM_STARTING_MESSAGESS_NUMBER = %WM_APP
     WM_SOCKET
    END ENUM
    
    '------------------------------------------------------------------------------
     FUNCTION PBMAIN () AS LONG
    
       MAIN_DIALOG
       MY_WSACleanup
    
     END FUNCTION
    '------------------------------------------------------------------------------
     FUNCTION MAIN_DIALOG() AS LONG
    
       LOCAL RESULT            AS LONG
       LOCAL DIALOG_HEIGHT     AS LONG
       LOCAL DIALOG_WIDTH      AS LONG
       LOCAL DIALOG_STYLE      AS LONG
       LOCAL DIALOG_STYLE_EX   AS LONG
       LOCAL LISTBOX_STYLE     AS LONG
       LOCAL LISTBOX_STYLE_EX  AS LONG
    
       DIALOG_HEIGHT = 320
       DIALOG_WIDTH  = 575
    
       DIALOG_STYLE    = %WS_POPUP OR %WS_CAPTION OR %WS_SYSMENU OR %WS_MINIMIZEBOX OR %WS_VISIBLE OR %DS_MODALFRAME OR %DS_CENTER OR %DS_3DLOOK OR %DS_NOFAILCREATE OR %DS_SETFONT
       DIALOG_STYLE_EX = %WS_EX_CONTROLPARENT OR %WS_EX_LEFT OR %WS_EX_LTRREADING OR %WS_EX_RIGHTSCROLLBAR
    
       LISTBOX_STYLE    = %WS_CHILD OR %WS_VISIBLE OR %WS_BORDER OR %WS_TABSTOP OR %WS_VSCROLL OR %LBS_NOTIFY OR %LBS_DISABLENOSCROLL
       LISTBOX_STYLE_EX = %WS_EX_LEFT OR %WS_EX_LTRREADING OR %WS_EX_RIGHTSCROLLBAR
    
       DIALOG NEW %HWND_DESKTOP, "WinSock2_Test", 0, 0, DIALOG_WIDTH, DIALOG_HEIGHT, DIALOG_STYLE, DIALOG_STYLE_EX, TO GL_hDlg
    
       CONTROL ADD LISTBOX, GL_hDlg, %LISTBOX_VIEWER,,  10, 10, 575-20, 320-60, LISTBOX_STYLE, LISTBOX_STYLE_EX
    
       DIALOG SHOW MODAL GL_hDlg, CALL MAIN_DIALOG_CALLBACK
    
     END FUNCTION
    '------------------------------------------------------------------------------
     CALLBACK FUNCTION MAIN_DIALOG_CALLBACK
    
        SELECT CASE AS LONG CB.MSG
    
          CASE %WM_INITDIALOG
            SETTIMER(GL_hDlg,  %TIMER_1,  200)
    
          CASE %WM_TIMER
            KILLTIMER(GL_hDlg, %TIMER_1)
            START_COMM_THREAD
    
        END SELECT
    
     END FUNCTION
    
    '------------------------------------------------------------------------------
     FUNCTION START_COMM_THREAD()AS LONG
    
        LOCAL THREAD_HANDLE     AS LONG
        LOCAL THREAD_RESULT     AS LONG
    
        THREAD CREATE THREADED_COMM_THREAD(0) TO THREAD_HANDLE
        THREAD CLOSE  THREAD_HANDLE TO THREAD_RESULT
    
     END FUNCTION
    '------------------------------------------------------------------------------
     THREAD FUNCTION THREADED_COMM_THREAD(BYVAL NO_VAL AS LONG)AS LONG
      ON ERROR GOTO ERROR_CONTROL
    
       LOCAL RESULT_LONG          AS LONG
    
        LOCAL LISTENING_SOCKET     AS DWORD
        LOCAL ACCEPTING_SOCKET     AS DWORD
        LOCAL TYPE_SOCKET_ADDRESS  AS sockaddr_in
    
        LOCAL nBytesReceived       AS LONG
        LOCAL nFlags               AS LONG
    
        LOCAL MY_WSAOVERLAPPED     AS WSAOVERLAPPED
        LOCAL MY_WSAPROTOCOL_INFOA AS WSAPROTOCOL_INFOA
    
        LOCAL MY_WSADATA           AS WSADATA
    
       'The WSAStartup function initiates use of the Winsock DLL by a process.
        RESULT_LONG = WSAStartup(MAKWRD(2, 2), MY_WSADATA)
    
       'The WSASocket function creates a socket that is bound to a specific transport-service provider.
        LISTENING_SOCKET = WSASocket(%AF_INET, %SOCK_STREAM, %IPPROTO_TCP, BYVAL 0, 0, %WSA_FLAG_OVERLAPPED)
    
       'Set up the sock addr structure that the listening socket will be bound to. In this case, the structure holds the local IP address and the port.
        TYPE_SOCKET_ADDRESS.sin_family = %AF_INET
        TYPE_SOCKET_ADDRESS.sin_port = htons(4070)
        TYPE_SOCKET_ADDRESS.sin_addr.s_addr = inet_addr("192.168.1.193") 'or htonl(%INADDR_ANY)
    
    
       'The bind function associates a local address with a socket. Bind the listening socket to the IP address and port.
        RESULT_LONG = bind(BYVAL LISTENING_SOCKET, BYVAL VARPTR(TYPE_SOCKET_ADDRESS),  SIZEOF(TYPE_SOCKET_ADDRESS))
    
       'The listen function places a socket in a state in which it is listening for an incoming connection.
        RESULT_LONG = listen(LISTENING_SOCKET, 5)
    
    
        TEXT_TO_SCREEN "IM WAITING FOR THE LISTENING SOCKET TO BE CONTACTED BY SOMEONE WANTING TO COMMUNICATE"
        ACCEPTING_SOCKET =  WSAAccept(LISTENING_SOCKET, BYVAL VARPTR(TYPE_SOCKET_ADDRESS), SIZEOF(TYPE_SOCKET_ADDRESS), 0, 0)
        TEXT_TO_SCREEN "ACCEPTING_SOCKET " + FORMAT$(WSAGetLastError) + " - "+ FORMAT$(ACCEPTING_SOCKET)
    
        LOCAL PTR_IP_ADDRESS AS BYTE POINTER
        PTR_IP_ADDRESS = VARPTR(TYPE_SOCKET_ADDRESS.sin_addr)
    
        TEXT_TO_SCREEN "CONNECTED IP = " + USING$("#_.#_.#_.#", @PTR_IP_ADDRESS, @PTR_IP_ADDRESS[1], @PTR_IP_ADDRESS[2], @PTR_IP_ADDRESS[3])
    
        MY_WSAOVERLAPPED.hEvent = WSACreateEvent
    
        LOCAL my_pointer AS ASCIIZ POINTER
        LOCAL my_asciiz  AS ASCIIZ *4096
    
        my_pointer = VARPTR(my_asciiz)
    
        wsBuf.dLen = 4096
        wsBuf.buf  = my_pointer
    
    
        RESULT_LONG = WSARecv(ACCEPTING_SOCKET, wsBuf, 1, nBytesReceived, nFlags, MY_WSAOVERLAPPED, 0)
        WSARecv_Error = WSAGetLastError
    
        IF WSARecv_Error = 997 THEN
          TEXT_TO_SCREEN "Error 997 - I think this is good as it means an Overlapped I/O operation is in progress.
         ELSE
          TEXT_TO_SCREEN "Last Error = " + FORMAT$(WSAGetLastError)
        END IF
    
        TEXT_TO_SCREEN FORMAT$(nBytesReceived) + " - " + my_asciiz
    
    
    
       EXIT FUNCTION
    
       ERROR_CONTROL:
       MSGBOX ERROR$  
       END FUNCTION
    '------------------------------------------------------------------------------
     FUNCTION CompletionROUTINE(dwError AS DWORD, cbTransferred AS DWORD, lpOverlapped AS WSAOVERLAPPED, dwFlags AS DWORD)AS LONG
    
       MSGBOX "CompletionROUTINE"
    
     END FUNCTION
    '------------------------------------------------------------------------------
     FUNCTION MY_WSACleanup()AS LONG
    
       WSACleanup
       IF WSACleanup = 0 THEN TEXT_TO_SCREEN "WSACleanup"
    
     END FUNCTION
    '------------------------------------------------------------------------------
     FUNCTION TEXT_TO_SCREEN (TEXT_STRING AS STRING) AS LONG
       LOCAL LISTBOX_COUNT AS LONG
    
       LISTBOX ADD GL_hDlg, %LISTBOX_VIEWER, " " + TEXT_STRING
       LISTBOX GET COUNT GL_hDlg, %LISTBOX_VIEWER TO LISTBOX_COUNT
       CONTROL SEND GL_hDlg, %LISTBOX_VIEWER, %LB_SETTOPINDEX, LISTBOX_COUNT-1, 0
     END FUNCTION
    '------------------------------------------------------------------------------




    Last edited by David Clarke; 11 Jan 2020, 08:48 PM.

  • #2
    Is this working for you?
    Would like to replace SQLiteningServer (and other servers) requiring a thread for each client.
    Is there an include file missing for wsacleanup ... winsock2.inc ?

    Are you using Jose Roca includes?
    wsBuf.dLen = 4096 'missing definition
    How long is an idea? Write it down.

    Comment


    • #3
      I use Multicast, it's ideal for that kind of applications. As much clients as you like and no connection needed.
      Regards,
      Peter

      Comment


      • #4
        Does it work over the internet? I have some Pierre Bellisle code for multicast.
        Another thread on this subject
        https://forum.powerbasic.com/forum/u...ompletion-port
        How long is an idea? Write it down.

        Comment


        • #5
          If the remote location is accessible, then Multicast should work. Mind that some routers have special options to enable multicast.
          Regards,
          Peter

          Comment


          • #6
            I need to come back to this...

            Comment


            • #7
              I'm trying to get my multicast example compiled with the Jose Roca api includes, but the Winsock headers always give you a headache...
              (until now I always used my own include for both Winsock 1 an 2 called "Wsock_x.inc" but I rather use the default headers)
              One of the points is that all %IP_ equates are missing from Winsock2.inc for some reason?

              Or do I have to include more as only "Windows.inc"?

              Code:
              '==================================================================================================
              ' Multicast example - MCAST_WIN.BAS
              '==================================================================================================
              #Compiler PBWin 10
              #Compile Exe
              #Dim All
              #Resource Manifest, 1, "XPTheme.xml"
              #Include "WINDOWS.INC"
              %LB1  = 131 : %LB2  = 132 : %LBL1 = 133 : %LBL2 = 134 : %TXT1 = 135
              
              $HELLO_GROUP    = "233.1.1.10"      'IP address of multicast
              %HELLO_PORT     = 40001            'Port number
              
              Global hDlg As Dword, DataString As String, DataReady As Word
              Global hTxThread, hRxThread, TxPort, RxPort, Running, hEvents() As Long
              Global RxSock, TxSock As Long
              
              '==================================================================================================
              Sub AddToList(ByVal pBox As Word, pTxt As String)
              '--------------------------------------------------------------------------------------------------
                Local lRet As Long
                ListBox Add hDlg, pBox, pTxt
                ListBox Get Count hDlg, pBox To lRet
                ListBox Select hDlg, pBox, lRet
              End Sub
              
              '==================================================================================================
              Thread Function Rx(ByVal pRx As Long)As Long
              '--------------------------------------------------------------------------------------------------
                Local LRet, lLen, lOne, lPort As Long, LSockAddr_in As SockAddr_in, lzRxIp As AsciiZ * 16
                Local lSockAddr As Sockaddr, LMReq As Ip_MReq, RMsg As String * 1024, lzPtr As AsciiZ Ptr
                '------------------------------------------------------------------------------------------------
                RxSock = Socket (%AF_INET,%SOCK_DGRAM,ByVal 0)
                LSockAddr_In.Sin_Family = %AF_INET
                LSockAddr_In.s_Addr     = htonl (%INADDR_ANY)
                LSockAddr_In.Sin_Port   = htons (%HELLO_PORT)
                lOne                    = 1
                LRet                    = SetSockOpt(RxSock, %SOL_SOCKET, %SO_REUSEADDR, lOne, SizeOf(lOne))
                lRet                    = Bind (RxSock, lSockAddr_In, SizeOf (lSockAddr_In))
                '----------------------------------------------------------------------------
                LMReq.Imr_MultiAddress.s_Addr = Inet_Addr ($HELLO_GROUP)
                LMreq.Imr_Interface.s_Addr    = htonl (%INADDR_ANY)
                LRet = SetSockOpt(RxSock, %IPPROTO_IP, %IP_ADD_MEMBERSHIP, LMReq, SizeOf(LMReq))
                '----------------------------------------------------------------------------
                LSockAddr_In.Sin_Family = %AF_INET
                LSockAddr_In.s_Addr     = htonl (%INADDR_ANY)
                LSockAddr_In.Sin_Port   = htons (%HELLO_PORT)
                lPort                   = ntohs(lsockaddr_in.Sin_Port)
                Control Set Text hDlg, %LBL1, "Receiving port: " & $HELLO_GROUP & ":" & Format$(lPort) & "    [ready]"
                While Running
                  rMsg = ""
                  lLen = Len (rMsg)
                  LRet = RecvFrom (RxSock, rMsg, 1024, 0, lSockAddr_in, SizeOf (lSockAddr_in))
                  RxPort = ntohs(lsockaddr_in.Sin_Port)
                  If (LRet > 0) And (TxPort <> RxPort) Then
                    lzPtr = Inet_NtoA (ByVal LSockAddr_In.s_addr)
                    If lzPtr Then lzRxIp = @lzPtr
                    RxPort = ntohs(lsockaddr_in.Sin_Port)
                    If RxPort <> TxPort Then
                    AddToList %LB1, "From: " + lzRxIp & ":" & Format$(RxPort) & ", msg: " & Left$(rMsg,lret)
                    End If
                  End If                                      '
                Wend
                EndCall:
                LRet = SetSockOpt(RxSock, %IPPROTO_IP, %IP_DROP_MEMBERSHIP, LMReq, SizeOf(LMReq))
                If RxSock Then CLoseSocket(RxSock)
                AddToList %LB1, "Rx:Leaving session"
              End Function
              
              '==================================================================================================
              Thread Function Tx(ByVal pTx As Long)As Long
              '--------------------------------------------------------------------------------------------------
                Local LRet As Dword, LWsaData As WsaData, lOne, lTTL As Long, lSockout As SockAddr_In
                Local LDst_SockAddr As SockAddr_In, lLen As Long
                '------------------------------------------------------------------------------------------------
                LRet = WSAStartUp (&h0202,LWsaData)
                LDst_SockAddr.Sin_Family = %AF_INET
                LDst_SockAddr.s_Addr     = htonl (%INADDR_ANY)
                LDst_SockAddr.Sin_port   = 0
                TxSock                   = Socket (%AF_INET,%SOCK_DGRAM,ByVal 0)
              
                lOne = 1
                LRet = SetSockOpt(TxSock, %SOL_SOCKET, %SO_REUSEADDR, lOne, SizeOf(lOne))
                lRet = Bind (TxSock, lDST_SockAddr, SizeOf (LDST_SockAddr))
                lTTL = 255
                lRet = SetSockOpt(TxSock, ByVal %IPPROTO_IP, lTTL, ByRef lTTl , SizeOf(lTTl))
                lTTl = 1
              
                LDst_SockAddr.Sin_Family = %AF_INET
                LDst_SockAddr.s_Addr     = Inet_addr ($HELLO_GROUP)
                LDst_SockAddr.Sin_port   = htons (%HELLO_PORT)
                lRet                     = GetSockName (TxSock, lSockOut, SizeOf(lSockOut))
                TxPort                   = ntohs(lSockOut.Sin_port)
              
                Control Set Text hDlg, %LBL2, "Transmitting port: " & $HELLO_GROUP & ":" & _
                                              Format$(TxPort) & ", socket: " & Format$(TxSock)
                Running = %TRUE
                Thread Create Rx(1) To hRxThread
                DataString = "Joining session..." + Chr$(13)
                lLen = Len (DataString)
                LRet = Sendto (TxSock, ByVal StrPtr(DataString), ByVal lLen,0, LDst_SockAddr, SizeOf (LDst_SockAddr))
                While Running
                  While DataReady = 0
                    Sleep 10
                    If Running = 0 Then Exit Loop
                  Wend
                  If Running Then
                    DataString = DataString + Chr$(13)
                    lLen = Len (DataString)
                    LRet = Sendto (TxSock, ByVal StrPtr(DataString), ByVal lLen,0, LDst_SockAddr, SizeOf (LDst_SockAddr))
                    AddToList %LB2, "msg: " & DataString
                  End If
                  DataReady = 0
                Wend
                DataString = "Leaving session..." + Chr$(13)
                lLen = Len (DataString)
                LRet = Sendto (TxSock, ByVal StrPtr(DataString), ByVal lLen,0, LDst_SockAddr, SizeOf (LDst_SockAddr))
                EndCall:
                If TxSock Then CLoseSocket(TxSock)
              End Function
              
              '==================================================================================================
              Function PBMain () As Long
              '--------------------------------------------------------------------------------------------------
                Dialog New 0, "Multicast chat example",,, 250, 225, %WS_Caption Or %WS_SysMenu, 0 To hDlg
                Control Add ListBox, hDlg, %LB1 ,  , 10, 40, 230, 100, %WS_VScroll Or %WS_TabStop, %WS_Ex_ClientEdge
                Control Add ListBox, hDlg, %LB2 ,  , 10,140,230, 60, %WS_VScroll Or %WS_TabStop, %WS_Ex_ClientEdge
                Control Add TextBox, hDlg, %TXT1,"", 65,203,175, 14, %WS_Border, %WS_Ex_Windowedge
                Control Add Label,   hDlg, %LBL1,"", 10, 10, 220,12
                Control Add Label,   hDlg, %LBL2,"", 10, 22, 220,12
                Control Add Label  , hDlg, -1   ,"Enter text: ", 25,207,40,14
                Control Set Focus hDlg, %TXT1
                Dialog Show Modal hDlg Call DlgProc
              End Function
              
              '==================================================================================================
              CallBack Function DlgProc() As Long
              '--------------------------------------------------------------------------------------------------
                Local ln1, ln2 As Long, hList1, hList2 As Dword, lStr As String
                '------------------------------------------------------------------------------------------------
                Select Case As Long Cb.Msg
                  Case %WM_InitDialog
                    Thread Create Tx(1) To hRxThread
                   SetProcessWorkingSetSize(GetCurrentProcess, -1, -1)
                  Case %WM_Command
                    Select Case CbCtl
                      Case %IdOk
                        Control Get Text hDlg, %TXT1 To DataString
                        Control Set Text hDlg, %TXT1, ""
                        DataReady = 1
                      Case %IdCancel
                        Running = 0
                        Sleep 500
                        Dialog End hDlg
                    End Select
                  Case %WM_Destroy
                    If RxSock Then CLoseSocket(RxSock)
                    If TxSock Then CLoseSocket(TxSock)
                    Running = 0
                    WSACleanup
                    Sleep 500
                    TerminateThread hRxThread,1
                    TerminateThread hTxThread,1
                End Select
              End Function
              Regards,
              Peter

              Comment


              • #8
                Try:
                #INCLUDE "in32api.inc"
                #INCLUDE "ws2tcpip.inc"

                Also you may need:
                LSockAddr_In.s_Addr -> LSockAddr_In.sin_Addr

                Comment


                • #9
                  I then get:

                  Click image for larger version

Name:	error.png
Views:	54
Size:	2.6 KB
ID:	796917
                  Regards,
                  Peter

                  Comment


                  • #10
                    Hmmm,my WS2DEF.INC (José's dated 23 Aug 2015 - 39KB) contains:

                    FUNCTION IN_CLASSD (BYVAL i AS DWORD) AS LONG
                    FUNCTION = -((i AND &HF0000000???) = &HE0000000???)
                    END FUNCTION


                    Are you sure you are ONLY pointing to José's includes? Are you sure that Z:\PB\WINAPI is not the PB includes?.

                    Comment


                    • #11
                      If you want to use Winsock2 with my include files, use #include "WS2tcpip.inc", which in turn includes "winsock2.inc", "ws2ipdef.inc" and "MSTcpIP.inc".

                      The %IP_xxx equates for Winsok2 are inside "ws2ipdef.inc" and have different values that the ones for Winsock.

                      Forum: http://www.jose.it-berater.org/smfforum/index.php

                      Comment

                      Working...
                      X