Announcement

Collapse
No announcement yet.

Threads, performance, response times

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #41
    Originally posted by David Roberts View Post
    In the meantime you guys may like to have a look at this to avoid restarting your machines or writing massive files, Gösta.
    Am having a peculiar problem with your code, David. I C&P'ed, and while it compiles fine, it doesn't display anything on the screen, it doesn't appear to run, just compiles. I tried sprinkling the code with MsgBoxes but none display.

    {just occurred to me} Just chkd the folder and the .exe is there, and runs as advertised. Must be something in your code that causes JellyFish to not run the code after compiling. Have another prog loaded into JF at the same time which compiles and runs as normal. Strange.

    I've been playing around with the code a little and found that clearing the cache the first time takes about 2 seconds but clearing the cache the second time (after reading the file) is effectively instantaneous (.02 if at all).

    At any rate neat code. (Contest material?)
    It's a pretty day. I hope you enjoy it.

    Gösta

    JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
    LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

    Comment


    • #42
      > Must be something in your code that causes JellyFish to not run the code after compiling.

      I have seen this with JellyFish on a few occasions leaving me to run the exe directly. It used to happen on my old Win98 machine as well as my XP Pro machine. If you load the bas into the PB IDE then it should work with 'Run>Compile and Execute'. I can only think that on editing a non-printable character gets embedded that throws JellyFish but not the PB IDE.

      Sometimes, I have rewritten sections of code and deleted the section copied to find that all is well again, removing the offending character without knowing what it was.

      It happens so rarely that I've just accepted it especially since, to my knowledge, nobody had reported this behaviour.

      > At any rate neat code. (Contest material?)

      No, its just a trick and I cannot even prove why it works.

      Comment


      • #43
        > Real Men Don't Need No Stinkin' Handle...Write once, use many....The Power of MACROs.....<insert platitude of choice here>...

        That is exactly what my GetFileSize does but if the system handle is available why use a sledgehammer when three lines of PB does the job?

        Comment


        • #44
          Originally posted by David Roberts View Post
          >
          That is exactly what my GetFileSize does but if the system handle is available why use a sledgehammer when three lines of PB does the job?
          Because, .... errrr.... ahhh .... hrrrrummmmph..... REAL men use the API.

          ================================================================
          "Some editors are failed writers,
          but so are most writers."
          T. S. Eliot (1888-1965)
          ================================================================
          It's a pretty day. I hope you enjoy it.

          Gösta

          JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
          LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

          Comment


          • #45
            I don't believe this - I'm sure I've done it before - I must check.

            A comment following #COMPILE.EXE is getting treated as a compile only in JellyFish when Build and Execute is chosen. I added the comment just before posting.

            Comment


            • #46
              Dave, your comment sparked a memory so I looked Paul Squires JF forum and found this:

              http://planetsquires.com/support/ind...b&topic=1836.0

              It's a quirk in JFP. Can't have a comment on the same line as Compile:

              Compile Exe ' 8.04 '<== won't run after compile


              Compile Exe '<== will run after compile


              ================================================================

              I do not feel obliged to believe
              the same God who has endowed us
              with sense, reason, and intellect
              intended us to forgo their use.
              ~ Galileo Galilei
              ================================================================
              It's a pretty day. I hope you enjoy it.

              Gösta

              JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
              LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

              Comment


              • #47
                Blimey. Thanks Gösta.

                Comment


                • #48
                  Does the Cache have a Cache?

                  Been playing around this morning with the code. I put a Kill in before the first write and added a couple more timers. Here are the results:

                  Writing took = 3.95

                  Clearing First Cache = 1.98
                  First Read = 9.50
                  Clearing Second Cache = 0.02
                  Second Read = 10.08
                  Clearing Third Cache = 0.02

                  Clearing the cache the first time takes pretty consistently 100 times longer than subsequent Clear Caches.

                  Here's a run without the Kill:
                  Writing took = 0.00

                  Clearing First Cache = 1.97
                  First Read = 9.50
                  Clearing Second Cache = 0.00
                  Second Read = 10.25
                  Clearing Third Cache = 0.00
                  Is there a Catch to the Cache?

                  Okay, as I'm writing this it occurs to me the compiler is setting up internal values (or clears memory to work) when it first runs ClearCache and that's what takes the time and it isn't necessary subsequently. Does that make sense?

                  Okay, it doesn't. Here's another run with the first ClearCache remmed:
                  Writing took = 0.00

                  Clearing First Cache = 0.00
                  First Read = 10.17
                  Clearing Second Cache = 0.02
                  Second Read = 9.81
                  Clearing Third Cache = 0.02
                  Really curious. Anyway, if any want to try it themselves here's my David's code with my junk added:


                  '
                  Code:
                  'http://www.powerbasic.com/support/pbforums/showthread.php?t=35260
                  #Dim      All
                  #Compile  Exe 
                  #Register None
                  #Tools    Off
                  %NOMMIDS = 1
                  %NOGDI = 1
                  #Include "WIN32API.inc" ' 1 November 2006
                  '#Include "C:\Power Basic\Includes\Clear_File_Cache.inc"
                  '#Include "C:\Power Basic\Includes\clipboard.inc"
                  $sFile = "c:\test"
                  '         
                  'Clipboard stuff gotten from Poffs but I forget from whom
                  Function Clipboard_Set_Text Alias "Clipboard_Set_Text" _
                                (ByVal sText As String) Export As Long
                    Local hData As Long, hGlob As Long
                  ' ** Create a global memory object and copy the data into it
                    hData = GlobalAlloc(%GMEM_MOVEABLE Or %GMEM_DDESHARE, Len(sText) + 1)
                    hGlob = GlobalLock(hData)
                    Poke$ hGlob, sText + Chr$(0)
                    GlobalUnlock hData
                  ' ** Open the clipboard
                    If IsFalse(OpenClipboard(%Null)) Then
                      GlobalFree hData
                      Exit Function
                    End If
                  ' ** Paste the data into the clipboard
                    EmptyClipboard  'WinAPI
                    Function = SetClipboardData(%CF_TEXT, hData) 'WinAPI
                    CloseClipboard  'WinAPI
                  End Function
                  Function Clipboard_Get_Text Alias "Clipboard_Get_Text"() Export As String
                      Local zPtr As Asciiz Ptr
                      OpenClipboard 0 'WinAPI
                      zPtr = GetClipboardData(%CF_TEXT)
                      If zPtr <> 0 Then Function = @zPtr
                      CloseClipboard   'WinAPI
                  End Function
                  '                   
                  '         
                  ' '             
                  Sub ClearFileCache(sFile As String , Buffer As String)
                     Local hFile, Chunk, nbLoaded As Long, ptrBuffer As Dword Ptr
                     Dim zFile As Asciiz * 256
                   
                     zFile = sFile
                     hFile = CreateFile(zFile, %GENERIC_READ, 0, ByVal 0, %OPEN_EXISTING, %FILE_FLAG_SEQUENTIAL_SCAN Or %FILE_FLAG_NO_BUFFERING, ByVal 0)
                     Chunk = Len(Buffer)
                     ptrBuffer = StrPtr(Buffer)
                     Do
                       ReadFile( hFile, ByVal ptrBuffer, ByVal Chunk, nbLoaded, ByVal 0 )
                     Loop Until nbLoaded = 0
                     CloseHandle(hFile)
                  End Sub
                   '                   
                  Function PBMain( ) As Long
                    Dim strNull As String
                    Local i As Long, t1, t2 As Ext
                    Local Clr_Cache01, Clr_Cache02, Clr_Cache03, Write_01 As Ext
                    strNull = Nul$(1048576) ' 1 MB
                   
                  '  Kill $sfile 'just to test
                    Write_01 = Timer
                    If Dir$($SFile) = "" Then
                      ' Write a 100MB file
                      Open $sFile For Binary As #1
                      For i = 1 To 100
                        Put$ #1, strNull
                      Next
                      Close #1                   
                      ?"Wrote " & $sfile
                    End If          
                    Write_01 = Timer - Write_01
                   
                    ' Clear filecache after creation, if done, 
                    ' And before before first Read
                   
                  ?"Clearing original Cache"
                   Clr_Cache01 = Timer
                    ClearFileCache($sFile, strNull)  
                  '? Using$("Clearing any existing cache took #.## seconds", Timer - t1)  
                   Clr_Cache01 = Timer - Clr_Cache01
                   
                    ' Read file
                    t1 = Timer
                    t1 = Timer
                    Open $sFile For Binary As #1
                    For i = 1 To 100
                      Get$ #1, 10248576, strNull
                    Next
                    Close #1
                    t1 = Timer - t1
                   ?"Clearing Read cache"
                   
                    ' Clear file cache after first reading and before second read
                   Clr_Cache02 = Timer
                    ClearFileCache($sFile, strNull)
                    Clr_Cache02 = Timer - Clr_Cache02
                  '? Using$("Clearing Read Cache took #.## seconds", Timer - t3)  
                   
                    ' Read file again
                    t2 = Timer
                    Open $sFile For Binary As #1
                    For i = 1 To 100
                      Get$ #1, 10248576, strNull
                    Next
                    Close #1
                    t2 = Timer - t2
                   Clr_Cache03 = Timer
                    ClearFileCache($sFile, strNull)
                    Clr_Cache03 = Timer - Clr_Cache03
                   
                    Local m$
                   
                      M$ = Using$(" Writing took = ##.##", Write_01) + $CrLf + $CrLf  + _
                           Using$(" Clearing First Cache = ##.##", Clr_Cache01) & $CrLf & _
                           Using$(" First Read = ##.##",t1) +  $CrLf  + _
                           Using$(" Clearing Second Cache = ##.##", Clr_Cache02) & $CrLf & _
                           Using$(" Second Read = ##.##",t2) & $CrLf & _ 
                           Using$(" Clearing Third Cache = ##.##", Clr_Cache03)
                      ? m$
                      Clipboard_Set_Text(m$)     
                   End Function
                   '
                  It's a pretty day. I hope you enjoy it.

                  Gösta

                  JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
                  LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

                  Comment


                  • #49
                    Gösta, I found a typo in my code. A 2 crept in on reading the files. The first was a typo and then the error was duplicated on a 'Copy and Paste'. The 10248576 should read 1048576.

                    The moral here, of course, and I should know better after goodness knows how long is not to use use numbers in code but use a declaration of some sort. Of course when testing something it is very easy to slip into bad practice. We shouldn't, but I guess that I may not be the only one. Your relative observations may not change, just the absolute timings. I'll have a look at what you are doing.

                    Comment


                    • #50
                      I've added a new version of ClearFileCache in the Source Code forum.

                      Comment


                      • #51
                        Sorry, Gösta.

                        > Your relative observations may not change, just the absolute timings.

                        Isn't true. I couldn't figure out why you were getting such small values for second and subsequent filecache clearings. They should be all the same, or thereabouts, since ClearFileCache is reading the file.

                        When I removed the offending 2 your timings made more sense.

                        I thought only the GET timings would be affected.

                        I'll check out that error and see what is going on 'under the bonnet'.

                        Comment


                        • #52
                          Readings after correcting the code (removing the 2):

                          Code:
                           Writing took =  4.38
                           
                           Clearing First Cache =  2.06
                           First Read =  2.52
                           Clearing Second Cache =  1.95
                           Second Read =  2.53
                           Clearing Third Cache =  1.95
                          All is copesetic again PB Land.

                          ================================================================
                          "Every time I see an adult on a bicycle,
                          I no longer despair
                          for the future of the human race."
                          H.G. Wells
                          ================================================================
                          It's a pretty day. I hope you enjoy it.

                          Gösta

                          JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
                          LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

                          Comment


                          • #53
                            Breathes sigh of relief.

                            Still cannot figure out why ClearFilecache came beck so quickly on the second outing with the 2 error. I introduced the same error to ClearFileCache2 and the cache clearing took just as long, that is it behaved as expected. I'll leave it for now and look again - might spot something then. The reason may not be worth knowing but I won't know that until I find it.

                            Comment


                            • #54
                              Got it.

                              With the 2 error very early on in the FOR/NEXT LOOP there is nothing left of the file to read so strNull becomes empty. ClearFileCache gets an empty buffer and so has an early bath. In fact there is no need for ClearFileCache to use the same buffer - my logic is adrift there.

                              Funny thing is that on introducing error checking on GET I don't get one - should have got an end of file error. Must look into that now. Blast it!

                              With ClearFileCache2 it uses its own buffer and has no regard for how PBMain collects data so doesn't 'fall over'.

                              Actually, I think I'll remove ClearFileCache from the Source Code forum - the idea is OK but it was poorly implemented.

                              Comment


                              • #55
                                This exercise seemed like a good way to learn more about thread performance. From the code provided by Paul & Chris, I created three different code modules, but only created 1 test file with 10,000,000 records so the testing would push the processing time in each section further and each module would be using the same data:
                                Code:
                                 Record_Count = 10,000,000
                                 
                                ------------------ Paul's Code #1 - PBCC 4.04
                                    No Thread = 17.266
                                  With Thread = 10.406
                                         Diff =  6.86
                                     Increase = 39.73%
                                ------------------ Chris' Code #1 - PBWin 8.04
                                    No Thread = 18.594
                                  With Thread = 10.688
                                         Diff =  7.906
                                     Increase = 42.52%
                                ------------------ Paul's Code #2 - PBCC 4.04
                                    No Thread = 18.696
                                  With Thread =  9.064
                                         Diff =  9.632
                                     Increase = 51.52%
                                ------------------ END
                                 
                                ------------------ Computer Hardware & OS - CPU = XEON x 2 :
                                OS Name	Microsoft Windows XP Professional
                                Version	5.1.2600 Service Pack 2 Build 2600
                                OS Manufacturer	Microsoft Corporation
                                System Name	R9
                                System Manufacturer	ASUSTeK COMPUTER INC.
                                System Model	NCCH-DL
                                System Type	X86-based PC
                                Processor	x86 Family 15 Model 3 Stepping 4 GenuineIntel ~3200 Mhz
                                Processor	x86 Family 15 Model 3 Stepping 4 GenuineIntel ~3200 Mhz
                                Processor	x86 Family 15 Model 3 Stepping 4 GenuineIntel ~3200 Mhz
                                Processor	x86 Family 15 Model 3 Stepping 4 GenuineIntel ~3200 Mhz
                                BIOS Version/Date	Phoenix Technologies, LTD ASUS NCCH-DL ACPI BIOS Revision 1003, 10/1/2004
                                SMBIOS Version	2.3
                                Windows Directory	C:\WINDOWS
                                System Directory	C:\WINDOWS\system32
                                Boot Device	\Device\HarddiskVolume2
                                Locale	United States
                                Hardware Abstraction Layer	Version = "5.1.2600.2180 (xpsp_sp2_rtm.040803-2158)"
                                User Name	R9\Roger
                                Time Zone	Pacific Daylight Time
                                Total Physical Memory	2,048.00 MB
                                Available Physical Memory	1.33 GB
                                Total Virtual Memory	2.00 GB
                                Available Virtual Memory	1.96 GB
                                Page File Space	3.85 GB
                                Page File	C:\pagefile.sys
                                It seems that some threading can provide a lot of performance if the machine executing the threads has the capability to handle the work. In the test cases given in this posting and generated on my machine, it sure seems likes threads can save a lot of time because of their ability to better utilize the hardware. This performance is improved even more when the adjustments Paul made in his second code posting are employed.
                                Roger...

                                Comment

                                Working...
                                X