Announcement

Collapse
No announcement yet.

Replace-command continued...

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Replace-command continued...

    As the activity in the 'old' thread has died, I just want to say, to all of
    you, who has responded, and probably were offended by some of my replies,
    that troubleshooting is a difficult and precise art.
    It is not easy to help without seeing all the facts,
    I take the reality as it is. I can not 'make up' disturbing events, or use
    nonexistent code to explain how my code works or why it might fail.
    '--The Program------------------------------------------
    This is intended to be a high speed/high capacity worker.
    Therefore there is only ONE possible instance of the program on a physical pc.
    There is only ONE possible thread per fileoperation as every file is read from BOF to EOF
    and every byte (and byte position) is significant, and you cannot do more than one
    action at the time,(if duplicating a file you write the same data to more than one file
    but that is consecutive Put$-instructions)
    Normally this pc-s are in a locked room and rarely accessed by a human hand
    There are 5 diffrent DLL-s in my converion-suit, They all heavyly relay on the
    Replace-instruction. It is around 2000 files per day that is manipulated.
    All this five DLL-s is bulid in a similar manner. (There is not many ways to read
    a file from BOF to EOF.)
    All fails in the same way. Partly translated buffers. But it is only in two of this
    DLL-s I can detect it, when it happens.
    '--The Problem-------------------------------------------
    In this particular case it is a single Replace-instruction that fails.
    Replace any ASCII with EBCDIC in Buffer$
    If Buffer after this instruction is part EBCDIC and part ASCII. That can only
    mean that replace-instruction is not carried out to the end.
    This is described by me as a failing Replace-instruction.
    Many suggestions around this has been "it is not Replace that is failing it is
    something else, perhaps other strings involved are changed.."
    I wrote "I have never seen a trashed buffer" that means: The characters in the
    buffer are all valid (and correct placed) Ebcdic-characters or original ASCII-characters.
    Powerbasic staff can verify that if this happened (EBCDIC and ASCII strings were not the
    same size etc, would have produced a 'very easy to identify' pattern in the Buffer$
    '--The conclusion----------------------------------------
    Whatever is causing Replace to end prematurely it is the responsibility of compiler-
    generated code to handle that situation.
    My code is not involved in any string-housekeeping on behalf of Replace-instruction.
    My code is not calling any system dll to perform this instruction. If that sort of
    function is performed, and the compiler generate that sort of code, then this code
    should also be responsible for checking that it is correctly executed.
    '--The '****ing up'--------------------------------------
    I will never be able to write code that to 100% will run on every single computer in
    the world. I don't expect Powerbasic to write a bug free-compiler either.
    But I do expect Powerbasic to give me a correct answer on a direct question
    '---
    This is the direct question to [email protected]
    Is there any way to detect if Replace Any fails?
    What value in Err-variable?
    What value in ErrApi-variable?
    '--The final plea---------------------------------------
    I have taken a quick look att Semen's replacement for Replace-instruction.
    I am also making an inventory of Replace-commands in all my 'sensitive' code.
    It is unbeliveable how often it is used..
    I have to benchmark it as well to verify if Semens result is applicable to 'my'
    environment.
    '--New facts from the test-line-------------------------
    All NT4-units have been taken out of production and been replaced by WIN98SE
    For 5 days this WIN98-units have been working without one single error (normal)
    '--New testequipment------------------------------------
    One new fresh NT4 has been installed with SP4 and run for 2 hrs.
    First it was installed with Novell latest Client 4.7
    Just one testfile have been feed through Translated-retranslated-Translated-retranslated etc.
    The same file fails not everytime, but about every third translation.
    That means that 2 seperate DLL-s is failing in the same sort of code.
    One of those is the ASA_To_PCC-DLL you have seen, the other one is PCC_To_ASA-DLL.
    '--
    Then Novell client was replaced by Microsoft client
    Same result as with Novell client
    '--Other installed programs in NT-----------------------
    Hedit Pro and TextPad4 InterNet Explorer 4.01
    Nothing else
    '--What to do next--------------------------------------
    Rewrite Replace-instruction (thank you Semen..) and test again



    -------------
    Fred
    mailto:[email protected][email protected]</A>
    http://www.oxenby.se

    Fred
    mailto:[email protected][email protected]</A>
    http://www.oxenby.se

  • #2
    For the record,
    Powerbasic support (Thank you Lance) have promptly replied on
    my question.
    Unfortunatly I had submitted the text before I got the mail.
    Sorry Lance

    ------------------
    Fred
    mailto:[email protected][email protected]</A>
    http://www.oxenby.se

    Fred
    mailto:[email protected][email protected]</A>
    http://www.oxenby.se

    Comment


    • #3
      Code:
      I have modified Semens code for a Replace Any-replacment and turned 
      it into six fixed translation-routines
      ASCII->ANSI,ASCII->EBCDIC,ANSI->ASCII,ANSI->EBCDIC,EBCDIC->ASCII,EBCDIC->ANSI
      I think this will be faster, and keep OLE out of the picture
      This code reload the translation table every time it is executed, but I found
      during testing, that using static table did not make it noticeble faster.
      --Testfile---------------------------------------
        on local drive, 130 MB in size. Testresult include reading the file
      --Testresult PIII 500 Win2000--------------------
        BufferSize 8 kb
        Replace any    2:57
        XLTASC2EBC     2:49
        BufferSize 100 kB
        Replace Any    1:48
        XLTASC2EBC     0:17    (incl. reading 130 MB from disk?)
      --Testresult PII 400 Win98SE---------------------
        Buffersize 8 kB
        Replace Any    4:00
        XLTASC2EBC     2:57
        Buffersize 100 kB
        Replace Any    3:15
        XLTASC2EBC     0:48
        
      Just to be sure we did not have any caching disturbing the result I
      reversed the order of test-sequence but the result was exactly the same
              
      Sub XLTASC2EBC(ByRef S$)Export
      Local T() As Byte:ReDim T(0 To 255)
      Local P1 As Byte Ptr
      Register i As Long
      '--Assign translationtable-----------------------    
       T(000)=000:T(001)=001:T(002)=002:T(003)=003:T(004)=055:T(005)=045:T(006)=046:T(007)=047
       T(008)=022:T(009)=005:T(010)=010:T(011)=011:T(012)=012:T(013)=013:T(014)=014:T(015)=015
       T(016)=016:T(017)=017:T(018)=018:T(019)=019:T(020)=060:T(021)=061:T(022)=050:T(023)=038
       T(024)=024:T(025)=025:T(026)=063:T(027)=039:T(028)=028:T(029)=029:T(030)=030:T(031)=031
       T(032)=064:T(033)=079:T(034)=127:T(035)=099:T(036)=103:T(037)=108:T(038)=080:T(039)=125
       T(040)=077:T(041)=093:T(042)=092:T(043)=078:T(044)=107:T(045)=096:T(046)=075:T(047)=097
       T(048)=240:T(049)=241:T(050)=242:T(051)=243:T(052)=244:T(053)=245:T(054)=246:T(055)=247
       T(056)=248:T(057)=249:T(058)=122:T(059)=094:T(060)=076:T(061)=126:T(062)=110:T(063)=111
       T(064)=236:T(065)=193:T(066)=194:T(067)=195:T(068)=196:T(069)=197:T(070)=198:T(071)=199
       T(072)=200:T(073)=201:T(074)=209:T(075)=210:T(076)=211:T(077)=212:T(078)=213:T(079)=214
       T(080)=215:T(081)=216:T(082)=217:T(083)=226:T(084)=227:T(085)=228:T(086)=229:T(087)=230
       T(088)=231:T(089)=232:T(090)=233:T(091)=181:T(092)=113:T(093)=159:T(094)=095:T(095)=109
       T(096)=081:T(097)=129:T(098)=130:T(099)=131:T(100)=132:T(101)=133:T(102)=134:T(103)=135
       T(104)=136:T(105)=137:T(106)=145:T(107)=146:T(108)=147:T(109)=148:T(110)=149:T(111)=150
       T(112)=151:T(113)=152:T(114)=153:T(115)=162:T(116)=163:T(117)=164:T(118)=165:T(119)=166
       T(120)=167:T(121)=168:T(122)=169:T(123)=065:T(124)=187:T(125)=071:T(126)=220:T(127)=255
       T(128)=104:T(129)=161:T(130)=121:T(131)=066:T(132)=192:T(133)=068:T(134)=208:T(135)=072
       T(136)=082:T(137)=083:T(138)=084:T(139)=087:T(140)=086:T(141)=088:T(142)=123:T(143)=091
       T(144)=224:T(145)=156:T(146)=158:T(147)=203:T(148)=106:T(149)=205:T(150)=219:T(151)=221
       T(152)=223:T(153)=124:T(154)=252:T(155)=112:T(156)=177:T(157)=128:T(158)=191:T(159)=007
       T(160)=069:T(161)=085:T(162)=206:T(163)=222:T(164)=073:T(165)=105:T(166)=154:T(167)=155
       T(168)=171:T(169)=175:T(170)=186:T(171)=184:T(172)=183:T(173)=170:T(174)=138:T(175)=139
       T(176)=043:T(177)=044:T(178)=009:T(179)=033:T(180)=040:T(181)=101:T(182)=098:T(183)=100
       T(184)=180:T(185)=056:T(186)=049:T(187)=052:T(188)=051:T(189)=176:T(190)=178:T(191)=036
       T(192)=034:T(193)=023:T(194)=041:T(195)=006:T(196)=032:T(197)=042:T(198)=070:T(199)=102
       T(200)=026:T(201)=053:T(202)=008:T(203)=057:T(204)=054:T(205)=048:T(206)=058:T(207)=090
       T(208)=140:T(209)=172:T(210)=114:T(211)=115:T(212)=116:T(213)=037:T(214)=117:T(215)=118
       T(216)=119:T(217)=035:T(218)=021:T(219)=020:T(220)=004:T(221)=204:T(222)=120:T(223)=059
       T(224)=238:T(225)=089:T(226)=235:T(227)=237:T(228)=207:T(229)=239:T(230)=160:T(231)=142
       T(232)=174:T(233)=254:T(234)=251:T(235)=253:T(236)=141:T(237)=173:T(238)=188:T(239)=190
       T(240)=202:T(241)=143:T(242)=027:T(243)=185:T(244)=182:T(245)=067:T(246)=225:T(247)=157
       T(248)=144:T(249)=189:T(250)=179:T(251)=218:T(252)=250:T(253)=234:T(254)=062:T(255)=074
      '--Translate the string--------------------------
       p1 = StrPtr(S$)
       For i = 0 To Len(S$) - 1: @p1[i] = T(@p1[i]): Next i
      End Sub
      ------------------
      Fred
      mailto:[email protected][email protected]</A>
      http://www.oxenby.se

      Fred
      mailto:[email protected][email protected]</A>
      http://www.oxenby.se

      Comment


      • #4
        Fred --
        It's better to include "static" arrays as DB statements.
        To receive address of array you simply use CodePtr(Label).

        In my training code (see bellow) you can find Translate and TranslatePb subs.

        I should notice, that your seriously decrease effect, because
        1) your timing includes reading of file.
        2) call DLL function requires a lot of time. "A lot", comparing with time of execution "translate" function.

        It's interesting to compare clean statements: Replace Any against my sub.

        To have more correct results I decided to generate "random" string.

        Timing on my PC (Win2000, P-III-550):
        Replace Any (Pb) - 15.43 sec
        My ReplaceAny - 0.28 sec
        Translate - 0.12 sec (thanks, Steve for LOOP)
        TranslatePb - 0.29 sec

        120 : 1 !!!

        Code:
            
           #Compile Exe
           #Register None
           #Dim All
           #Include "Win32Api.Inc"
           '==========================================================
           Sub ReplaceAny (Src As String, Dst As String, Tr As String)
              Static Tabl() As Byte, f As Byte
              Register i As Long
              If f = 0 Then
                 ReDim Tabl(0 To 255): f = 1
                 Static p1 As Byte Ptr, p2 As Byte Ptr
                 For i = 0 To 255: Tabl(i) = i: Next
              End If
              p1 = StrPtr(Src): p2 = StrPtr(Dst)
              For i = 0 To Min(Len(Src), Len(Dst)) - 1
                 Tabl(@p1[i]) = @p2[i]
              Next
              p1 = StrPtr(Tr)
              For i = 0 To Len(Tr) - 1: @p1[i] = Tabl(@p1[i]): Next
           End Sub
           '==========================================================
           Sub Translate (Tr As String)
              Register i As Long, j As Long
              i = Len(Tr): j = StrPtr(Tr)
            ! LEA EBX, TrtTable
              ! MOV ECX, i
              ! MOV EDX, j
              ! ADD ECX, EDX
           LbTrt:
              ! MOV AL, [EDX]
              ! XLAT
              ! MOV [EDX], AL
              ! INC EDX
              ! CMP EDX, ECX
              ! JNE LbTrt       
              Exit Sub
         TrtTable:
              ! Db 000,001,002,003,055,045,046,047,022,005,010,011,012,013,014,015
              ! Db 016,017,018,019,060,061,050,038,024,025,063,039,028,029,030,031
              ! Db 064,079,127,099,103,108,080,125,077,093,092,078,107,096,075,097
              ! Db 240,241,242,243,244,245,246,247,248,249,122,094,076,126,110,111
              ! Db 236,193,194,195,196,197,198,199,200,201,209,210,211,212,213,214
              ! Db 215,216,217,226,227,228,229,230,231,232,233,181,113,159,095,109
              ! Db 081,129,130,131,132,133,134,135,136,137,145,146,147,148,149,150
              ! Db 151,152,153,162,163,164,165,166,167,168,169,065,187,071,220,255
              ! Db 104,161,121,066,192,068,208,072,082,083,084,087,086,088,123,091
              ! Db 224,156,158,203,106,205,219,221,223,124,252,112,177,128,191,007
              ! Db 069,085,206,222,073,105,154,155,171,175,186,184,183,170,138,139
              ! Db 043,044,009,033,040,101,098,100,180,056,049,052,051,176,178,036
              ! Db 034,023,041,006,032,042,070,102,026,053,008,057,054,048,058,090
              ! Db 140,172,114,115,116,037,117,118,119,035,021,020,004,204,120,059
              ! Db 238,089,235,237,207,239,160,142,174,254,251,253,141,173,188,190
              ! Db 202,143,027,185,182,067,225,157,144,189,179,218,250,234,062,074
        
           End Sub
           
           '======================================================================
           Sub TranslatePb (Tr As String)
              Register i As Long
              Static p1 As Byte Ptr, p2 As Byte Ptr
              p1 = StrPtr(Tr): p2 = CodePtr(TrtTablePb)
              For i = 0 To Len(Tr) - 1: @p1[i] = @p2[@p1[i]]: Next
              Exit Sub
         TrtTablePb:
              ! Db 000,001,002,003,055,045,046,047,022,005,010,011,012,013,014,015
              ! Db 016,017,018,019,060,061,050,038,024,025,063,039,028,029,030,031
              ! Db 064,079,127,099,103,108,080,125,077,093,092,078,107,096,075,097
              ! Db 240,241,242,243,244,245,246,247,248,249,122,094,076,126,110,111
              ! Db 236,193,194,195,196,197,198,199,200,201,209,210,211,212,213,214
              ! Db 215,216,217,226,227,228,229,230,231,232,233,181,113,159,095,109
              ! Db 081,129,130,131,132,133,134,135,136,137,145,146,147,148,149,150
              ! Db 151,152,153,162,163,164,165,166,167,168,169,065,187,071,220,255
              ! Db 104,161,121,066,192,068,208,072,082,083,084,087,086,088,123,091
              ! Db 224,156,158,203,106,205,219,221,223,124,252,112,177,128,191,007
              ! Db 069,085,206,222,073,105,154,155,171,175,186,184,183,170,138,139
              ! Db 043,044,009,033,040,101,098,100,180,056,049,052,051,176,178,036
              ! Db 034,023,041,006,032,042,070,102,026,053,008,057,054,048,058,090
              ! Db 140,172,114,115,116,037,117,118,119,035,021,020,004,204,120,059
              ! Db 238,089,235,237,207,239,160,142,174,254,251,253,141,173,188,190
              ! Db 202,143,027,185,182,067,225,157,144,189,179,218,250,234,062,074
        
           End Sub
        
        
           Function PbMain ()
              #Register Default
              Dim glbEBCDIC As String, glbASCII As String, SJobC As String
              glbEBCDIC = Chr$(000,001,002,003,055,045,046,047,022,005,010,011,012,013,014,015, _
                               016,017,018,019,060,061,050,038,024,025,063,039,028,029,030,031, _
                               064,079,127,099,103,108,080,125,077,093,092,078,107,096,075,097, _
                               240,241,242,243,244,245,246,247,248,249,122,094,076,126,110,111, _
                               236,193,194,195,196,197,198,199,200,201,209,210,211,212,213,214, _
                               215,216,217,226,227,228,229,230,231,232,233,181,113,159,095,109, _
                               081,129,130,131,132,133,134,135,136,137,145,146,147,148,149,150, _
                               151,152,153,162,163,164,165,166,167,168,169,065,187,071,220,255, _
                               104,161,121,066,192,068,208,072,082,083,084,087,086,088,123,091, _
                               224,156,158,203,106,205,219,221,223,124,252,112,177,128,191,007, _
                               069,085,206,222,073,105,154,155,171,175,186,184,183,170,138,139, _
                               043,044,009,033,040,101,098,100,180,056,049,052,051,176,178,036, _
                               034,023,041,006,032,042,070,102,026,053,008,057,054,048,058,090, _
                               140,172,114,115,116,037,117,118,119,035,021,020,004,204,120,059, _
                               238,089,235,237,207,239,160,142,174,254,251,253,141,173,188,190, _
                               202,143,027,185,182,067,225,157,144,189,179,218,250,234,062,074)
              '--ASCII teckenuppsättning----
              glbASCII =  Chr$(000,001,002,003,004,005,006,007,008,009,010,011,012,013,014,015, _
                               016,017,018,019,020,021,022,023,024,025,026,027,028,029,030,031, _
                               032,033,034,035,036,037,038,039,040,041,042,043,044,045,046,047, _
                               048,049,050,051,052,053,054,055,056,057,058,059,060,061,062,063, _
                               064,065,066,067,068,069,070,071,072,073,074,075,076,077,078,079, _
                               080,081,082,083,084,085,086,087,088,089,090,091,092,093,094,095, _
                               096,097,098,099,100,101,102,103,104,105,106,107,108,109,110,111, _
                               112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127, _
                               128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143, _
                               144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159, _
                               160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175, _
                               176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191, _
                               192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207, _
                               208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223, _
                               224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239, _
                               240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255)
              Dim SJob1 As String, sJob2 As String, SJobCt As String, i As Long, j As Long
              Dim t1 As Double, t2 As Double, t3 As Double, t4 As Double
              %TestLen = 15000: %nTest = 1000
              sJobCT = Space$(%TestLen)
              j = Fix(Timer) Mod 256
              For i = 1 To %TestLen
                 Mid$(sJobCT, i, 1) = Chr$(j)
                 j = ((j * 11) + 7) Mod 256
              Next
              sJob1 = sJobCT: sJob2 = sJobCT
              ' Test for my function
              Lset sJob1 = sJobCT: Replace Any glbAscii With glbEbcdic In SJob1
              Lset sJob2 = sJobCT: ReplaceAny glbAscii, glbEbcdic, SJob2
              If sJob1 <> sJob2 Then MsgBox "Incorrect 1"
              Lset sJob1 = sJobCT: Translate SJob1
              If sJob1 <> sJob2 Then MsgBox "Incorrect 2"
              Lset sJob2 = sJobCT: TranslatePb SJob2
              If sJob1 <> sJob2 Then MsgBox "Incorrect 3"
              t1# = Timer: For i = 1 To %nTest: Lset sJob1 = sJobCT: Next: t2# = Timer
              t3# = Timer
              For i = 1 To %nTest
                 Lset sJob1 = sJobCT: Replace Any glbAscii With glbEbcdic In SJob1
              Next
              t4# = Timer
              MsgBox Format$((t4# - t3#) - (t2# - t1#), "#.00"), , "Replace Any"
              t3# = Timer
              For i = 1 To %nTest
                 Lset sJob2 = sJobCT: ReplaceAny glbAscii, glbEbcdic, SJob2
              Next
              t4# = Timer
              MsgBox Format$((t4# - t3#) - (t2# - t1#), "#.00"), , "My ReplaceAny"
              t3# = Timer
              For i = 1 To %nTest
                 Lset sJob1 = sJobCT: Translate SJob1
              Next
              t4# = Timer
              MsgBox Format$((t4# - t3#) - (t2# - t1#), "#.00"), , "My Translate"
              t3# = Timer
              For i = 1 To %nTest
                 Lset sJob2 = sJobCT: TranslatePb SJob2
              Next
              t4# = Timer
              MsgBox Format$((t4# - t3#) - (t2# - t1#), "#.00"), , "My TranslatePb"
        End Function
        [This message has been edited by Semen Matusovski (edited April 16, 2000).]

        Comment


        • #5
          Fred,

          If I understand the original task you are doing, it involves loading a
          file across a network that is in EBCDIC format, converting it on the fly
          to ASCII and writing it to disk. I have a task coming up in the encryption
          design I am doing at the moment which needs to do a similar thing to the
          conversion you are doing.

          The following sub has a string handle passed to it by reference and the
          length of the string and it directly modifies the string at its address.

          It has been written as a sub to avoid any assignment as it does not need
          to be done. On this box, it will run a 75 meg test string in about 880 ms
          so the actual processing speed is not far off 100 meg/sec.

          I copied the table from the code that Semen posted as I have no reference
          material for EBCDIC but I am sure you have the correct table conversions.

          If this code will do what you need, it means you can get the block size
          you need from the network, crunch it with the sub and write it directly
          to disk. I remember you tested what was the best size for a block but if
          you can get it in big sections, the processing time will drop some more.

          Regards,

          [email protected]

          Code:
            ' #########################################################################
            
            SUB ConvertIt(txt$,ByVal ln as LONG)
            
                #REGISTER NONE
            
                LOCAL lpTxt as LONG
                LOCAL lpTbl as LONG
            
                ! jmp PastIt
            
                eTable:
            
                  ! db 000,001,002,003,055,045,046,047,022,005,010,011,012,013,014,015
                  ! db 016,017,018,019,060,061,050,038,024,025,063,039,028,029,030,031
                  ! db 064,079,127,099,103,108,080,125,077,093,092,078,107,096,075,097
                  ! db 240,241,242,243,244,245,246,247,248,249,122,094,076,126,110,111
                  ! db 236,193,194,195,196,197,198,199,200,201,209,210,211,212,213,214
                  ! db 215,216,217,226,227,228,229,230,231,232,233,181,113,159,095,109
                  ! db 081,129,130,131,132,133,134,135,136,137,145,146,147,148,149,150
                  ! db 151,152,153,162,163,164,165,166,167,168,169,065,187,071,220,255
                  ! db 104,161,121,066,192,068,208,072,082,083,084,087,086,088,123,091
                  ! db 224,156,158,203,106,205,219,221,223,124,252,112,177,128,191,007
                  ! db 069,085,206,222,073,105,154,155,171,175,186,184,183,170,138,139
                  ! db 043,044,009,033,040,101,098,100,180,056,049,052,051,176,178,036
                  ! db 034,023,041,006,032,042,070,102,026,053,008,057,054,048,058,090
                  ! db 140,172,114,115,116,037,117,118,119,035,021,020,004,204,120,059
                  ! db 238,089,235,237,207,239,160,142,174,254,251,253,141,173,188,190
                  ! db 202,143,027,185,182,067,225,157,144,189,179,218,250,234,062,074
            
                PastIt:
            
                lpTbl = CodePtr(eTable)
                
                lpTxt = StrPtr(txt$)  ' source address
            
                  ! mov ebx, lpTbl    ; table address
                  ! mov esi, lpTxt
                  ! mov edi, lpTxt
                  ! mov ecx, esi      ; copy esi into ecx
                  ! add ecx, ln       ; add file length to ecx
                xlSt:
                  ! mov al,[esi]
                  ! inc esi
                  ! xlatb
                  ! mov [edi], al
                  ! inc edi
                  ! cmp esi, ecx
                  ! jne xlSt
            
            END SUB
            
            ' #########################################################################
          ------------------
          hutch at movsd dot com
          The MASM Forum

          www.masm32.com

          Comment


          • #6
            Steve --
            ConvertIt - 0.12 sec against Translate - 0.20 sec (PB Replace Any - 15.43 sec).
            Cool !!!
            PS. Interesting, what does PB Replace Any ? (calculates stars on the sky ?)

            [This message has been edited by Semen Matusovski (edited April 15, 2000).]

            Comment


            • #7
              Interesting discussions folks.

              Just to be clear, we (Tech Support) have still not received any code that duplicates the REPLACE [ANY] problem Fred has described.

              Has anyone else been able to duplicate this problem, or...?

              ------------------
              Lance
              PowerBASIC Support
              mailto:[email protected][email protected]</A>
              Lance
              mailto:[email protected]

              Comment


              • #8
                Lance,

                I seriously doubt that you will get an example, I use the direct
                Replace command to do some reasonably crude things, one of them is
                to do 96 replacements of HTML notation when I convert HTML to text
                as I found it a lot faster than trying to determine what was in the
                string first.

                It is fast enough to do brute force replacements rather than mess
                around with test / replace cycles.

                [email protected]

                ------------------
                hutch at movsd dot com
                The MASM Forum

                www.masm32.com

                Comment


                • #9
                  Semen,

                  Just a suggestion which I think will make your loop a lot faster,
                  the "LOOP" instruction is a very slow one on post 386 machines
                  and you usually get faster code by expanding it to a compare/jump.
                  Code:
                      i = Len(Tr): j = StrPtr(Tr)
                    
                      ! LEA EBX, TrtTable
                      ! MOV EDX, j
                      ! MOV ECX, EDX
                      ! ADD ECX, i
                    
                    LbTrt:
                      ! MOV AL, [EDX]
                      ! XLAT
                      ! MOV [EDX], AL
                      ! INC EDX
                    
                      ! CMP EDX, ECX      ; exit when edx = ecx
                      ! JNE LbTrt
                    
                      ''' ! LOOP LbTrt    ; loop is a slow instruction
                  Regards,

                  [email protected]

                  ------------------
                  hutch at movsd dot com
                  The MASM Forum

                  www.masm32.com

                  Comment


                  • #10
                    Steve --
                    You are right. This change was enough to make a time of Translate the same as for ConvertIt (0.12 sec in sample).

                    I am very doubt that Fred's problem is in Replace Any.
                    But if he will change Replace Any to my or your sub, and problem will remained, it will be confirmation that his problem has no relation to Replace Any. (to say true, I am very afraid a code without Register None)

                    ------------------

                    Comment


                    • #11
                      I hear what you are saying Semen.Loud and clear
                      I have spent my weekend to recompile Exe's and DLL-s to include
                      #Register None.
                      This version (whithout any other changes) will be tested on monday morning at customer site.
                      '---
                      I have also created a new suit of DLL-s with yours/Hutches replacement for Replace Any.
                      Also included the advice from Chris Boss to make all global variables into local ones.
                      It was a pain in the ***, but now its done.
                      '---
                      My testing show a favourable speed-increase in translating buffers,
                      It is that significant, I have to include Hutch's previous example
                      on string-concatenation, to match up on the output side...
                      '--
                      Problem is almost always a new oppertunity..


                      ------------------
                      Fred
                      mailto:[email protected][email protected]</A>
                      http://www.oxenby.se

                      Fred
                      mailto:[email protected][email protected]</A>
                      http://www.oxenby.se

                      Comment


                      • #12
                        Semen,

                        > I am very doubt that Fred's problem is in Replace Any.

                        Me too but if you really want to know you should make a slower version of your routine, so that it takes just as long as REPLACE.
                        If Fred can test with that version too, you can eliminate REPLACE being the problem for sure.


                        Peter.


                        ------------------
                        [email protected]

                        Comment


                        • #13
                          There is IMHO no doubt about it.
                          It is Replace-instruction that fails,
                          and there is no code in my program that can change part
                          of the translated buffer or the other two strings.
                          But this does not mean that the PB-coding of Replace-instruction
                          is bad or wrong.
                          I have been using the same code for nearly two years running
                          on win95 and NT4, without no (confirmed) problem at all.
                          The big difference is that the failing code is in a DLL
                          As I suggested in my very first posting, NT is switching away
                          from this Replace-code while it is working, and somehow, when it returns one of the string-adress is invalid, probably not the
                          Buffer-address. And this cause Replace to end prematuerly.
                          Only R&D at PB can tell if this is possible.
                          My question to this board was:
                          Can this 'context-switching' in the middle of Replace be avoided.
                          Perhaps you need to be closer to the OS than my program is, to be able to do that.
                          A critical section would not help, as it only prevent others to use the same code, and this someone is not there..
                          Can NT4 be loading a new instance of the DLL? That is the only
                          part of the code (DLL_ATTACH) that change the content of global
                          strings.
                          I can spend all night speculating but my small chicken-brain cant coope with such issues.
                          I am a complete novice on this matters and I dont have the skill or the tools to dig any deeper.
                          I will settle with problem resolved. I dont need to know where in the unknown windings of NT (or PB),
                          this problem has its origin.
                          '---Now someting complete diffrent-----------

                          Hutch and Semen,
                          I have incorporated your Translation-routines.
                          This gave such a speed-increase on the reading side,
                          that I also had to incorporate Hutches 'old' string-
                          concatenation example, to match up on the writing side.
                          I wanted to be able to not only add strings, but also
                          mixed strings and expression. I remembered Hutces word
                          that subs are faster than functions, so it is implemented
                          as a sub:
                          Code:
                          Sub String_Append(ByRef CurPos&,ByRef Buffer$,ByVal AddOn$)Export
                          #Register None 
                          Local pBuff&,pAddOn&,lenAddOn&,Cp&    
                            Cp& = CurPos&
                            pBuff&    = StrPtr(Buffer$)
                            pAddOn&   = StrPtr(AddOn$)
                            lenAddOn& = Len(AddOn$)
                              ! cld                 ' read forwards
                              ! mov edi, pBuff&     ' put buffer address in edi
                              ! add edi, Cp&        ' add starting offset to it
                              ! mov esi, pAddOn&    ' put string address in esi
                              ! mov ecx, lenAddOn&  ' length in ecx as counter
                              ! rep movsb           ' copy ecx count of bytes from esi to edi
                              ! mov edx, Cp&        '
                              ! add edx, lenAddOn&  ' add CurPos and lenAddOn 
                              ! mov Cp&, edx        ' put new value in CurPos
                              CurPos& = Cp&
                          End Sub
                           
                          It is used like this:
                          .....    
                               String_Append udtIn.UtCurLen,UtBuffer$,Left$(cmd$ & Filler$,udtIn.BlkLen)
                          ..... 
                          Local TmpBuffer$
                               If udtIn.UtCurLen > udtIn.UtMaxBuff Then
                                TmpBuffer$ = Mid$(UtBuffer$,1,udtIn.UtCurLen)
                                udtIn.UtCurLen = 0
                                If udtIn.CharSet = %EBC2ASC Then XltEbc2Asc TmpBuffer$
                                If udtIn.CharSet = %EBC2ANS Then XltEbc2Ans TmpBuffer$
                                ErrClear:Put$ udtIn.UtFilNr,TmpBuffer$
                                If ErrClear > 0 Then Function = 103: Exit Function
                                If udtIn.Dupit <> 0 Then 
                                 ErrClear:Put$ udtIn.UtFilNr2,TmpBuffer$
                                 If ErrClear > 0 Then Function = 103:Exit Function
                                End If 
                               End If
                               If udtIn.BytesToRead < 1 And udtIn.BuffPos >= udtIn.BuffSize Then Exit Do
                              Loop
                          Perhaps TmpBuffer$ = Left$(UtBuffer$,udtIn.UtCurLen) would be faster,
                          I dont know....
                          But ONE thing I know, that I don't know and that is how to say
                          THANK YOU GUYS.... in a proper way, because I did not believe my eyes
                          when this new DLL were tested.
                          In the test-report below you will see when the DLL was called and when it returned.
                          This test-times include everything open/close/read/write the file
                          I am still stunned...
                          Code:
                          -----Routine------------|MM:SS|MB read from disk and reWritten to disk
                          Start ZIPFIL    01:52:04
                          Stopp ZIPFIL    01:53:06  1:02  132 MB  705 KB  (No space but same pattern)
                          Start ZIPFIL    01:55:49
                          Stopp ZIPFIL    01:56:57  1:08  132 MB
                          Start UNZIPFIL  02:00:33
                          Stopp           02:01:45  1:12  132 MB
                          Start VAR2FIX   02:04:03
                          Stopp           02:06:41  2:38  132 MB
                          Start FIX2VAR   02:09:18
                          Stopp           02:11:36  2:18  205 MB
                          Start XLTEBCANS 02:15:37
                          Stopp           02:17:38  2:01  205 MB
                          Start PC2ASAANS 02:22:35
                          Stopp           02:24:57  2:22  132 MB
                          Start PC2PCC    02:27:41
                          Stopp           02:30:08  2:27  132 MB
                          Start PC2XFP    02:44:09
                          Stopp           02:46:39  2:30  132 MB
                          Start ASA2PCC   02:50:15
                          Stopp           02:52:57  2:42  132 MB
                          Start FIXASAPCC 03:54:43
                          Stopp           03:57:57  3:14  205 MB
                          Start VARASAXFP 04:03:29
                          Stopp           04:06:13  2:44  132 MB
                          Start PCC2ASA   04:10:29
                          Stopp           04:12:32  2:03  137 MB
                          Start PCC2ASA   04:17:57
                          Stopp           04:20:30  2:33  137 MB READ 205 MB WRITE
                          Start PCC2PCC   04:34:28
                          Stopp           04:35:50  1:38  139 MB (ALMOST ONLY COPY)
                          Start PCC2XFP   04:42:23
                          Stopp           04:44:14  1:51  139 MB (Sama as above but swapping Cmds)
                          ------------------
                          Fred
                          mailto:[email protected][email protected]</A>
                          http://www.oxenby.se



                          [This message has been edited by Fred Oxenby (edited April 17, 2000).]
                          Fred
                          mailto:[email protected][email protected]</A>
                          http://www.oxenby.se

                          Comment


                          • #14
                            Fred;

                            It is important to understand that you have to be "very careful" what kind of code you put in your LibMain function. I don't remember what your LibMain looked like, so its hard to comment about it.

                            It is always best to test for "all four" possible values for fwdReason . Your DLL could be loaded in a thread, which means the %DLL_THREAD_ATTACH event could be fired as well as %DLL_PROCESS_ATTACH .

                            It is important to make sure LibMain returns a value of 1 (means it successfully loaded).

                            Also it is important to remember that you have to be very careful in initializing globals from LibMain. Some times it is better to use a STATIC flag in your other function calls and when the flag is set the first time, "then" initialize your globals (rather than do it in LibMain).

                            Is it possible there is some "threading" going on that you may not be aware of ?

                            Writing Thread safe code is not easy.

                            Since, the problem occurs only in DLL form, then it worth investigating the differences between a DLL and EXE as far as the order in which code is executed and the use of Globals and when they are initialized.

                            Code:
                            FUNCTION LibMain(BYVAL hInstance   AS LONG, _
                                             BYVAL fwdReason   AS LONG, _
                                             BYVAL lpvReserved AS LONG) EXPORT AS LONG
                                SELECT CASE fwdReason
                                    CASE %DLL_PROCESS_ATTACH    ' =1 - Where DLL starts
                                    CASE %DLL_THREAD_ATTACH
                                    CASE %DLL_THREAD_DETACH
                                    CASE %DLL_PROCESS_DETACH    ' =0 - Where DLL exits
                                    CASE ELSE
                                END SELECT
                                LibMain=1
                            END FUNCTION

                            ------------------
                            Chris Boss
                            Computer Workshop
                            Developer of "EZGUI"
                            http://cwsof.com
                            http://twitter.com/EZGUIProGuy

                            Comment


                            • #15
                              Chris, I check for CASE %DLL_PROCESS_ATTACH and initialize
                              the globals here, not anywhere else.
                              I belive that %DLL_TREAD_ATTACH use the same 'data area' and
                              therefore need no init of globals. Am I wrong? In that case
                              this can be the answer..
                              The globals are 'static' and used only as 'read_only' strings.
                              But I have followed your advice and use no globals no more.
                              It was a leftover from good old dos-time, and I believe that
                              global strings are never moved around in memory,perhaps that is only the descriptor...
                              Anyhow, once I found how to design a new UDT to suit all my needs it was only plain hard work to rewrite the code.
                              It also gave me the oppertunity to stunn myself when looking
                              at the speed...
                              I just want to see this on an ATM-network...witch is coming.


                              ------------------
                              Fred
                              mailto:[email protected][email protected]</A>
                              http://www.oxenby.se

                              Fred
                              mailto:[email protected][email protected]</A>
                              http://www.oxenby.se

                              Comment


                              • #16
                                Fred;

                                I have a little trick for "testing" which event occurs in LibMain. You can't use a messagebox from LibMain (it can GPF), so what I do is simply put the BEEP command in one of the LibMain Select case sections.

                                ie.

                                Code:
                                    SELECT CASE fwdReason
                                        CASE %DLL_PROCESS_ATTACH
                                        CASE %DLL_THREAD_ATTACH
                                            BEEP
                                        CASE %DLL_THREAD_DETACH
                                        CASE %DLL_PROCESS_DETACH
                                        CASE ELSE
                                    END SELECT
                                This will help you find out if the DLL is getting loading and initialized by a Thread.

                                I use the BEEP command a lot to "verify" that certain code got executed.


                                ------------------
                                Chris Boss
                                Computer Workshop
                                Developer of "EZGUI"
                                http://cwsof.com
                                http://twitter.com/EZGUIProGuy

                                Comment


                                • #17
                                  Well, the #Register None version of Spider have been running
                                  at customer-site for 48 hrs. There has been no problem at all.
                                  After 24 hrs of continues testing, we installed SP6a and Novell
                                  4.7 NT4/Win2000 client.
                                  A new 24 hrs period, and as I said no problem at all.
                                  Tomorrow, the 'highly optimized' version vill be tested with
                                  same accuracy. and the result then compared. (all actions are timed)
                                  '---So what does this mean----------------
                                  It seems that including #Register None fixed the problem.
                                  I have no Idea how this can be so, as there are no variables
                                  in the failing function, as I understand it, which could be
                                  assigned to the 'special register-memory'
                                  Anyhow there are many things I dont understand, and I'm still happy so ....
                                  '--Thank you everybody, including Lance, for your help and
                                  clarifications in this matter.



                                  ------------------
                                  Fred
                                  mailto:[email protected][email protected]</A>
                                  http://www.oxenby.se

                                  Fred
                                  mailto:[email protected][email protected]</A>
                                  http://www.oxenby.se

                                  Comment

                                  Working...
                                  X