Number to String or Byte Array Conversion

[Peter House]

I am converting some numbers to a string which will be inserted in a UDP Packet for transfer to an embedded device I am working on.

I am wondering what is the most efficient way to accomplish this goal and have come up with the following two methods.

Which one of these if more efficient and do you know of a better way?

Local Msg As String
Local OutID As Word
Local ChangeTime As Dword
Local x As Byte Ptr

Word to String:
x = VarPtr(OutID)
msg = Chr$(x[0]) & Chr$(x[1])

Or

Msg$ = Chr$(Lo(Byte, OutID)) & Chr$(Lo(Byte, OutID))

DWord to String:
x = VarPtr(ChangeTime)
msg = Chr$(x[0]) & Chr$(x[1]) & Chr$(x[3]) & Chr$(x[4])

Or

Msg$ = Chr$(Lo(Byte, (Lo(Word, ChangeTime))) & _
Chr$(Lo(Byte, (Hi(Word, ChangeTime))) & _
Chr$(Hi(Byte, (Lo(Word, ChangeTime))) & _
Chr$(Hi(Byte, (Hi(Word, ChangeTime)))

[Peter Lameijn]

Maybe MKDWD$?

[Dave Biggs]

As you would expect PB's built-in MK...$ functions perform best -with less likelyhood of typos too

#COMPILE  EXE
#Register None
#Dim      All
#Include "WIN32API.INC"
 
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Macro InitializeTimer                              'TT David Roberts
 Local qFreq, qOverhead, qStart, qStop As Quad
 Local f As String
  f = "#####.##"
  QueryPerformanceFrequency qFreq 
  QueryPerformanceCounter qStart ' Intel suggestion. First use may be suspect 
  QueryPerformanceCounter qStart ' So, wack it twice <smile>
  QueryPerformanceCounter qStop
  qOverhead = qStop - qStart     ' Relatively small 
End Macro
 
Macro StartTimer = QueryPerformanceCounter qStart
 
Macro StopTimer = QueryPerformanceCounter qStop
 
Macro TimeTaken = Using$(f,(qStop - qStart - qOverhead)*1000/qFreq) + "ms"
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
Function PbMain()
 Local Msg As String
 Local OutID As Word, ChangeTime As Dword
 Local x As Byte Ptr
 Local i As Long
 Local T1, T2, T3 As String 
  InitializeTimer
  OutID = 65534 :  ChangeTime = 4224262224     'e.g.
 
 'Word to String:
  x = VarPtr(OutID)
  StartTimer
  For i = 1 To 100000
    msg = Chr$(@x[0]) & Chr$(@x[1])
  Next
  StopTimer
  T1 = TimeTaken
 
'Or
  StartTimer
  For i = 1 To 100000
    Msg = Chr$(Lo(Byte, OutID)) & Chr$(Hi(Byte, OutID))
  Next
  StopTimer
  T2 = TimeTaken
 
'Or  
  StartTimer
  For i = 1 To 100000
    Msg = MKWRD$(OutID)
  Next
  StopTimer
  T3 = TimeTaken
 
  MsgBox  "T1 "+ T1 + $CRLF + _                ' ~ 124ms (on my PC)
          "T2 "+ T2 + $CRLF + _                ' ~ 118ms
          "T3 "+ T3, , "Word to String"        ' ~  40ms
'---------------------------------------------
 
 'DWord to String:
  x = VarPtr(ChangeTime)
  StartTimer
  For i = 1 To 100000
    msg = Chr$(@x[0]) & Chr$(@x[1]) & Chr$(@x[2]) & Chr$(@x[3])
  Next
  StopTimer
  T1 = TimeTaken
 
'Or
  StartTimer
  For i = 1 To 100000
    Msg = Chr$(Lo(Byte, (Lo(Word, ChangeTime)))) & _
    Chr$(Hi(Byte, (Lo(Word, ChangeTime)))) & _
    Chr$(Lo(Byte, (Hi(Word, ChangeTime)))) & _
    Chr$(Hi(Byte, (Hi(Word, ChangeTime))))
  Next
  StopTimer
  T2 = TimeTaken
 
'Or  
  StartTimer
  For i = 1 To 100000
    Msg = MKDWD$(ChangeTime)
  Next
  StopTimer
  T3 = TimeTaken
 
  MsgBox  "T1 "+ T1 + $CRLF + _                ' ~ 275ms
          "T2 "+ T2 + $CRLF + _                ' ~ 257ms
          "T3 "+ T3, , "DWord to String"       ' ~  39ms
 
End Function
'------------------/PBMain

[Michael Mattias]

Instead of..

 Msg = Chr$(Lo(Byte, OutID)) & Chr$(Hi(Byte, OutID))

with both the CHR$() and string concatenation, creating an output buffer and using a translation table would be a heck of a lot faster.

**BUT ***
with the TCP Sends in there, the 'convert numbers to characters' is such a small part of your total overhead that it really is like rearranging the deck chairs on Titanic.

I'm surprised you didn't notice this when you PROFILEd your program.

Translation table demo:

http://www.powerbasic.com/support/pbforums/showthread.php?t=25176

Those go to/from Hex/Binary, but you could do the same kind of thing for Decimal/Binary easily enough.

MCM

[John Petty]

MCM
Such sensative help for a new user. Might help next time if you actually read the post. He is using UDP not TCP which is a tad faster.
John

[Michael Mattias]

OK, I missed that it was a 'first post.'

UDP or TCP ==> in context, immaterial.

That context being, the dynamic string functions (eg CHR$() and concatenation) are quite time consuming (relatively speaking) and that is a good place to look for savings.. to avoid using them needlessly.

However, in the even larger context of the application as a whole, the UPD or TCP statements are going to be using so much more time than the CHR$() or "&" that any overall savings will be moot.

My opinion on this being just that - an opinion - one can validate or disprove it for this particular application by using the PROFILE function to see where in your program you are actually spending most of your time.

MCM

[John Petty]

MCM
Of course many CHR$ can be slow, probably why 9 has BUILD. Of course Peters suggestion is very fast though the result is different from say FORMAT$, is late and I have a life so havn't exactly worked out the posters need. As for TCP vs UDP I have noticed from prior posts that the net is not your strong point, can't even remember you attaching a reference to your posts of an example.
John

[Michael Mattias]

Nah, I don't know net stuff... but I do know that any I-O.. net, disk, peripheral, speakers is LOTS slower than any memory operation.

And I do know that spending time trying to optimize a handlful couple of string operations in an I-O intensive application is not the best use of your time.

And last but not least, I also know any optimization which requires the use of QueryPerformanceCounter to measure it is absolutely impossible for the user to detect.

Sure, there's a satisifaction inherent in knowing you've done something as fast as it can be done, but I actually practice what I preach: First you make it work; only then do you make it work better.

I am also very big on the ultimate testing technique: "What happened when you tried it?"



MCM

[Dave Biggs]

It's interesting to me to note that there are further efficiencies to be had by utilizing an alternate form of the Chr$() function. i.e.

msg = Chr$(@x[0], @x[1], @x[2], @x[3])

Instead of:

msg = Chr$(@x[0]) + Chr$(@x[1]) + Chr$(@x[2]) + Chr$(@x[3])

In the test code above this results in the conversion time being reduced by 2/3 or more.
e.g. 275ms -> 82ms (Arguably more readable code too?)

MK...$ is still the winner here though coming in at ~ 40ms

Could MCM's translation table challenge that I wonder? (Yeah I know - try it and see!)

[Michael Mattias]

> Yeah I know - try it and see!

Say Amen, Hallelujah! HE SEETH THE LIGHT!

[John Petty]

Nah, I don't know net stuff... but I do know that any I-O.. net, disk, peripheral, speakers is LOTS slower than any memory operation.

And I do know that spending time trying to optimize a handlful couple of string operations in an I-O intensive application is not the best use of your time.

MCM

Not sure you are correct there Michael, The WSOCK32 sendto function is normally non-blocking (unless its buffers are full) and there are no ACK's to wait for so it will return to a program that can then continue preparing the next packet or other work while the OS uses its own time slices to send the packet. Of course if the program is just continually creating simple packets and doing nothing else (which would seem pretty useless), then it will probably fill the buffer.
As for "try it and see" it is your demo so why are you not posting comparative times?

[Michael Mattias]

>why are you not posting comparative times?

Because the demo shows how to use translation tables; it does not purport to provide any specific performance.

Your mileage may vary. Void where prohibited.

[Dave Biggs]

with both the CHR$() and string concatenation, creating an output buffer and using a translation table would be a heck of a lot faster.

Just how much is a heck of a lot ?

I've still to figure out how to apply the table techniques to Peter's goal but timing MCM's posted function didn't yield any real difference in the processing time.. e.g.

  For n = 1 To 390
  FOR I = 1 TO LEN(FileData)               ' for each byte of input
     @pDest   =  @pHex[@pSrc]              ' move 2 hex characters to output
     INCR  pSrc                            ' and advance in input & output
     INCR  pDest
     Incr c
  Next
   pSrc        =  STRPTR (FileData)         ' point to start of input
   pDest       =  STRPTR (OutputBuff)       ' and output
  Next
  StopTimer
  T2 = TimeTaken

This is about 100000 iterations as a comparison to the loops that are timed in the code posted eariler.
Not yet 'apples to apples' but possibly indicative anyway that there's no great improvements over MK...$ or the quicker form of Chr$() to be had with this technique in Peter's application.

[John Petty]

>Because the demo shows how to use translation tables; it does not purport to provide any specific performance.

Your mileage may vary. Void where prohibited.

Then why are you being so posative about telling people they should waste time with "try and see" if you don't even know the result. Using large tables has long been known to sometimes speed up math functions depending on the hardware, ie log tables for powers, in early days before dedicated multiplication chips they were even used for that.
What is your point, the poster asked a specific speed question and now your replies claim to be merely to teach people how to use tables. I didn't see anything in his post that suggested he needed lessons on tables.
Think its time you PUT UP OR!!

[Michael Mattias]

It is my considered opinion that all optimization is application-specific (I think you'll find more than one such post of mine here).

Given that view, "try it and see" in the application itself is the only way to test such optimizations, isn't it?

>Think its time you PUT UP OR!!

Put up "what?"

Or what?

If your goal was to stump me, you win.

[John Gleason]

Hey, look at this. I timed it 4x faster than MKDWD$

#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

    LOCAL Msg AS STRING
    LOCAL ChangeTime AS DWORD
    LOCAL ii AS LONG, t AS SINGLE

     changeTime = CVDWD("1234")

    'using POKE
     t = TIMER
     FOR ii = 1 TO 10000000
        msg = "    "
        POKE DWORD, STRPTR(Msg), ChangeTime
     NEXT
     ? "POKE used" & STR$(TIMER - t)

    'using MKDWD$
     t = TIMER
     FOR ii = 1 TO 10000000
        msg = MKDWD$(changeTime)
     NEXT
     ? "MKDWD$ used" & STR$(TIMER - t)

#IF %DEF(%PB_CC32)
  WAITKEY$
#ENDIF

END FUNCTION

[Dave Biggs]

Yeah! Move the assignment of msg = " " outside the loop and it's a heck of a lot faster

    'using POKE
     msg = "    "
     t = TIMER
     FOR ii = 1 TO 10000000
        '
        POKE DWORD, STRPTR(Msg), ChangeTime
     NEXT
     ? "POKE used" & STR$(TIMER - t)

[John Gleason]

Yes indeed, assuming you make a msg for the DWORD of 4 bytes and one for the WORD of 2 bytes, say msg4 and msg2, you can move the STRPTR out too:

    'using POKE
        msg4 = "    "
        x4 = STRPTR(Msg4)
        msg2 = "  "
        x2 = STRPTR(Msg2)

     t = TIMER
     FOR ii = 1 TO 1000000000
        POKE DWORD, x4, ChangeTime
     NEXT
     ? "POKE used" & STR$(TIMER - t)

     t = TIMER
     FOR ii = 1 TO 1000000000
        POKE WORD, x2, OutID
     NEXT
     ? "POKE used" & STR$(TIMER - t)

Net is actually < 2 tix, or about 270x faster.

[Michael Mattias]

Unfortunately... POKEing WORDs or DWORDs or MKWRD$() or MAKDWD$() into a string results in entities requiring an 8-bit transmission protocol. The internet is 7 bits. UDP and TCP both use the Internet.

Faster, yes; but in context, useless.

To transmit "numbers" over the internet they must be in character form. You can use MIME or UUENCODE (if you remember that) or convert to decimal or hex or other base characters.

[Dave Biggs]

Nah, I don't know net stuff...

You're kidding - right?