Simple Question on TYPEs

[Lance Edmonds]

Although this is off topic, telecommuting does reduce the need to be physically in the same locale these days, but it does really depend on the job.

Unfortunately, not all jobs are suitable for telecommuting and not all Bosses permit telecommuting or telecommuters... I just happen to have a good boss!

oh... er... um... Hi Bob...! What are you doing in here?!


------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Borje Hagsten]

Oh - I automatically placed you where PB is located - terribly
sorry for that, Lance.

Gives an interesting aspect to the job offers from PB earlier
this year. I thought you had to live in USA, but since one of
PB's absolutely best support guys even lives on the other side
of the planet.. hm, there's food for some new thoughts there.


------------------

[Lance Edmonds]

PS. Borje, as I understand, Lance is located in KiwiLand

Yup! I live in New Zealand and telecommute... it is no secret. Also, because I'm at GMT+12, I live in the "future" as compared to most people in the world!

------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Warren Daniel]

Thanks for the suggestions, I'll give them a try. A sample line from the database looks like this:

12|TJGA|89.00|TJGA6 (Swinging Bridges II)|<IMG SRC="/toerings/tjga6.jpg" BORDER=0>|Same Design as above only a touch wider. 2mm in width.|6.50&#0124; &#0124;&#0124; &#0124;&#0124; &#0124;|

The reason I'm not using binary or something else is because this database is going to be read by a Perl CGI script and will serve up a shopping cart with it, and I need to conform to the way the script does things. As far as the optimizations go, Borje, I'll give them a whirl, but the database is only going to be about 15kb (Pales in comparison to your 100,000 word database) and i'm not worried about speed too much.

------------------

[Semen Matusovski]

Guys --
When I saw your discussion, first question, which I wanted to ask -- is it really necessary to use UDT or it's possible to use a group of string arrays ?
As I see, input information stored as | delimited, not like UDT.

Anyway, If there are reasons to use UDT, may be will be easier to declare fixed-length strings, for example, ... Id As string * 8.
PS. Borje, as I understand, Lance is located in KiwiLand

------------------

[Borje Hagsten]

Kind of amazing, isn't it? I'm located in Sweden - Lance in USA,
but that still doesn't stop us from giving almost identical
answers, at almost the same time.

Warren - if this is by any means confusing - trust Lance's
answers in the first hand. I'm nothing but a poor amateur,
compared to his extensive knowledge in PB programming..


------------------

[Borje Hagsten]

Hopefully, my mind isn't too tired so at least some of the
following will work..

"StrData = StrData & TmpStr" must be a bottleneck here. You are
slowly building up a dynamic string, piece by piece. Why not
read the entire file into a string at once instead and then
remove all CRLF's?

OPEN gCommandLine$ & "\data.file" FOR BINARY AS #1
  StrData = SPACE$(LOF(1))
  GET #1,, StrData
CLOSE #1
StrData = REMOVE$(StrData , CHR$(13,10))

Then a minor improvement (remove one parsecount):

LOCAL p AS LONG
p = PARSECOUNT(StrData, ANY "|")
REDIM Tmp(p)
FOR X = 1 TO p
  Tmp(X) = PARSE$(StrData, ANY "|", X)
NEXT X

Finally, change the TYPE declare to: (ID AS LONG)

TYPE Product
  ID AS LONG
  Catagory AS STRING PTR
  PName AS STRING PTR
  Price AS STRING PTR
  ShipCost AS STRING PTR
  PImage AS STRING PTR
  Description AS STRING PTR
END TYPE
 
A = 0
FOR X = 0 TO UBOUND(Tmp) -13 STEP 13
  Table(A).ID = A
  Table(A).Catagory = VARPTR(Tmp(X + 2)) 'None of these compile.
  Table(A).Price = VARPTR(Tmp(X + 3))
  Table(A).PName = VARPTR(Tmp(X + 4))
  Table(A).PImage = VARPTR(Tmp(X + 5))
  Table(A).Description = VARPTR(Tmp(X + 6))
  Table(A).ShipCost = VARPTR(Tmp(X + 7))
  INCR A
NEXT X

Note - StrData must remain as it is, so the arrays can point at
it all the time.


------------------

[Lance Edmonds]

What sort of format does the data in "DATA.FILE" use? Is it not possible to create an array of UDT's based on the "largest" of each relative field in this file, and build the UDT as you load the file from disk?

I'd also look at using a BINARY file to load the data file, rather than LINE INPUT,but it will depend on the exact format fo the data.

So, without knowing more about the data, here is how I'd do it:

' For this code to be useful, the string array must be GLOBAL so that the
' UDT of pointers can reference the data at all times.
#COMPILE EXE
'#DIM ALL

TYPE Product
    ID          AS STRING PTR
    Catagory    AS STRING PTR
    PName       AS STRING PTR
    Price       AS STRING PTR
    ShipCost    AS STRING PTR
    PImage      AS STRING PTR
    Description AS STRING PTR
END TYPE

GLOBAL Table() AS Product
GLOBAL Tmp()   AS STRING

FUNCTION PBMAIN() AS LONG
    DIM Table(255) AS GLOBAL Product
    CALL LoadData()
    ' now reference the targets of the pointers in the UDT
    MSGBOX "The ID of the 1st item is " & Table(1)[email protected]
    ' Change the content of the 3rd' items description:
    Table(3)[email protected] = "New Description"
END FUNCTION

SUB LoadData()
    DIM Tmp() AS GLOBAL STRING
    DIM TmpStr AS STRING
    DIM StrData AS STRING
    DIM X AS LONG   ' use longs instead of integer for efficiency
    DIM A AS LONG

'    CALL GetRealPath()
    OPEN gCommandLine$ & "\data.file" FOR INPUT AS #1

    ' assuming parse loops works I did not test it
    DO WHILE NOT EOF(1)
        LINE INPUT #1, TmpStr
        StrData = StrData & TmpStr
    LOOP
    CLOSE #1

    ' store the individual strings in the array
    REDIM Tmp(PARSECOUNT(StrData, ANY "|"))
    FOR X = 1 TO PARSECOUNT(StrData, ANY "|")
        Tmp(X) = PARSE$(StrData, ANY "|", X)
    NEXT X
    TmpStr  = "" ' release the memory
    StrData = ""

    ' Create a UDT of pointers to the string array subscripts
    A = 0
    FOR X = 1 TO UBOUND(Tmp(1)) - 1 STEP 13
        Table(A).ID          = VARPTR(Tmp(X))
        Table(A).Catagory    = VARPTR(Tmp(X + 2))
        Table(A).Price       = VARPTR(Tmp(X + 3))
        Table(A).PName       = VARPTR(Tmp(X + 4))
        Table(A).PImage      = VARPTR(Tmp(X + 5))
        Table(A).Description = VARPTR(Tmp(X + 6))
        Table(A).ShipCost    = VARPTR(Tmp(X + 7))
        INCR A
    NEXT X
END SUB

------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Warren Daniel]

Yep, I come from a VB background. I guess this is not such a "Simple Question on TYPEs" after all. I'm trying to port one of my old VB apps over to PB, and for simplicity's sake, I filtered out all the programs using TCP/IP, databases (ok, other than flat file databases), etc... I had no idea I should be filtering out apps that use TYPE too. I can get rid of the pointers and just throw TRIM$'s all over the place, but that'll be a little bit awkward. Let me just throw a nice chunk of code in and see if anyone can think of something...

SUB LoadData()
DIM Tmp() AS STRING
DIM TmpStr AS STRING
DIM StrData AS STRING
DIM X AS INTEGER
DIM A AS INTEGER
CALL GetRealPath()
OPEN gCommandLine$ & "\data.file" FOR INPUT AS #1
DO WHILE NOT EOF(1)
LINE INPUT #1, TmpStr
StrData = StrData & TmpStr
LOOP
CLOSE #1

REDIM Tmp(PARSECOUNT(StrData, ANY "|"))
FOR X = 1 TO PARSECOUNT(StrData, ANY "|")
Tmp(X) = PARSE$(StrData, ANY "|", X)
NEXT X

FOR X = 0 TO UBOUND(Tmp) -13 STEP 13
'Table(A)[email protected] = STR$(VARPTR(A)) 'Not a CLUE what to do HERE.
Table(A)[email protected] = VARPTR(Tmp(X + 2)) 'None of these compile.
Table(A)[email protected] = VARPTR(Tmp(X + 3))
Table(A)[email protected] = VARPTR(Tmp(X + 4))
Table(A)[email protected] = VARPTR(Tmp(X + 5))
Table(A)[email protected] = VARPTR(Tmp(X + 6))
Table(A)[email protected] = VARPTR(Tmp(X + 7))
A = A + 1
NEXT X
END SUB

------------------

[Borje Hagsten]

With some knowledge about Windows programming, PB actually is
both easier to use and more flexible than VB, because PB allows
us to "take command" in a much more powerful way.

The wrappers VB provides you with comes with a price - memory
and low speed. Sure, it's nice to get it all in just one line
of code, but you also get a lot of overhead, because MS writes
code that's supposed do work in many different situations, so
it is not optimized for any specific task, so to speak.

As an example: With PB code, I read a hundred thousand words from
disk into a dynamic array and then sort and index the array in
less than a second on a fast machine. It's for a spell checker I
have written. The exact same procedure takes more than 10 seconds
with VB5 code..


------------------

[Lance Edmonds]

Warren, I'm not trying to sound demeaning or to put you down, but please don't run before you can walk. Pointers are very powerful, but they are not the easiest thing to comprehend at first glance. I'd suggest using standard string variables and UDT's before employing the use of a pointer.

If you could give us some told us more about what you want to achieve, we can probably help guide you along. My guess is that you have a VB background, right?

Don't ever feel that any question you ask here is too simple or too complex to answer - but in order to help often need a wider appreciation of what you are aiming to achieve...



------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Borje Hagsten]

Yeah, it probably can be a bit confusing sometimes, but also
extremely rewarding. PB's help file has a lot of info about
pointers, etc. - well worth reading.

The advantage of using "ordinary" dynamic arrays is that you
can use ARRAY SORT/SCAN/DELETE, etc, which enables you to speed
up things a lot.

The disadvantage is a bit slower reading from disk, but unless
you have hundreds of thousands of records in the database, that
is not a problem.


------------------

[Warren Daniel]

If PB isn't easy enough to use, It's because PB strives to be so powerful and flexible. If PB isn't flexible enough, it's because PB strives to be so fast. If allowing resizeable strings as part of a UDT slows things down, is it possible to allow the programmer to decide what he'd prefer: Speed or Flexibility? I could set the length of the string when I need speed, and not set it when I need flexibility? I guess there may be all sorts of computer limitations that'd make it hard for a compiler to do, but I've done it in VB without problems. (Although i don't know if in VB defining the size of the string has any effect on speed later on.)

------------------

[Warren Daniel]

Say WHAT?? I was about to ask why doing:

Table(A)[email protected] = VARPTR(Tmp(X + 2))

Cry's for a string operand... Tried STRPTR, demanded the same thing. Holy crap all this POWER in PowerBASIC is confusing.

------------------

[Borje Hagsten]

I must be tired. When you use a STRING PTR member in TYPE declared
array, you must of course make sure each item exists somewhere.
This means you'll have to create a second "normal" array for the
actual strings - each STRING PTR pointing to a unique item in this
array.

In other words - it can be done, but you'll get a four byte
overhead on each item (pointers are DWORD). Probably better
to use plain old dynamic arrays instead and skip dynamic
strings in UDT's..

------------------


[This message has been edited by Borje Hagsten (edited June 17, 2000).]

[Lance Edmonds]

The next problem you're likely to face is storing this UDT - simply writing the UDT into a file will be worthless because you will just be storing a set of pointers and no target data. Worse, even if you stored the pointers and read them in again later, their target addresses would be meaningless. Hmmmm!

To solve this, you'd need to write of the target strings to disk rather than the pointers. If/when you re-read these strings, you'll have to place them in suitable memory storage positions, and fill in the pointers in your UDT again.

This is a poor approach for storing database records - you'll probably need to go back to using fixed-length string in your UDT, which may waste "unused" bytes in each record, but will work out much faster at runtime compared to reading disrete dynamic strings into memory & assigning pointers, etc.

However, without knowing more about what you application is actually going to achieve, I cannot offer more concrete advice.

Pointers are very useful creatures, but you have to choose your use of them carefully if you wish to exploit their power.


------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Borje Hagsten]

My fault - sorry. It's a STRING PTR, so of course it needs
something to point to.

LOCAL TmpStr AS STRING
REDIM Table(255)

TmpStr = "TJGC"
Table(0).ID = VARPTR(TmpStr)
MSGBOX Table(0)[email protected]

------------------

[Lance Edmonds]

Warren, you came back with another question while I typed my last response.


A pointer is that - it points to where the storage is maintained, but you are trying to use a pointer without allocating storage for the data!

You need to do it this way instead:

A$ = "TJGC"
Table(0)[email protected] = VARPTR(A$) ' the .ID member must be a STRING PTR




------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Lance Edmonds]

The reason for requiring a fixed length is because the concept of using a UDT is that all elements occupy as fixed position and size within the structure which is set at compile time.

If dynamic strings were part of the actual structure itself, the position of members would have to be calculated whenever a reference was made to the UDT - this would add significant overhead and affect performance. Need I remind you that performance is something that PowerBASIC strives to deliver!

By comparison, Borje's suggestion of using a string pointer is is almost exactly how VB handles dynamic strings within UDT's - it utilizes a form of pointer to the string data which is usually stored elsewhere in memory. (Actually, VB is even more complicated than this because it internally stores dynamic strings as Unicode/Wide characters which forces it to translate all strings and UDT structures before passing them to DLL's - this creates a lot of additional overhead and a drop in it's performance).

------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Warren Daniel]

Well, Good old Dr. Watson doesn't like doing it that way, here's the code I've written:

-------------------------
TYPE Product
ID AS STRING PTR
Catagory AS STRING PTR
PName AS STRING PTR
Price AS STRING PTR
ShipCost AS STRING PTR
PImage AS STRING PTR
Description AS STRING PTR
END TYPE

GLOBAL Table() AS Product

REDIM Table(255)
Table(0)[email protected] = "TJGC"
-------------------------

It "Generates Errors" and dies when trying to assign the value.

------------------