Announcement

**Don Dickinson** · 16 Mar 2004, 09:05 AM

i haven't worked with a treeview via the api lately, but i seem to recall that you can specify how many nodes windows should allocate at a time. by allocating a bunch of nodes at once (say 1000) and using LockWindowUpdate while loading, you can speed it up significantly.
i'd beware of the multi-threaded interface ... that can be tricky to deal with (see the Add Software/Windows components applet in the control panel of win2k for an example how not to do this).

--don

------------------
Don Dickinson
Author of ddoc Print and Preview A dll-based print-preview engine for windows
www.greatwebdivide.com
Author of Tsunami Tcp Data Server
http://ttds.greatwebdivide.com

**Michael Mattias** · 16 Mar 2004, 09:52 AM

>but i seem to recall that you can specify how many nodes windows should allocate at a time.

I found that for the LISTview control (LVM_SETITEMCOUNT), but I can't find one for the TREEview control...

.. but that gives me another idea...

Maybe I can use a listview control with the "indent" feature instead of a treeview...should look close enough for government work..

I already disable the drawing (WM_SETREDRAW) whilst the tree is loading, so I don't know what benefits LockWindowUpdate might bring.
MC

**Don Dickinson** · 16 Mar 2004, 09:58 AM

if you're thinking of a listview ... you might want to checkout siGrid from www.softwareinnovators.com - their pb grid is very, very fast.
same with elias's grid - i don't recall the url, but it looks pretty good too.

--don

------------------
Don Dickinson
Author of ddoc Print and Preview A dll-based print-preview engine for windows
www.greatwebdivide.com
Author of Tsunami Tcp Data Server
http://ttds.greatwebdivide.com

**Russ Srole** · 16 Mar 2004, 11:45 AM

Michael,

If you end up using the listview, be sure to use it as a virtual listview. It's basically an ownerdraw listview, the os sends you a message and tells you what cells to fill and you draw them with data stored in an array. If you don't do that you'll end up with the same speed penalty.

Russ Srole

------------------

**Michael Puckett** · 16 Mar 2004, 12:13 PM

Forgive me if this has already been suggested and I missed
it ...

I remember reading (can't remember if it was Petzold or Balena)
about a "demand loaded" tree technique that *may* help out your
performance woes if the data is indeed a tree. If the data is
essentially flat (all nodes at the primary level) there won't be
any improvement.

Anyway, sorry to ramble ... The technique involves only loading
those nodes at a given level, when required.

For example,

Code:

Initial load

+-- Root
|   +-- Node 1
|   +-- Node 2
|   +-- Node 3
|   +-- Node 4
|   +-- Node 5
  
Initally 5 nodes loaded, now click Node 1:
  
+-- Root
|   +-- Node 1
|   |   +-- Node 1.1
|   |   +-- Node 1.2
|   |   +-- Node 1.3
|   |   +-- Node 1.4
|   |   +-- Node 1.5
|   +-- Node 2
|   +-- Node 3
|   +-- Node 4
|   +-- Node 5
  
Nodes 1.1 thru 1.5 loaded, now click Node 1.1

+-- Root
|   +-- Node 1
|   |   +-- Node 1.1
|   |   |   +-- Node 1.1.1
|   |   |   +-- Node 1.1.2
|   |   |   +-- Node 1.1.3
|   |   |   +-- Node 1.1.4
|   |   |   +-- Node 1.1.5
|   |   +-- Node 1.2
|   |   +-- Node 1.3
|   |   +-- Node 1.4
|   |   +-- Node 1.5
|   +-- Node 2
|   +-- Node 3
|   +-- Node 4
|   +-- Node 5
|

Nodes 1.1.1 thru 1.1.5 loaded.

Hope that makes sense/is not a repeat of previous suggestion(s) or
your initial post; like idea (1).

Found the article:

Page 514. "Loading nodes on demand.", from the book "Programming
Visual Basic 6.0", Francesco Balena.

As well as a coding example by Brad Martinex: "TVFastLoad: How to
load the TreeView quickly".

http://www.mvps.org/btmtz/treeview (about 2/3 down page)
http://www.mvps.org/btmtz/treeview/tvfastload.zip

Cheers,

Michael

[This message has been edited by Michael Puckett (edited March 16, 2004).]

**Steven Pringels 3** · 16 Mar 2004, 12:27 PM

Hi Michael,

If you don't mind spending $500 buck , try the SFT/Tree DLL

http://www.softelvdm.com/treedll.html

You probably could recupe the cost by the price of the software
yuo're selling. I don't have this control but I am tempted though,
although for me $500 is a bit much for a control.

Regards

Steven

------------------

**Michael Mattias** · 16 Mar 2004, 01:05 PM

> ... The technique involves only loading those nodes at a given level, when required

That's a good idea, and I have thought about it..

My problem is the one you suggest: in most cases, the vast majority of the the nodes will be at a single level (level three).

So I could load up levels one and two, but as soon as the user clicks on the little box to expand level two, WHAMMO, I'm loading a boatload of stuff.. and in this case, the user gets to wait at exactly the wrong time. Users can be reasonably patient on "file/load" but they just won't go for "please wait" when they click to expand a tree node.

Maybe I will just do some kind of "load only if visible" thing and trap scrollbar, arrow and "page key" messages or something like that. That way I'd only have to load 20-30 nodes at a crack. Figgering out what would be visible might not be a whole lotta fun, though... (and then there's the 'search' feature, where I (currently) enumerate the tree... can't enumerate nodes which haven't been added yet, so I'd have to figure out another way to handle the search).

MCM

**Michael Puckett** · 16 Mar 2004, 01:14 PM

Originally posted by Michael Mattias:
... The technique involves only loading those nodes at a given level, when required

That's a good idea, and I have thought about it..

My problem is the one you suggest: in most cases, the vast majority of the the nodes will be at a single level (level three).

So I could load up levels one and two, but as soon as the user clicks on the little box to expand level two, WHAMMO, I'm loading a boatload of stuff.. and in this case, the user gets to wait at exactly the wrong time. Users can be reasonably patient on "file/load" but they just won't go for "please wait" when they click to expand a tree node.

Maybe I will just do some kind of "load only if visible" thing and trap scrollbar, arrow and "page key" messages or something like that. That way I'd only have to load 20-30 nodes at a crack. Figgering out what would be visible might not be a whole lotta fun, though... (and then there's the 'search' feature, where I (currently) enumerate the tree... can't enumerate nodes which haven't been added yet, so I'd have to figure out another way to handle the search).

MCM

Could you impose an arbitrary structure above level 3 to subdivide the data?

e.g. Put in Alpha indexes A-Z, thereby breaking the data into another 26 pseudo branches. You get the idea. Who wants to scroll a bazillion items anyway?

Cheers,

Michael.

[This message has been edited by Michael Puckett (edited March 16, 2004).]

**George Bleck** · 16 Mar 2004, 09:06 PM

mike,

sounds like re-thinking the gui may be in order. that many items in
a tree view sounds like a nightmare to search for a piece of data.

since we don't know the data structure/relationship it's hard to figure out
a more appropriate gui (if one would apply).

alternative using treeview...
you could do an on-demand fill using threads so when the client clicks on a node
it starts filling the items in the background, my bff example in the source code
forum does this. http://www.powerbasic.com/support/pb...ad.php?t=23778

------------------
every day i try to learn one thing new,
but new things to learn are increasing exponentially.
at this rate i’m becoming an idiot faster and faster !!!
------------------
george w. bleck
lead computer systems engineer
keyspan corporation
my email

[this message has been edited by george bleck (edited march 16, 2004).]

**Michael Mattias** · 17 Mar 2004, 06:23 AM

Since we don't know the data structure/relationship it's hard to figure out a more appropriate GUI (if one would apply

Basic ANSI ASC X12 EDI:

Code:

FILE
    - Interchange
         - Functional group
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
         - Functional group
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment

    - Interchange
         - Functional group
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
         - Functional group
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment
              - Transaction set
                  - Segment
                  - Segment
                  - Segment

"Tree" is perfect for this.

I started on this project yesterday. OK, so I worked on other parts first whilst digesting all these ideas.

First thing I'm going to try is storing the text externally to see what that does.

MCM

[This message has been edited by Michael Mattias (edited March 17, 2004).]

**George Bleck** · 17 Mar 2004, 10:32 AM

OK, the data does seem to fit the "tree" model.

Have you looked into the threaded background fill?

------------------
Every day I try to learn one thing new,
but new things to learn are increasing exponentially.
At this rate I’m becoming an idiot faster and faster !!!
------------------
George W. Bleck
Lead Computer Systems Engineer
KeySpan Corporation
My Email

**Chris Boss** · 17 Mar 2004, 12:10 PM

Michael,

Some suggestions:

The data file looks like it would be quite large. The best way
to load it is to avoid reading it line by line, but to read it
in using buffers. Assuming you have to load say 500,000 items
which are 50 characters each you would be loading about 23 meg
of data. By using 32 KB buffers to read the data file in blocks
it would take about 5 to 20 seconds to load (depending upon CPU
and speed of harddrive).

Now to fill the treeview with this much data would definitely
be problematic. Using what you posted, the "Functional group" level
would be 2 levels in the tree. The next levels beyond this
should not be filled in the tree.

One way to tackle this so the end user doesn't have to wait too
long is to use a tab control with say 10 tabs on it (1 to 10).

Put a Treeview control on each tab. Disable all the tabs on the
tab control and disable the treeview on the first tab. Now read
the data in quickly using high speed buffers (large blocks of data)
of at least 16 to 32 KB in size (after about 8 KB the buffer
size only increases speed of reading slightly).

Fill the first treeview until you reach some predefined limit.
Only fill the tree to the first two levels. The higher levels
should be in memory by this time and arranged in some kind of
data array (just not loaded into the treeview). Once done, enable
the first tab on the tab control plus the treeview associated
with that tab. The user can now access this data. When the user
selects a level not yet loaded (3rd level and above) load the
data into the treeview dynamically. By processing the TVN_ITEMEXPANDING
notification message, you can fill the next levels in the treeview
when needed. If the level is expanding, fill in the extra data.
If the level is collapsing then delete the items in that level.
By dynamically loading and unloading data as needed for levels
3 and above, the treeview should never become overloaded with too
much data.

Now continue reading in the rest of the data in the file and fill
the second treeview on tab 2 of the tab control. When it is filled
(up to level 2 and the rest in memory), then enable the second
tab on the tab control and its treeview.

Continue to do this until as many tabs/treeviews are filled to
handle all the data.

You could use a thread to fill each treeview , one at a time.
Since a treeview can't be accessed until it is filled and enabled
there shouldn't be a problem with the thread getting in the way
of user interaction. I would use a Global array for storing the
data. Using critical sections, a worker thread could be filling
the global array with data. When the array is filled, simply send
a message to the main thread (use sendmessage to force a contect
switch) and fill the treeview from the main GUI thread. Once done
sendmessage would return and the worker thread would continue on.

You can get fancy with the tab control, by using imagelists and
displaying one image for a loaded treeview/tab and another for
a treeview/tab in the process of being loaded and another for an
unused treeview/tab.

Assume the character R (ready) stands for the loaded image, L (loading) for the
process of being loaded image and U for unused, the tabs on
the tab control would lool like this to the end user:

Code:

_________________________________________________
|  R  |  L  |  U  |  U  |  U  |  U  |  U  |  U  |
|     |_____|_____|_____|_____|_____|_____|_____|_____
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|                                                    |
|____________________________________________________|

The end user is fully aware of what data is loaded, what
data is in the process of being loaded and what data does
not exist yet. As long as one of the tabs displays the
in processing of loading (L) image, the users knows the
data is still being read in.

------------------
Chris Boss
Computer Workshop
Developer of "EZGUI"
http://ezgui.com

[This message has been edited by Chris Boss (edited March 17, 2004).]

**Michael Mattias** · 17 Mar 2004, 01:59 PM

Now read the data in quickly using high speed buffers (large blocks of data)
of at least 16 to 32 KB in size (after about 8 KB the buffer
size only increases speed of reading slightly).

Who reads data from a file this way? I memory-map the file and parse it character by character. (You have to parse ANSI EDI data character-by-character anyway, so why not let Windows carry the load?)

Besides, my "worst case scenario" is a 30 MB file.. which results in one (1) interchange containing one (1) functional group containing one transaction set (1).

Unfortunately, the "segments" is where I have a little problem...Here is my scan report:

Code:

 EDI File Summary  Analysis of File S:\pbwin70\work\testdata\2551121.clp
Created : Tuesday, January 20, 2004  4:11 PM
Accessed: Thursday, February 26, 2004  6:00 AM
Written : Tuesday, January 20, 2004  4:11 PM

Interchange 000000069 Group Count     1
  Group ID HC # 1069 Transaction Count     1
    Transaction Set 837 #        1 Segment Count 1156344

 *** END OF REPORT ***

MCM

**Michael Mattias** · 18 Mar 2004, 05:52 AM

Prelim results:

Changing to external storage of text and specifying LPSTR_TEXTCALLBACK in psxText member of TV_ITEM : Load time reduced to less than five (5) percent of time required when control must store the text.

Ninety-Five (95) percent improvement is a good start.

**Michael Mattias** · 28 Mar 2004, 07:47 AM

..and just to demo the best source for performance improvement is design...

- Yes, I got the 95% improvement by going to LPSTR_TEXTCALLBACK and have stuck with that.

But - I got another 50% from what was left by..

- Changing the algorithm I used to expand the tree when initially presented. I was using a 'generic' treeview enumerator I had developed, but given the specific data possible, I was able to rewrite that to "never mind checking this, because that can never happen"
- I found a place where I had forgotten to disable the redraw when the display was being changed; now I always disable redraw, set up everything for user presentation, then enable redraw and force a repaint.
- When applicable, I moved a couple of operations which include a progress bar to a separate thread, and simply posted bar-update messages. So what if the bar is not quite in synch? Show me a user who can tell the difference between 50% complete and 55% percent complete when there is nothing but a bar (no numbers)? Better still, show me one who cares.
- I did larger memory allocations earlier, meaning fewer re-allocations (REDIM or HeapRealloc)).

No, I didn't even bother looking at how I did loop counters. I went where they keep the money.

[This message has been edited by Michael Mattias (edited March 28, 2004).]

**Brad D Byrne** · 28 Mar 2004, 07:54 AM

hehe funny!! MCM

------------------
Wash DC Area
Borje's "Poff's" is likely the BEST tool for learning PB.. http://www.tolkenxp.com/pb
and a few PBTool's:http://sweetheartgames.com/PBTools/PBTools.html

Announcement

Treeview Control Replacement?

Treeview Control Replacement?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment