Virtual ListView Scrolling Performance. Can it be improved?

[Erik Christensen]

Thanks a lot. I have downloaded your code – indeed an impressive
work. I will study it. Thanks also for your description of your
array sorting method. Very elegant. By the way, did you try the
index (in-place) sort I posted in source code? It is extremely
fast and effective. It could probably quite easily be adapted to
serve within your framework.

Erik


------------------

[Borje Hagsten]

Enitire POFFS2 code, icons, include files and all can now be downloaded
from http://www.tolken99.com/pb/poffswrk.zip

Somewhere in all that mess there is a files called PBVLIST.INC, and
in that file, under WM_PAINT, you can see one way of doing it. Also
look in poffs2.bas, under CheckListproc, where the checkbox list resides.
Ownerdrawn list - can do same kind of stunts there.

How I do it, is by using one string array, with each item separated by
a TAB character (comma, whatever can be used). When it's time to draw,
I parse out each item and paint them in proper place. Since only the
items on screen need to be handled, PB's PARSE$ is enough fast to make
scrolling, etc. pretty fast and smooth.

When/if it's time to sort based on column, I dimension a temporary buffert
array and in a loop, parse out proper item to that one. Then simple ARRAY
SORT on buffert array and TAGARRAY with main array tagged does the trick.
Code for this resides in poffs2.bas, in LblBackProc.

While not the most efficient way, it is fast enough for most needs. One day,
I will rewrite the list to handle multiple columns better, but until then,
this way can also do the job..


------------------


[This message has been edited by Borje Hagsten (edited September 30, 2001).]

[Erik Christensen]

Borje,

Thank you for your fine comments. You are right that the program
does a large amount of work in updating the list at every move,
but with the speed of present days computers, it is difficult to
see a significant delay. I will implement the thumbtrack code and
the update improvement proposed by you.

I have seen your fine virtual list in the source code forum and
many of your other fine programs. However, I did not think your
virtual list could also be used for a multicolumn listview-like
display. POFFS2 is certainly a very fine program, but is the
source code available for further study by someone like me?

Best Regards,
Erik



------------------

[Borje Hagsten]

Nice piece of work, Erik. Don't think you can get better performance,
since you have to clear list and add new items all the time. Not sure
how MS wrote those standard calls, but since listbox is old, code maybe
isn't very efficient (probably not changed since 1857, something..

Ownerdrawn list can do better. Can use multi-dimensional array and
paint all, including gridlines, etc, in WM_DRAWITEM. Only thing needed
is to use LB_INITSTORAGE to tell list it has so and so many items, so
no need for LB_RESETCONTENT or LB_ADDSTRING, etc. Very fast and you get
full control over drawing.

Own list from start is best, IMHO. Like Jules, I have used the Virtual List
I once posted to source code forum in many different ways. POFFS2 shows one
variation, with multi-column, grid-lines and different colors in columns,
etc. Easy to adjust it to own needs in WM_PAINT.

BTW, suggestion: Both %SB_THUMBTRACK and %SB_THUMBPOSITION have 16-bit limits
(if to use HI/LOWRD like that). To become truly virtual, it should be possible
to view millions of items. (well..) So, in ListDialogProc, under %WM_VSCROLL,
you can use the following instead: (no need for %SB_THUMBPOSITION)

      CASE %SB_THUMBTRACK 'enable tracking above 16-bit limit, >65536 lines..
         siY.cbSize = SIZEOF(siY)
         siY.fMask = %SIF_TRACKPOS
         CONTROL SEND hListForm&, %VertScrollbar, %SBM_GETSCROLLINFO, 0, VARPTR(siY)
         siY.nPos = siY.nTrackPos

Also, in UpdateWindowAndScrollbars - can never move up/down - sideways
at same time, so can use ELSEIF to save a milli-second there, like:

  IF xPos <> xPrevPos THEN     ' Moved in X-direction (horizontal)
     '..
  ELSEIF yPos <> yPrevPos THEN ' Moved in Y-direction (vertical)
     '..
  END IF

------------------

[Erik Christensen]

i did not succeed in improving the performance speed of scrolling.
in the meantime i have made a virtual multicolumn listbox which
simulates a listview like "grid control" in the source code forum
at this address:
http://www.powerbasic.com/support/pb...ad.php?t=23159

comments and suggestions for improvement should however go to the present
thread.

best regards,
erik


------------------

[Erik Christensen]

Hi Fellows,

Thanks to all of you for your fine suggestions. They look very
good. I will look more into it and present the result when it
is ready. I am quite busy with other matters right now.

Regards,

Erik

------------------


[This message has been edited by Erik Christensen (edited August 19, 2001).]

[Roberto Valois]

Erik,
You can use the message LVM_HITTEST to determine the first
(to the left) and the last (to the right) subitem visible.
Ignore the notification %LVN_GETDISPINFO message for all the
ones that are not in this range.
You do not need to determine the first and the last during the
notification process. These values are going to change only
whether the user resize or scroll to the left or scroll to the right or ...
(perhaps there are other situations).
This is not going to solve the problem, just to reduce time
expense in the case of the obtainment of the data be time
consuming. The problem is that there will be a message anyway
for each visible item. In the case of a ListView of 200 columns
with 25 items on screen means 5000 calls just to obtain the
text of each visible item.

RValois

------------------


[This message has been edited by Roberto Valois (edited August 18, 2001).]

[Michael Mattias]

IF (@lpLVDispInfo.item.mask AND %LVIF_TEXT) THEN 
  szString =  "Item " & STR$(@lpLVDispInfo.item.iItem) & _  
              " - Column " & STR$(@lpLVDispInfo.item.iSubItem) 
  @lpLVDispInfo.item.pszText = VARPTR(szString)
END IF

Well,one thing for sure, if you are building strings like this (concatenation) each time, that takes a lot of time.

I would try updating the LV control text using pre-built-strings, perhaps when you first insert the items to see what that does to performance.

I use this method in an LV I have (which is NOT owner-draw; rather, it is a simple callback for the label text) and on my 400Mhz machine I get no flicker at all.

MCM

[Jules Marchildon]

Erik;

I just want to add that all controls that are OwnerDraw or CustomDraw
capable will add painting&processing delays of some sort and that is one
of it's main disadvantages of this type of drawing.
(note: small sized controls don't seem to suffer, but the larger the control,
the slower the painting/processing and the more flickering while scrolling.)

The only way to improve on it is to make your own control. I have started a
Report Control (mimic's a listview in report mode) that is based on Borje Hagsten's
PBVLIST.INC custom control. I have only put a few hours into it to-date and
it may take several more weeks to complete (other priorities) but I will release
it to this forum once it is done. Right now, it's not a user friendly control
since there are so many features you can have inside the WM_PAINT handler, but
I hope to fix that once the control is near to my satisfaction.

'---
Added later after I tested your snippet out...

The quickest way to improve both problems is to update only the visible
window Rect. By looking at the response time I am sure you are updating the
columns that are not visible. How to do it using CustomDraw? I Don't have a clue
at this time.
'---

Regards,
Jules


[This message has been edited by Jules Marchildon (edited August 18, 2001).]

[Erik Christensen]

Thanks to Roberto Valois for his Virtual ListView program posted
in the source code forum July 26, 2001. With this elegant technique
the time otherwise used for filling the total listview grid with data
is eliminated. Only the lines presented in the listview window at
any one time are just being picked by listview when needed. This
is a major advantage with large amounts of data.

Another issue affecting performance speed of listview (virtual or
not) is the number of subitems or columns. With many columns the
scrolling speed is markedly reduced. The listview (virtual or
not) seems to process each entire row even if only few of the
subitems are actually being displayed at any one time.

The question is: How can you limit the listview processing to
displayed subitems and thereby improve the scrolling performance?

Regards,
Erik

[email protected]

The code below, which is a shortened version of the program of
Roberto Valois, illustrates the slow scrolling performance
(the column number is set to 234 which seems to be the maximum
allowed).

#DIM ALL
#COMPILE EXE
#INCLUDE "WIN32API.INC"
#INCLUDE "COMMCTRL.INC"
%ID_LISTVIEW = 300
'
' **********************
' You can change this to lower values to see the effect on scrolling performance
' Try for example %COLUMN_COUNT = 5  and see the difference.
%COLUMN_COUNT = 234
' **********************
%ITEM_COUNT = 20000 ' This number does not affect scrolling performance
'
DECLARE FUNCTION InitApplication() AS LONG
DECLARE FUNCTION InitInstance(LONG) AS LONG
DECLARE FUNCTION MainWndProc(BYVAL LONG, BYVAL LONG, BYVAL LONG, BYVAL LONG) AS LONG
DECLARE FUNCTION ListViewNotify(BYVAL LONG, BYVAL LONG) AS LONG
DECLARE FUNCTION CreateListView(BYVAL LONG, BYVAL DWORD) AS LONG
DECLARE SUB ResizeListView(BYVAL LONG, BYVAL LONG)
DECLARE SUB InitListView(BYVAL LONG)
DECLARE FUNCTION MakeFont(Font AS ASCIIZ, Charset AS LONG, Bold AS LONG, Italic AS LONG, PointSize AS LONG) AS LONG
'*******************************************************************************
GLOBAL g_hInst AS LONG
GLOBAL g_szClassName AS ASCIIZ * 32
GLOBAL hFont AS LONG
'*******************************************************************************
FUNCTION WINMAIN (BYVAL hInstance     AS LONG, _
                  BYVAL hPrevInstance AS LONG, _
                  lpCmdLine           AS ASCIIZ PTR, _
                  BYVAL nCmdShow      AS LONG) AS LONG
    LOCAL Msg AS tagMSG
    LOCAL i AS LONG, j AS LONG
    LOCAL InitCommCtrl AS INIT_COMMON_CONTROLSEX
    g_hInst = hInstance
    IF (ISFALSE(InitApplication())) THEN
        FUNCTION = %False
        EXIT FUNCTION
    END IF
    InitCommCtrl.dwICC = %ICC_LISTVIEW_CLASSES
    InitCommCtrl.dwSize = SIZEOF(InitCommCtrl)
    InitCommonControlsEx InitCommCtrl
    IF (ISFALSE(InitInstance(nCmdShow))) THEN
        FUNCTION = %False
        EXIT FUNCTION
    END IF
    WHILE GetMessage(Msg, %NULL, 0, 0)
        TranslateMessage Msg
        DispatchMessage Msg
    WEND
  FUNCTION = msg.wParam
END FUNCTION
'*******************************************************************************
FUNCTION InitApplication() AS LONG
    LOCAL wcex AS WNDCLASSEX
    g_szClassName = "GridClass"
    wcex.cbSize        = SIZEOF(wcex)
    wcex.style         = 0
    wcex.lpfnWndProc   = CODEPTR( MainWndProc )
    wcex.cbClsExtra    = 0
    wcex.cbWndExtra    = 0
    wcex.hInstance     = g_hInst
    wcex.hCursor       = LoadCursor( %NULL, BYVAL %IDC_ARROW )
    wcex.hbrBackground = GetStockObject( %WHITE_BRUSH )
    wcex.lpszMenuName  = %NULL
    wcex.lpszClassName = VARPTR( g_szClassName )
    wcex.hIcon         = LoadIcon( %NULL, BYVAL %IDI_APPLICATION )
    wcex.hIconSm       = LoadIcon( %NULL, BYVAL %IDI_APPLICATION )
    FUNCTION = RegisterClassEx (wcex)
END FUNCTION
'*******************************************************************************
FUNCTION InitInstance(nCmdShow AS LONG) AS LONG
    LOCAL hWnd AS LONG
    LOCAL szTitle AS ASCIIZ * 64
    szTitle = "Grid.bas - Virtual ListView Sample with Fonts and Colors"
    hWnd = CreateWindowEx(  0, _
                            g_szClassName, _
                            szTitle, _
                            %WS_OVERLAPPEDWINDOW, _
                            %CW_USEDEFAULT, _
                            %CW_USEDEFAULT, _
                            %CW_USEDEFAULT, _
                            %CW_USEDEFAULT, _
                            BYVAL %NULL, _
                            BYVAL %NULL, _
                            g_hInst, _
                            BYVAL %NULL)
    IF (ISFALSE(hWnd)) THEN
        FUNCTION = %False
        EXIT FUNCTION
    END IF
    ShowWindow hWnd, nCmdShow
    UpdateWindow hWnd
    FUNCTION = %True
END FUNCTION
'*******************************************************************************
FUNCTION MainWndProc (  BYVAL hWnd AS LONG, _
                        BYVAL uMessage AS LONG, _
                        BYVAL wParam AS LONG, _
                        BYVAL lParam AS LONG) EXPORT AS LONG
    STATIC hwndListView AS LONG
    LOCAL pnmh AS NMHDR PTR
    SELECT CASE uMessage
        CASE %WM_CREATE
            hwndListView = CreateListView(hWnd, %ID_LISTVIEW)
            InitListView hwndListView
            hFont =MakeFont ("Times New Roman", %ANSI_CHARSET, %FW_BOLD, %False, 12)
        CASE %WM_NOTIFY
            pnmh = lParam
            IF @pnmh.idFrom = %ID_LISTVIEW THEN
                FUNCTION = ListViewNotify(@pnmh.hwndFrom, lParam)
            END IF
            EXIT FUNCTION
        CASE %WM_SIZE
            ResizeListView hwndListView, hWnd
        CASE %WM_COMMAND
            SELECT CASE LOWRD(wParam)
            END SELECT
        CASE %WM_DESTROY
            PostQuitMessage 0
    END SELECT
    FUNCTION = DefWindowProc(hWnd, uMessage, wParam, lParam)
END FUNCTION
'*******************************************************************************
FUNCTION CreateListView(    BYVAL hwndParent AS LONG, _
                            BYVAL ListViewID AS DWORD) AS LONG
    LOCAL dwStyle AS DWORD
    LOCAL dwExStyle AS DWORD
    LOCAL hwndListView AS LONG
    dwStyle =   %WS_TABSTOP OR _
                %WS_CHILD OR _
                %WS_BORDER OR _
                %WS_VISIBLE OR _
                %LVS_AUTOARRANGE OR _
                %LVS_REPORT OR _
                %LVS_OWNERDATA     'Virtual ListView will request for items when needed
                                   'through %LVN_GETDISPINFO message

    hwndListView = CreateWindowEx ( %WS_EX_CLIENTEDGE, _
                                    $WC_LISTVIEW, _
                                    "", _
                                    dwStyle, _
                                    0, _
                                    0, _
                                    0, _
                                    0, _
                                    hwndParent, _
                                    ListViewID, _
                                    g_hInst, _
                                    BYVAL %NULL )
    IF (ISFALSE(hwndListView)) THEN
        FUNCTION = %NULL
        EXIT FUNCTION
    END IF
    ResizeListView hwndListView, hwndParent
    dwExStyle = %LVS_EX_GRIDLINES OR %LVS_EX_FULLROWSELECT
    ListView_SetExtendedListViewStyleEx hwndListView, dwExStyle, dwExStyle
    FUNCTION = hwndListView
END FUNCTION
'*******************************************************************************
SUB ResizeListView( BYVAL hwndListView AS LONG, _
                    BYVAL hwndParent AS LONG)
   LOCAL rc AS RECT
   GetClientRect hwndParent, rc
   MoveWindow   hwndListView, _
                rc.nleft, _
                rc.ntop, _
                rc.nright - rc.nleft, _
                rc.nbottom - rc.ntop, _
                %TRUE
END SUB
'*******************************************************************************
SUB InitListView (BYVAL hwndListView AS LONG)
    LOCAL lvColumn AS LV_COLUMN
    LOCAL i AS LONG
    LOCAL szString AS ASCIIZ * 16

    lvColumn.fmt = %LVCFMT_LEFT
    lvColumn.cx = 130
    lvColumn.mask = %LVCF_FMT OR _
                    %LVCF_WIDTH OR _
                    %LVCF_TEXT OR _
                    %LVCF_SUBITEM
    FOR i = 0 TO %COLUMN_COUNT
        szString = "Column " & STR$(i)
        lvColumn.pszText = VARPTR(szString)
        ListView_InsertColumn hwndListView, i, lvColumn
    NEXT i
   ListView_DeleteAllItems hwndListView
   ListView_SetItemCountEx hwndListView, %ITEM_COUNT, %LVSICF_NOINVALIDATEALL
END SUB
'*******************************************************************************
FUNCTION ListViewNotify(    BYVAL hwndListView AS LONG, _
                            BYVAL lParam AS LONG) AS LONG
    LOCAL pnmh AS NMHDR PTR
    LOCAL lpLVDispInfo AS LV_DISPINFO PTR
    LOCAL row AS LONG, column AS LONG
    LOCAL lplvcd AS NMLVCUSTOMDRAW PTR
    LOCAL szString AS ASCIIZ * 256
    pnmh = lParam
    SELECT CASE @pnmh.code
        CASE %NM_CUSTOMDRAW ' = -12
            lplvcd = lParam
            IF(@lplvcd.nmcd.dwDrawStage = %CDDS_PREPAINT) THEN
                FUNCTION = %CDRF_NOTIFYITEMDRAW
                EXIT FUNCTION
            END IF
            IF(@lplvcd.nmcd.dwDrawStage =  %CDDS_ITEMPREPAINT)  THEN
              '  IF (@lplvcd.nmcd.dwItemSpec MOD 2) = 0 THEN
              '      SelectObject @lplvcd.nmcd.hdc, hFont        'Item Font
              '  ELSE
              '      @lplvcd.clrTextBk = RGB(200,200,200)        'Item Text Background Color
              '      @lplvcd.clrText = RGB(128,0,0)              'Item Text Color
              '  END IF
                FUNCTION = %CDRF_NEWFONT                        'Return CDRF_NOTIFYSUBITEMREDRAW
                                                                'to customize the item's subitems individually then
                                                                'case CDDS_SUBITEM | CDDS_ITEMPREPAINT
                EXIT FUNCTION
            END IF
        CASE %LVN_GETDISPINFO '= -150   'Virtual ListView ask for Item text
            lpLVDispInfo = lParam
            IF (@lpLVDispInfo.item.mask AND %LVIF_TEXT) THEN
                szString =  "Item " & STR$(@lpLVDispInfo.item.iItem) & _
                            " - Column " & STR$(@lpLVDispInfo.item.iSubItem)
                @lpLVDispInfo.item.pszText = VARPTR(szString)
            END IF
        CASE %LVN_ODCACHEHINT
    END SELECT
    FUNCTION = 0
END FUNCTION
'*******************************************************************************
FUNCTION MakeFont(Font AS ASCIIZ, Charset AS LONG, Bold AS LONG, Italic AS LONG, PointSize AS LONG) AS LONG
      LOCAL hDC      AS LONG
      LOCAL CyPixels AS LONG
      hDC = GetDC(%HWND_DESKTOP)
      CyPixels  = GetDeviceCaps(hDC, %LOGPIXELSY)
      ReleaseDC %HWND_DESKTOP, hDC
      FUNCTION = CreateFont(MulDiv(PointSize, CyPixels, 72), 0, 0, 0, Bold, Italic, 0, 0, _
         CharSet, %OUT_TT_PRECIS, %CLIP_DEFAULT_PRECIS, %DEFAULT_QUALITY, %FF_DONTCARE, Font)
END FUNCTION

[email protected]
------------------




[This message has been edited by Erik Christensen (edited August 18, 2001).]