NM_CUSTOMDRAW, slow!

[Chris Holbrook]

I'm redrawing a listbox just to put a background color on alternate line in "listing style". The underlying process, without using %NM_CUSTOMDRAW, takes 0.8 seconds or thereabouts. When the customdraw feature is added, it takes 2.4 seconds - that's madness, surely. Or more likely, my code (derived from MCM's example ISTR) is wrong:

'                CASE %NM_CUSTOMDRAW  ' set bg color'  
'                  IF LOWRD(CBWPARAM) <> %IDC_QR_LV THEN EXIT SELECT
'                    pnm = CBLPARAM
'                    SELECT CASE @pnm.nmcd.dwDrawStage
'                        CASE %CDDS_PREPAINT
'                            SetWindowLong CBHNDL,%DWL_MSGRESULT,%CDRF_NOTIFYITEMDRAW
'                            FUNCTION = 1: EXIT FUNCTION
'                        CASE %CDDS_ITEMPREPAINT
'                            SetWindowLong CBHNDL,%DWL_MSGRESULT,%CDRF_NOTIFYSUBITEMDRAW
'                            FUNCTION = 1: EXIT FUNCTION
'                        CASE %CDDS_SUBITEM OR %CDDS_ITEMPREPAINT
'                            FUNCTION = [B]QLVCustomDraw(BYVAL pnm)[/B]
'                            EXIT FUNCTION
'                    END SELECT

'------------------------------------------------------------------------------
FUNCTION QLVCustomDraw ( BYVAL ptlvcd AS NMLVCUSTOMDRAW PTR ) AS LONG

  LOCAL szItem          AS ASCIIZ * %MAX_PATH   ' item text
  LOCAL trcItem         AS RECT                 ' bounding rectangle of item/subitem
  LOCAL tlv_item        AS LV_ITEM              ' listview item information
  LOCAL tlb             AS LOGBRUSH             ' specifies information used to create background brush
  LOCAL hWndHdr         AS DWORD                ' handle of header child control
  LOCAL hBrush          AS DWORD
  LOCAL hBrushOld       AS DWORD
  LOCAL hPenOld         AS DWORD
  LOCAL dwBackColor     AS DWORD
  LOCAL nBkModeOld      AS INTEGER
  LOCAL hwin            AS DWORD
  LOCAL lresult         AS LONG

  ' Get the item or subitem info
  tlv_item.mask       = %LVIF_TEXT OR %LVIF_IMAGE OR %LVIF_STATE
  tlv_item.stateMask  = %LVIS_FOCUSED OR %LVIS_SELECTED
  tlv_item.iItem      = @ptlvcd.nmcd.dwItemSpec
  tlv_item.iSubItem   = @ptlvcd.iSubItem
  tlv_item.pszText    = VARPTR(szItem)
  tlv_item.cchTextMax = %MAX_PATH
  SendMessage @ptlvcd.nmcd.hdr.hwndFrom, %LVM_GETITEM, 0, BYVAL VARPTR(tlv_item)

  ' Get the bounding rectangle of the subitem(cell)
  trcItem.nLeft = %LVIR_BOUNDS
  trcItem.nTop  = @ptlvcd.iSubItem
  lresult = SendMessage( @ptlvcd.nmcd.hdr.hwndFrom, %LVM_GETSUBITEMRECT, @ptlvcd.nmcd.dwItemSpec, BYVAL VARPTR(trcItem))

  ' Color the background

    IF @ptlvcd.nmcd.dwItemSpec MOD 2 = 0 THEN
        dwBackColor = RGB(255,225,225)   ' background
    ELSE
        dwBackColor = %WHITE           ' white background
    END IF

  tlb.lbStyle = %BS_SOLID
  tlb.lbColor = dwBackColor
  tlb.lbHatch = 0
  hBrush = CreateBrushIndirect(tlb)
  FillRect @ptlvcd.nmcd.hdc, trcItem, hBrush
  DeleteObject hBrush

  ' Draw the text
  nBkModeOld = SetBkMode(@ptlvcd.nmcd.hdc, %TRANSPARENT)
  DrawTextEx @ptlvcd.nmcd.hdc, szItem, LEN(szItem), trcItem, %DT_SINGLELINE OR %DT_LEFT OR %DT_VCENTER OR %DT_END_ELLIPSIS, BYVAL %NULL
  SetBkMode @ptlvcd.nmcd.hdc, nBkModeOld

  FUNCTION = %CDRF_SKIPDEFAULT

END FUNCTION

[Kev Peel]

The callback function is called many many times, so perhaps declaring some variables (ie. szItem) as STATIC and initializing static values once will cut the time down? I tend to do this when using the ListView's sort function - by minimizing memory allocations, you can cut execution time considerably.

[Chris Holbrook]

The callback function is called many many times, so perhaps declaring some variables (ie. szItem) as STATIC and initializing static values once will cut the time down?

Didn't make a lot of difference. Commenting out various bits tells me that 50% of the delay is in the Drawtext, the rest in in initialization and two messages.

I think my listview will remain unadorned - it makes the database performance look bad!

[Kev Peel]

Did you try DT_NOCLIP? MSDN says thus:

DT_NOCLIP

Draws without clipping. DrawText is somewhat faster when DT_NOCLIP is used.

SetBkMode with transparency could be another contender for today's "deoptimization" prize

[Knuth Konrad]

Also: when doing huge fill operations, do a LockWindowUpdate(<Control>.hWnd) before you start and LockWindowUpdate(0) after control is filled. This speeds up this type of operation enormous. Use a progress bar/label instead, if you think you should give visual feedback to the user about the state of the current operation.

[Michael Mattias]

> IF LOWRD(CBWPARAM) <> %IDC_QR_LV THEN EXIT SELECT

Well, that's wrong. You should be checking the control ID versus @pmn.nmcd.hdr.idfrom Not that I can see that as the source of a performance problem, but as long as you are fixing this up.....

Also...
The Use of CBHNDL/CBWPARAM and " SetWindowLong CBHNDL,%DWL_MSGRESULT" are inconsistent. CBWParam and CBHNDL are DDT syntax, but DDT does not look at DWL_MSGRESULT at all.

If this is a control in a DDT-created dialog I think you need to return the CDRF_xxx constant directly from your dialog procedure. (Not a DDT guy).

[Michael Mattias]

Also, use SELECT CASE AS LONG instead of 'naked' SELECT CASE. (Fringe benefit: does full 32-bit compares, eliminating DWORD/LONG casting issues).

[Chris Boss]

I think your error may be:

FUNCTION = %CDRF_SKIPDEFAULT

That would work for a window procedure, but remember you are working with DDT (dialog procedure). %CDRF_SKIPDEFAULT needs to be returned via SetWindowLong using the DWL_MSGRESULT flag, like you did with the other messages.

Because you didn't do this correctly, the listview may be drawing the items twice, once using defaults and again using your draw code.

[Michael Mattias]

PROFILE is your friend here. It will show you all the entries and parameters to both your window procedure and your drawing procedure. It will also show you the relative times spent in both.

A little extra restructurning.. specifically, put ALL the WM_NOTIFY/NM_CUSTOMDRAW processing in its own procedure.. could help a lot. Then the PROFILE will tell you how many times you get WM_NOTIFY/NM_CUSTOMDRAW versus how many times your Draw procedure is called. Redundant calls should then stick out like a sore thumb.

[Chris Holbrook]

Did you try DT_NOCLIP?

Yes, I read that, but the impression created was of a saving of 25-35%in the Drawtext. This would be significant, and I will use it, though it alone is not the magic bullet I'm looking for.

[Michael Mattias]

> though it alone is not the magic bullet I'm looking for.

There is no magic bullet, Grasshopper.

Ever.

[Chris Holbrook]

Also: when doing huge fill operations, do a LockWindowUpdate(<Control>.hWnd) before you start and LockWindowUpdate(0) after control is filled. This speeds up this type of operation enormous.

Knuth, that is just the kind of saving I am looking for - enormous!

So let's try it - I place the Lockwindow commands around the other stuff I'm doing in the QLVCustomdraw process. Let's see, the elapsed time for the operation is reported at 29.10 seconds - oh-er, that is over 10 x the current worst case! The funniest part is, I left it running and returned after 5 1/2 hours - the screen is still being refreshed about 3 times per second! I assume that is a backlog of messages being processed.

So that has taught me two lessons - one, that the NM_CUSTOMDRAW message is sent many, many times, two, that there is an overhead to Lockwindow, as well as the benefits of which you are aware!

[Chris Holbrook]

I haven't yet implemented the advice of MCM and Chris Boss, but I shall. For the moment, I am satisfied that I have made two good decisions, first to not use NM_CUSTOMRAW for now, and second, to post my problem here - many thanks for your help chaps! And to MCM, also for an amazing cultural reference.

[Chris Holbrook]

So that has taught me two lessons

...and counting - I think that LockWindow may itself cause NM_CUSTOMDRAW to be sent, anyway, it is not a good thing to put in a NM_CUSTOMDRAW handler.

Why I think this is that my test query results in "only" 5530 calls to the function, whereas with Lockwindow wrapping the function, I killed the process after 54,000 approx. occurred!

Chris Boss - Thanks for your suggestion that the function might be returning the wrong value which might result in the default processing being applied AFTER I had painted the background, but I don't think so. My reason is that the background is correctly colored so the default color has been used, so default processing has not occurred. To test this, I replaced the callback function return value with %CDRF_DODEFAULT and as expected, the control was repainted with the default background color.

MCM - thanks for your various points. The return value seems to get back to windows (well to the Class handler I presume?) either by the SetWindowLong or by returning it via DDT, so I suppose that the class handler caches the value from SetWindowLong.


The nub of it seems to be that the subitem message (with CDDS_ITEMPREPAINT OR CDDS_SUBITEM) is called, in this case, 5530 times, about 100 times for each visible subitem! whereas the item message (with CDDS_TEMPREPAINT)is called only 158 times. Only five columns are in view, but windows sends the subitem mesage for every subitem in the row (there being 35 columns), not just for those in view, and appears to redraw everything when the listview is scrolled horizontally. Maybe the listview styles could affect this, I don't know -yet. It would also be good to know why the item message gets sent 158 times when only 15 rows are in view ( the listview has about 4K rows total).

The listview styles used are:

%ws_child OR %WS_TABSTOP OR %WS_VISIBLE OR %LVS_SHOWSELALWAYS OR %LVS_REPORT or %LVS_OWNERDATA
extended:
%DEFAULT_LV_EX_STYLES = %WS_EX_CLIENTEDGE OR %LVS_EX_GRIDLINES OR %WS_EX_LEFT OR %WS_EX_RIGHTSCROLLBAR

[Knuth Konrad]

...and counting - I think that LockWindow may itself cause NM_CUSTOMDRAW to be sent, anyway, it is not a good thing to put in a NM_CUSTOMDRAW handler.

Uuh, of course not. It should be used like this:

If IsTrue(LockWindowUpdate(<control>.hWnd)) Then

   Do 

      ' Fill Listbox with data

   Loop

   Call LockWindowUpdate(0)

End If

You call it *once* before a huge update operation that would cause the window to repaint itself over and over. You release it after you're done.

[Chris Holbrook]

It should be used like this:

The MSDN help is the best guide as to the applicability of LockWindowUpdate. It points out, inter alia, that :

LockWindowUpdate is not intended for general-purpose suppression of window redraw.

Which may explain the GPFs when I tried it in the context which you suggested. I've decided to draw a line under Lockwindowupdate.

The question remains, why does the message get sent 5530 times when there are only 75 subitems visible?

[Michael Mattias]

>or %LVS_OWNERDATA

Your problem may be caused by misuse of styles and messages.

When using LVS_OWNERDATA (virtual) style, the rules are different. eg, you ask in your drawing routine for LVIF_TEXT in ListView_getItem... except virtual listviews don't hold the text. The only data maintained by the control are the selected and focused states.

It may be sending WM_NOTIFY/LVN_GETDISPINFO each time in response to requests for text. (?)

You do not need OWNERDATA style to process NM_CUSTOMDRAW sequences. (Frankly I can't figure out how this is showing anything, since the control is not storing the text).

Show ALL WM_NOTIFY processing code for this control.


>The question remains, why does the message get sent 5530 times when there are only 75 subitems visible?

"Which" message? WM_NOTIFY? That gets sent for all kinds of reasons other than "you may draw me now." NM_CUSTOMDRAW? That gets sent for each visible item every time the control is scrolled.

[Kev Peel]

In ownerdata ListViews, the control will continuously request information about the display text. For one example, it will request the data for all items in order to get the display text extent used when sizing column headers and scrolling. When using this method, it is a good idea to keep all data easily accessible to the control, say in a large array or data block (on disk is very slow).

The owner-data style is much more intensive than the 'lesser' LPSTR_TEXTCALLBACK method, as you have to handle the attributes of all items, not just text. For example, sorting the ListView cannot be done using LVITEM structures, there is no lParam or item storage available, etc.

[Chris Holbrook]

Regarding the mesage counts which I previously gave, these are counts of the WM_NOTIFY/ NM_CUSTOMDRAW / CDDS_ITEMPREPAINT | CDDS_SUBITEM messages.

It may be sending WM_NOTIFY/LVN_GETDISPINFO each time in response to requests for text

yes it is, and I would expect it to do so, but in fact it is sending about twice as many LVN_DISPINFOs when custom draw is enabled, but that still does not explain why it is sending so many NM_CUSTOMDRAW / CDDS_ITEMPREPAINT | CDDS_SUBITEM messages in the first place!

You do not need OWNERDATA style to process NM_CUSTOMDRAW sequences.

No, but I need it to populate the visible part of the listview. the data source could have huge numbers of rows and storage is already allocated to them, so storing data in the listview is a very bad idea.

(Frankly I can't figure out how this is showing anything, since the control is not storing the text).

My guess is that the NM_CUSTOMDRAW etc results in an LVN_DISPINFO (the extra one) and the data are cached just during the NM_CUSTONDRAW message handling

Show ALL WM_NOTIFY processing code for this control.

just the LVN_DISPINFO stuff for this control:

               CASE %LVN_GETDISPINFO
                    IF LOWRD(CBWPARAM) <> %IDC_QR_LV THEN EXIT SELECT
                    INCR gdispinfo ' ................my debugging counter
                    lpLVDispInfo = CBLPARAM
                    @lpLVDispInfo.item.pszText = VARPTR(sz)
                    IF (@lpLVDispInfo.item.mask AND %LVIF_TEXT) THEN
                        @lpLVDispInfo.item.pszText = _
                            pqueryresulttable(((@lpLVDispInfo.item.iitem + 1)* sep.cols) + @lpLVDispInfo.item.iSubItem)
                    END IF

BTW the CBWPARAM/CBLPARAM references are correct.

The perfomance may be better if I redraw the background at the item level (NM_CUSTOMDRAW/CDDS_ITEMREPAINT), that's next on the list.

[Chris Holbrook]

For one example, it will request the data for all items in order to get the display text extent used when sizing column headers and scrolling.

That makes the numbers balance exactly, the difference between the number of subitem messages initially sent and the number of LVN_DISPINFO messages subsequently sent in resonse (I believe) to the nm_customdraw "subitem messages" is exaclty the no. of rows + no.of columns!

When using this method, it is a good idea to keep all data easily accessible to the control, say in a large array or data block (on disk is very slow).

Agreed.

The owner-data style is much more intensive than the 'lesser' LPSTR_TEXTCALLBACK method

thanks, I'll look it up!