RichEdit Character Formatting Speed Improvement?

[Gary Beene]

In an earlier post I whined about the speed of the syntax highlighting algorithm I was using. In looking closer, it appears that the RichEdit control is perhaps the real culprit. I've seen hints about this before, but only this morning ran a speed test to get more details.

Within the syntax highlighting loop, the following code is used to set the color of selected text in a RichEdit control. It's part of the original code legacy from Borje.

This code takes up 80% of the time in the syntax highlighting loop!

The actual code to separate out the keywords/comments/quotes in a large snippet was 0.7 seconds (20%) out of a total 3.9 seconds - still not instantaneous, but much less of the problem that I might have expected.

So, the question is, does anyone have a suggestion on how to significantly speed up this code?

Function setRichTextColor( ByVal NewColor As Long) As Long
   Local cf As CHARFORMAT
   cf.cbSize      = Len(cf)       'Length of structure
   cf.dwMask      = %CFM_COLOR    'Set mask to colors only
   cf.crTextColor = NewColor      'Set the new color value
   SendMessage(hRichEdit, %EM_SETCHARFORMAT, %SCF_SELECTION, VarPtr(cf))
End Function

[Gary Beene]

BTW, I left off some important information - code elsewhere in the loop that disables the redraw of the RichEdit control until all highlighting is applied.

This code runs before the syntax highlighting loop.

  'Disable the event mask, for better speed
  xEvents = SendMessage(hRichEdit, %EM_GETEVENTMASK, 0, 0)
  SendMessage(hRichEdit, %EM_SETEVENTMASK, 0, 0)

  'Turn off redraw for faster and smoother action
  SendMessage(hRichEdit, %WM_SETREDRAW, 0, 0)

And this code runs at the end of the loop

  'Turn on Redraw again and refresh - this one causes some flicker in Richedit..
  SendMessage hRichEdit, %WM_SETREDRAW, 1, 0
  InvalidateRect hRichEdit, ByVal %NULL, 0 : UpdateWindow hRichEdit

  'Reset the event mask
  If xEvents Then SendMessage(hRichEdit, %EM_SETEVENTMASK, 0, xEvents)

[Gary Beene]

And, continuing the conversation with myself , applying syntax highlighting only to the visible lines could be a work around.

I'm not sure what message is fired when the display exposes additional/different lines, but I'll go check that out.

If I'm scrolling or paging up/down the page, I'm also not sure if the "syntax highlight visible lines only" approach would be fast enough to keep up with the scrolling action. I'll have to test that also.

And, such an approach would mean that if I want to do something with text that is not visible, I have to code a way to apply syntax in such cases.

Overall, it would be much simpler if the speed of applying character formatting to the entire RichEdit control could be speeded up significantly.

[Gösta H. Lovgren-2]

And, continuing the conversation with myself ,

Of course you are. Post times:
11:32
11:44
11:52

That speed talking in Forum Time. You don't take a breath long enough to let anyone else get a word in edgewise {deep sigh}

==========================================
"God is a comedian playing to an audience
too afraid to laugh."
Voltaire (1694-1778)
==========================================

[Michael Mattias]

I'm not sure what message is fired when the display exposes additional/different lines, but I'll go check that out

EN_VSCROLL, EN_HSCROLL

Overall, it would be much simpler if the speed of applying character formatting to the entire RichEdit control could be speeded up significantly

Maybe you should consider using something other than a RichEdit control?

Or maybe you should handle highlighting "as each character is typed?"

Or, maybe you could ask one of the third-party IDE vendors how they do keyword highlighting? (I'm not saying they will tell you, but asking is free).

MCM

[Gary Beene]

True, Gosta.

Well, as you know, in these forums a mistake can be quickly pointed out - to the poster's embarrassment. So if a thought occurs which could remove the public thrashing - better sooner than later.

Plus, as I was telling my wife earlier, sometimes just in the asking of a question, the effort to putting it forward logically sometimes makes a person think of a possible solution or of an aspect to the solution that hadn't occurred to them before.

So it is with posts. Just writing down the question can make you (me in this case) think of something pertinent to the discussion that I want to add to the thread.

[Gary Beene]

Hi MCM,

Part of the code is character by character. So for editing within the control on a single line, the speed is excellent.

But when switching from one snippet to another, an entire snippet of code gets highlighted. Also, pasting a big block of code is another example of when character-by-character has to give way to block highlighting.

As for asking other 3rd party vendors, I'd be happy to do that. But if you read my post about proud aliens from a few months back, you'll understand that I feel the need to make a valid attempt on my own before asking for help (well, ok, so I sorta break the rule with this post).

I've also thought to look at other controls (EDM32 and Scintilla in particular) but worry that too much of my existing code would be broken. Those are still on the table as solutions, but I just haven't yet given up on the option of using the RichEdit control. It works well enough, but with a few areas I'd still like to see improvement in (not to mention the line numbering and alternating line color desires I mentioned in an earlier thread).

Thanks for the thoughts!

[Michael Mattias]

I've also thought to look at other controls (EDM32 and Scintilla in particular) but worry that too much of my existing code would be broken

And to think so many people thought George W. Bush was stubborn.....

[Gary Beene]

Ouch! That shoe fits way too well!

.... and did you know - he just lives about 2-3 miles from my house (I live in N. Dallas).

[Chris Holbrook]

Gary, isn't there a user wordwrap function which can be called for every word by the Richedit control? Maybe this could be used to decide which color to use also. I'm a bit hazy on this as it has been a while since I was messing about with it. Also I've been having weird dreams lately.

PS Does GB know he lives near you?

[Edwin Knoppert]

A single word being converted doesn't eat much time isn't?
In your case you may try to color the RTF data itself and insert that as 'selection'.

I have been busy with RTF controls regarding syntax highlighting, it is not working properly, slow or the RTF does not draw properly (bleeding over the NC edge, measuring sizes are so so..)
In that time i saw an RTF control based version which drew on the DC and was very fast.
I never understood that assembly code where the specific speed came from unf.
I tried but i could not get that part right.
Imo it had nothing to do with assembly coding itself.

The suggestion being made to write the control yourself is actually the best.
It's pretty easy to get started with a caret being set.
After that the basics need to get in of course like storing characters, much more code..

[Gary Beene]

Hi Chris,

Yes, there are some RichEdit functions which jumps from word to word - as best I can tell there's no way to decide which characters constitute a word. I played around with it the other day, and it worked well, for what it did. It just seemed to have too many limitations to be a generic solution for syntax highlighting.


Edwin,
You may be right - a custom control being the way to go. I wasn't wanting to be an expert on an edit control - just a user. But even with the RichEdit there's a lot to learn. With a custom control, as you noted, the work would be even greater.

MCM,
Thinking about it, my comment to Edwin is probably more the reason why I seem "stubborn". I want an edit control to use, not one that takes learning/development time away from doing things with the control. I've kept thinking that the RichEdit, with it's years of development, was a good way to go. But it's limitations are making me put more time into it than I had wanted. Had I started out with a different control I might have been happier!

[Michael Mattias]

>as best I can tell there's no way to decide which characters constitute a word.

REGEXPR/REGREPL, "\b" metacharacter?

[Gary Beene]

MCM,

Thanks for the reminder about using RegEx for word detection in a text string, but I was referring to the limitations of the built-in word detection capability of the RichEdit control, the EM_FindWordBreak message.

But, in looking at the documentation more closely, I see that there's perhaps more flexibilty in the RichEdit control, as far as built-in work detection, than I thought. Here's the MSDN text that tells more - but I've not dug into it just yet.

Word and Line Breaks

A rich edit control calls a function called a word-break procedure to find breaks between words and to determine where it can break lines. The control uses this information when performing word-wrap operations and when processing CTRL+LEFT ARROW key and CTRL+RIGHT ARROW key combinations. An application can send messages to a rich edit control to replace the default word-break procedure, to retrieve word-break information, and to determine what line a given character falls on.
Word-break procedures for rich edit controls are similar to those for edit controls, but they have additional capabilities: word-break procedures for both kinds of controls can determine whether a character is a delimiter and can find the nearest word break before or after the specified position. A delimiter is a character that marks the end of a word, such as a space. Usually, in an edit control, a word break occurs only after delimiters. However, different rules apply to most Asian languages.
Word-break procedures for rich edit controls also group characters into character classes, each identified by a value in the range 0x00 through 0x0F. Breaks occur either after delimiters or between characters of different classes. Thus, a word-break procedure with different classes for alphanumeric and punctuation characters would find two word breaks in the string "Win.doc" (before and after the period).
A character's class can be combined with zero or more word-break flags to form an 8-bit value. When performing word-wrap operations, a rich edit control uses word-break flags to determine where it can break lines. Rich Edit uses the following word-break flags.
FlagsDescriptionWBF_BREAKAFTERLines may be broken after the character.WBF_BREAKLINEThe character is a delimiter. Delimiters mark the ends of words. Lines may be broken after delimiters.WBF_ISWHITEThe character is a white-space character. Trailing white-space characters are not included in the length of a line when wrapping.The WBF_BREAKAFTER value is used to allow wrapping after a character that does not mark the end of a word, such as a hyphen.
You can replace the default word-break procedure for a rich edit control, with your own procedure by using the EM_SETWORDBREAKPROC message. For more information about word-break procedures, see the description of the EditWordBreakProc function.
Note This replacement is not recommended for Rich Edit control 2.0 and later, due to the complexity of multilingual word breaking.
For Rich Edit 1.0, you can use the EM_SETWORDBREAKPROCEX message to replace the default extended word-break procedure with an EditWordBreakProcEx function. This function provides additional information about the text, such as the character set. You can use the EM_GETWORDBREAKPROCEX message to retrieve the address of the current extended word-break procedure. Note that Rich Edit 2.0 and later do not support EditWordBreakProcEx, EM_GETWORDBREAKPROCEX, and EM_SETWORDBREAKPROCEX.
You can use the EM_FINDWORDBREAK message to find word breaks or to determine a character's class and word-break flags. In turn, the control calls its word-break procedure to get the requested information.

[Gösta H. Lovgren-2]

why I seem "stubborn".

When some people use the word "stubborn" to describe you, what they really mean is "Please stop doing what you are doing or you just might prove me wrong." (Hey, that sounds like it just might be another "saying" {laughing}).

=============================================
Not from the whole wide world I chose thee,
Sweetheart,
Light of the land and the sea!
The wide, wide world could not inclose thee,
For thou art the whole wide world to me.
Richard Watson Gilder
=============================================

[Michael Mattias]

I'd like a nickel for every time I've had to go to "Plan B" because "Plan A" - in which I had a total, absolute, unfettered and unshakeable faith - crapped out.

[Stanley Durham]

MM made post at 10:31 AM
google "richedit syntax highlight"
this page is on results, page #2

[Gary Beene]

? Stanley, I had trouble with the directions.

I googled the phrase, but nothing on page 2 had MM on it.

Tell me more.

[Stanley Durham]

google "richedit syntax highlight"
go to page 2
halfway down page, click this link;
"RichEdit Character Formatting Speed Improvement? - PowerBASIC Peer ..."