SourceCode Only

[StanHelton]

Stan,



We probably ought to put our collective heads together to come up with the definitive set of tools so we don't overlap efforts.

I'll start a new thread for tools. Let's think about putting them under the hood in vb2pb so the user won't get frustrated at the outset. Command buttons to activate.

[StanHelton]

New gConcordance() array -- Comments please

Guys,

New definition of gConcordance(). Makes it possible to track VB6 project files and corresponding PB code. I haven't
fully determined the exact usage, but I'm working on it today.

John,
How difficult would it be for the Editor to pass the vbPathFileName and vbLineNo within the file along with the code block? I can make the Tokenizer keep track, but it seems to me that the Editor would be a more logical place to keep track of the VB files and line nos. Then the Tokenizer will keep track of which PB code goes with which converted VB code block.

gConcordance() is a GLOBAL array. Size will vary significantly based on the size of the VB project being converted. Also, I'm thinking this will be more efficient if vbCodeString and pbCodeString are changed to pointers. Then we could use a dynamic array for the code blocks and eliminate the limitation of size with a single conversion.

TYPE ConcordanceType                    'use in Concordance() to keep track of which VB lines matche which PB lines
    vbPathFileName AS STRING * 256      'to control all VB6 files in project
    vbLineNo AS LONG                    'to contorl lines in each VB6 file
    vbCodeString AS STRING * 256        'the active VB6 code block
    VB2PB_ConversionLevel AS LONG       'to control how deep to go with conversion
    pbPathFileName AS STRING * 256      'to control all PB files in project
    pbLineNo AS LONG                    'to control lines in each PB file
    pbCodeString AS STRING * 256        'the active PB code block
END TYPE

[John R. Heathcote]

Stan,

How difficult would it be for the Editor to pass the vbPathFileName and vbLineNo within the file along with the code block?

The file name is no problem since it is part of the title caption of the MDI dialog box. The line number can be passed as well since that is how I read the file from the MDI dialog as well. Does the Tokenizer require the line number to be absolute, or does it have to be relative to the procedure? I would assume the line number is absolute, but either way this value can be passed no problem.

gConcordance() is a GLOBAL array. Size will vary significantly based on the size of the VB project being converted.

The UDT looks good.

pbCodeString AS STRING * 256        'the active PB code block

For most lines this will be adequate, but how will you handle continued lines that would span more than 256 characters?

Also, I'm thinking this will be more efficient if vbCodeString and pbCodeString are changed to pointers. Then we could use a dynamic array for the code blocks and eliminate the limitation of size with a single conversion.

I don't know of any way to pass a pointer that points to a specific line in the Scintilla control. The only pointer I know of is the one to provide direct and faster access to the control itself, but not the data it contains. I will research this further and let you know my results.

Given the above limitations would it be better if the editor keeps track of some of this information? My thought here is since the source code line(s) and their location is extracted from the editor all we would really need is the conversion supplied by the Tokenizer, then let the editor determine where the converted code should be inserted in the file.

That way there would be reduced or no duplication of effort, or am I way off based here?

[StanHelton]

...
Does the Tokenizer require the line number to be absolute, or does it have to be relative to the procedure? ...

Absolute within the VB file. I want to use these as reference points for x-referenced procedures, variables, & etc. in the VB code.

The UDT looks good.

Let's change the members pbCodeString to ppbCodeString and vbCodeString to pvbCodeString and define them as pointers. It means 2 more arrays for the actual code, but increases the efficiency of the index and therefore the conversion as long as the Tokenizer keeps them all properly indexed -- easy enough to do.

...
For most lines this will be adequate, but how will you handle continued lines that would span more than 256 characters?

By making these 2 members pointers to elements of dynamic string arrays the length limit is removed. I've set it up to catch the line continuation character ("_") and treat logical lines as one physical line for the actual conversion.

If the Tokenizer passes a really long PB string back to the Editor will that be a problem? In other words, perhaps the Editor should parse long logical lines into physical lines in the text. I think the Tokenizer should deal with logical lines and let the Editor handle all the formatting details. There shouldn't be any problem with line numbers if they refer to the first physical line in a logical line. Don't even have to be sequential so long as each line number is larger by at least 1 than the previous line number.

I don't know of any way to pass a pointer that points to a specific line in the Scintilla control. The only pointer I know of is the one to provide direct and faster access to the control itself, but not the data it contains. I will research this further and let you know my results.

Just a thought: the messaging system could send the actual line number without having to deal with pointers in the Editor. whadayathink ... maybe?

Given the above limitations would it be better if the editor keeps track of some of this information? My thought here is since the source code line(s) and their location is extracted from the editor all we would really need is the conversion supplied by the Tokenizer, then let the editor determine where the converted code should be inserted in the file.

Good point there. I have a rather fuzzy answer, but I want to review the Tokenizer code before I go trying to explain how it works -- it's been a long Monday. The code has evolved quite a bit over the last month and I made several changes this weekend after discussion in this forum. Let me give you a more cogent answer tomorrow when my brain gets back from the cleaners .... oops, didn't mean to let that slip.

[John R. Heathcote]

Stan,

I've taken a look at the Scintilla documentation and there doesn't appear to be a way to pass a pointer to access the text within the control the way (I think) you envisioned. A pointer is passed, but it is to a dynamic string that is already created. The following code illustrates how you read text line by line:

  k = 0
  nLine = -1
  'Number of lines
  nLines = SciMsg(pSciVB, %SCI_GETLINECOUNT, 0, 0)

  redim uControl(0 to 25) as VBProperties
	
  do
    incr nLine
			
      if nLine > nLines - 1 then exit do

    nLen = SciMsg(pSciVB, %SCI_LINELENGTH, nLine, 0)
    sRecord = SPACE$(nLen)
    SciMsg pSciVB, %SCI_GETLINE, nLine, STRPTR(sRecord)

    sRecord = utl_ReplaceSpecialChars(sRecord)

      if left$(trim$(sRecord), 20) = "Attribute VB_Name = " THEN exit do

The line:

SciMsg pSciVB, %SCI_GETLINE, nLine, STRPTR(sRecord)

Has two pointers, pSciVB to provide direct access to the Scintilla control message functionality and STRPTR(sRecord) to point to the dynamic string "sRecord" which has already been created. "sRecord" could also be an element of an array. So it appears to me that the Tokenizer will have to use the text retrieved from the Scintilla control as individual strings or an array and passed as a parameter accordingly. Or we could create a global string array called gsVBSOURCE() and have the Tokenizer read this specific array so we wouldn't have to pass it as a parameter. While these solutions point out a limitation of the Scintilla control, I don't think it is a serious one.

If the Tokenizer passes a really long PB string back to the Editor will that be a problem?

Not really, I can break the lines at a logical place for formatting purposes. This does have the potential to invalidate any indexing you may be tracking on the fly by line number by altering the number and content of the lines written back out to the editor, after conversion changes in the text, re-formatting long lines, etc.

In other words, perhaps the Editor should parse long logical lines into physical lines in the text. I think the Tokenizer should deal with logical lines and let the Editor handle all the formatting details.

Agreed. I think I have some code in another routine to identify a range of continued lines, shouldn't be too hard to convert this to return one physical line of text. I'm assuming here that the reformatted physical lines of text are the ones sent to the Tokenizer and not shown in the editor, correct?

There shouldn't be any problem with line numbers if they refer to the first physical line in a logical line. Don't even have to be sequential so long as each line number is larger by at least 1 than the previous line number.

Not a problem. If you take a look at my code snippet I set a variable called "nLine" to sequentially access the lines of text within the Scintilla control. The value of "nLine" will always comply with the Tokenizer requirement. Scintilla demands "nLine" be zero based, is this a problem?

Just a thought: the messaging system could send the actual line number without having to deal with pointers in the Editor. whadayathink ... maybe?

I think we need to determine the standard of how we intend to pass the VB source code text to the Tokenizer before we get caught up in the above details.

Good point there. I have a rather fuzzy answer, but I want to review the Tokenizer code before I go trying to explain how it works -- it's been a long Monday.

Fair enough.

While fiddling around with some VB form stuff I came to the conclusion that the original VB code should be treated as sacred, IOW it shouldn't be changed at all. Taking this precaution will give the user the ultimate fall back position should something go awry during the conversion process. With this in mind I set up a process whereby should the user load the VB source code in the editor another MDI edit dialog is created for the output. Will this play havoc with Tokenizer tracking and indexing?

If it does, I can always re-read the converted procedure, send it to the Tokenizer, and re-index the necessary stuff. Does that work for you? As a thought: (Gee, I didn't have to take a shower to realize it ) maybe we should have a specific indexing procedure as part of the Tokenizer that could be called upon to do its function at any time during conversion? This may be a bit cleaner to (re)index the necessary things in the source code after most of the conversion is completed and we know where critical items are located in the source code.

BTW, how was your trip to our fair state? Did you get to do any traveling?

[StanHelton]

...
I've taken a look at the Scintilla documentation and there doesn't appear to be a way to pass a pointer ...

Not a problem. I've defined a GLOBAL string (gvbCodeBlock) to hold the text you pass to me. The Tokenizer uses pointers internally so the Editor doesn't have to worry about them for this. When the block is converted the PB code is in gpbCodeBlock -- no changes to the VB block, we just read it from the Tokenizer. So if you know where the VB block is supposed to go all the the Editor has to do is keep things in order. Tokenizer is set up so that it doesn't care how big or small the VB block is.

BTW: If you have a more descriptive name for these 2 globals (only 1 letter difference in my names)...

...
Or we could create a global string array called gsVBSOURCE() and have the Tokenizer read this specific array so we wouldn't have to pass it as a parameter. While these solutions point out a limitation of the Scintilla control, I don't think it is a serious one.

Doesn't have to be an array. And I see my great idea is actually your's slightly modified.

...
This does have the potential to invalidate any indexing you may be tracking on the fly by line number by altering the number and content of the lines written back out to the editor, after conversion changes in the text, re-formatting long lines, etc.

I think the Editor is a more logical place for that, what with the formatting and all. Tokenizer will just read what it's given and post back in the PB block string.

The PB code will come back to you as logical lines w/o line continuation chars unless you think otherwise. That gives you more freedom with the formatting. We already introduced the $CRLF substitution; I think maybe we should keep internal translations to a minimum between modules. Just let the Tokenizer do it's thing within it's own space and give you back something that won't add extra work (ie.: a text string with $CRLF line terminators).

So you'll put logical lines in the VB code block and I'll put logical lines in the PB code block.

...
I'm assuming here that the reformatted physical lines of text are the ones sent to the Tokenizer and not shown in the editor, correct?

Yes, absolutely.

... "nLine" to sequentially access the lines of text ... The value of "nLine" will always comply with the Tokenizer requirement. Scintilla demands "nLine" be zero based, is this a problem?

Not a problem. Good thing I know about it though. (Note to self: option base zero, option base zero, option base ze... :deal

I think we need to determine the standard of how we intend to pass the VB source code text to the Tokenizer before we get caught up in the above details.

I agree completely! If we keep following the KISS principle, those details don't even happen.

... I came to the conclusion that the original VB code should be treated as sacred, IOW it shouldn't be changed at all. ... With this in mind I set up a process whereby should the user load the VB source code in the editor another MDI edit dialog is created for the output. Will this play havoc with Tokenizer tracking and indexing?

Not a problem. After reading your post (with a fresh clean mind) I realized the Tokenizer needs to do 98+% of it's stuff within it's own space. That way we don't have to pass icky things like pointers around between our modules.

...
As a thought: (Gee, I didn't have to take a shower to realize it ) maybe we should have a specific indexing procedure as part of the Tokenizer that could be called upon to do its function at any time during conversion? This may be a bit cleaner to (re)index the necessary things in the source code after most of the conversion is completed and we know where critical items are located in the source code.

I agree. It's on the list. (Note to Group: any volunteers?)

BTW, how was your trip to our fair state? Did you get to do any traveling?

The trip was an easy one, once I got on the plane. The weather is beautiful! I've got a very good view of the mountains from my hotel. (Is that THE mountains or just the foothills?) No snow in Boulder so far, but there were white capped peaks in the distance this weekend. Been awfully busy the past weeks, but I'm planning a trip to Rocky Mountain State Park on Saturday. Got a brand new camera just for this!

[John R. Heathcote]

Stan,

I agree that the editor and Tokenizer should do what they do best, that will serve to clean up many programming issues.

I also agree that the gvbCodeBlock and gpbCodeBlock variable names are too close to each other. How about renaming gvbCodeBlock to gvbSource?

I think the Editor is a more logical place for that, what with the formatting and all. Tokenizer will just read what it's given and post back in the PB block string.

The PB code will come back to you as logical lines w/o line continuation chars unless you think otherwise. That gives you more freedom with the formatting. We already introduced the $CRLF substitution; I think maybe we should keep internal translations to a minimum between modules. Just let the Tokenizer do it's thing within it's own space and give you back something that won't add extra work (ie.: a text string with $CRLF line terminators).

So you'll put logical lines in the VB code block and I'll put logical lines in the PB code block.

I agree. Easy to implement. Logical line splits should be a user defined option, but for now we should probably hard code the line split point at, say, 80 characters so it is readable without scrolling around too much to read it, just to get this beast out the door.

After reading your post (with a fresh clean mind) I realized the Tokenizer needs to do 98+% of it's stuff within it's own space. That way we don't have to pass icky things like pointers around between our modules.

Again, I agree. Limiting the direct editor/Tokenizer interface will also make it easier to add editor functionality without worrying about invalidating some functionality in the Tokenizer.

The trip was an easy one, once I got on the plane. The weather is beautiful! I've got a very good view of the mountains from my hotel. (Is that THE mountains or just the foothills?) No snow in Boulder so far, but there were white capped peaks in the distance this weekend.

Glad you enjoyed your trip. Yes, those mountains you saw were the mountains, but only a small part, a very small part of mountains in Colorado (the "bumps" you saw below the mountains are the foot hills). As I mentioned previously if you get on I-70 and head west you will see many more mountains without contending with Denver's smog problem.

If you want to see some truly rugged mountains head to the southwest corner of our state and visit the San Juan mountain range near Durango. There you will see some mountains. While you are there in Durango take a ride on the Durango to Silverton narrow gage railroad train which has been featured in many old western movies. It's worth the trip for the scenery alone. Should you keep on heading southwest from Durango you will see Monument Valley, also a film location site for many old John Wayne movies, equally impressive.

Some trivia for your amusement, Colorado boasts more 14,000+ foot mountains than any other state in the U.S. Eishenhower Tunnels (on I-70) are the highest traffic tunnels in the world. Also, Colorado is only one of two states in the U.S. where water flows out of, but not into the state. The other state is Hawaii.

Been awfully busy the past weeks, but I'm planning a trip to Rocky Mountain State Park on Saturday. Got a brand new camera just for this!

Maybe on a return trip you might want to consider visiting the Black Canyon on the Gunnison. It is sort of Colorado's western slope answer to the Grand Canyon. You can get some idea of what it looks like from the musical film "The Unsinkable Molly Brown", particularly when Harve Presnell sings his song at the top of what appears to be a big cliff, that is the Black Canyon.

[StanHelton]

...
How about renaming gvbCodeBlock to gvbSource?

gvbSource it is.

...
user defined option, but for now we should probably hard code the line split point at, say, 80 characters so it is readable without scrolling around too much to read it, just to get this beast out the door.

This is the standard then.

Again, I agree. Limiting the direct editor/Tokenizer interface will also make it easier to add editor functionality without worrying about invalidating some functionality in the Tokenizer.

I think we have a working paradigm.

...
As I mentioned previously if you get on I-70 and head west you will see many more mountains ...

Three of us are planning to drive up to Rocky Mtn State Park in the morning. I'll be taking pictures to prove I've been there. Let you know how that turns out later this weekend.

Alas, this is the only free weekend I was able to work out. That's the problem with business trips -- business gets in the way having fun. Good thing I like the job.

Interesting that you mentioned all those John Wayne movies. While taking a walk the other day I imagined myself in that old classic, Shane. I'm not exactly where I was, but I felt like I was walking through those old westerns.

...
Maybe on a return trip you might want to consider visiting the Black Canyon on the Gunnison. It is sort of Colorado's western slope answer to the Grand Canyon. ...

It may be a while before I get back this way, but I will be back this way. I would like to take a 3-4 week vacation (in about a year or so) and spend it being a tourist in this part of the country. I've seen Texas and California, so the parts of Colorado that aren't Boulder are very high on my list for next time.

[John R. Heathcote]

Stan,

That's the problem with business trips -- business gets in the way having fun.

I had to do a lot of traveling with my old job, stay in lousy hotels, sleep in lumpy beds, and use over priced room service. So I know what you mean.

Interesting that you mentioned all those John Wayne movies. While taking a walk the other day I imagined myself in that old classic, Shane. I'm not exactly where I was, but I felt like I was walking through those old westerns.

That is one of the reasons I recommended the Durango to Silverton railroad trip, you can get that real western look and feel. The train depot in both towns has been restored to circa 1870 conditions and has been featured in many a western movie. In fact most of Silverton has the 1870's look and it is really strange to see cars running up and down the streets instead of horses, buckboards, and covered wagons, although there are a few of those too. It's been a while since I've been to Silverton, but I do believe there are a couple of novelty photo studios there that can take your picture in authentic period costumes including posing with the shootin' iron of your choice. The town still may even do the daily gun fight on main street. Lots of fun!

Now back to business, does anybody have some code snippets for VB form conversion so I can see how difficult it will be to integrate this with the Scintilla edit control? Would greatly appreciate it.