uCalc Language Builder feedback (scripting and interpreted languages etc)

[Daniel Corbier]

I have been developing uCalc Language Builder, which allows you to create your own programming languages (scripting or interpreted etc). This month, if you download the beta and work through the interactive tutorial and fill out the questionnaire and submit the answers to me, you get a Google coupon that takes 85% off of the price of the full uCalc Fast Math Parser license. uCalc FMP is closely related to the Language Builder (in fact they are based on the same DLLs). Plus if you purchase uCalc FMP, you are entitled to the uCalc Language Builder for free when it first comes out officially.

In addition to the feedback I'm receiving privately from those who have tried it, I would also like to get some general feedback here online from everyone, including those who have clicked on this topic out of mild curiosity, but don't necessarily have the intention of downloading uCalc LB.

Would you consider uCalc LB something useful? Does the concept sound too complicated? Is it unrealistic? Did you click on this message thinking uCalc LB was one kind of thing, and realized that it's something different from what you had in mind? Is there a particular kind of demonstration you'd like to see? Is there something that would make uCalc LB more useful to you? If you've tried it, and came up with something interesting, would you like to show it to the rest of us? Would anyone like to see additional PowerBASIC code for native callback routines? Any kind of feedback would be appreciated. Thanks.

[Joe Byrne]

Would you consider uCalc LB something useful? Does the concept sound too complicated? Is it unrealistic? Did you click on this message thinking uCalc LB was one kind of thing, and realized that it's something different from what you had in mind? Is there a particular kind of demonstration you'd like to see? Is there something that would make uCalc LB more useful to you? If you've tried it, and came up with something interesting, would you like to show it to the rest of us? Would anyone like to see additional PowerBASIC code for native callback routines? Any kind of feedback would be appreciated. Thanks.

Ok, here's my take.

Useful? hard to say. Personally, I can't imagine using a scripting language. It seems counter productive to me when its so easy to compile a "real" program. I guess for web apps, it could be useful, but then I can't imagine why I'd buy a scripting language when there are so many others for free. ThinBasic comes to mind first.

particular demo On the LB page, you have 'examples of two plain text language definition files', but neither is in Basic so I can't tell w/o downloading, what the scripting syntax even looks like.

Something Different? Not really sure. The only thing I get from the description is that its some sort of scripting language (again, why one would use a scripting language is beyond me) that has a couple of different verbs... Basic-like, Forth-like, LISP-like.

more useful? A few examples of when and why one would use a scripting language would be nice. Personally, I just don't see any advantage in non-compiled code.

My other observation is that the math parser is pretty hefty price wise ($600) and since you have no indication of what the cost of the LB is, I can only assume that it too is priced out of my range. I don't know who your market is (can't really tell from the site) so I have to assume its not me.

[Emil Menzel]

>>Personally, I just don't see any advantage in non-compiled code.

Sorry, Joe, but that sounds to me like it comes from a member of the "technology priesthood -- the people who made it difficult for individual users to take control of their machines". The quote is from John Dvorak's column "Fast car in the slow lane" in PC mag May 2008, and he spells out the single greatest advantage of "non-compiled code": it gives the individual control of their machines and frees them of having to bow to the priests, or even to you and me.

[Joe Byrne]

>>Personally, I just don't see any advantage in non-compiled code.

Sorry, Joe, but that sounds to me like it comes from a member of the "technology priesthood -- the people who made it difficult for individual users to take control of their machines". The quote is from John Dvorak's column "Fast car in the slow lane" in PC mag May 2008, and he spells out the single greatest advantage of "non-compiled code": it gives the individual control of their machines and frees them of having to bow to the priests, or even to you and me.

I understand the benefit of open source, but there are a number of disadvantages too. For example, if Word or Excel were open source and anyone who wanted to change it could, support would be a nightmare. For me, my commercial apps generate a fair portion of my income and target a fairly small niche market. If I were to open the code up, I'm am 100% sure some smart person would get in there and start a business to support it, in direct competition with me. Its not his code, and I can't guarantee the quality of his work. So if he starts screwing it up, and his customers start telling others that the program is junk, I start to loose whatever potential customers are out there. Do you think a potential customer is going to call me and ask if the programs are terrific as long as they don't allow Mr. fixit to mess around with them?

Vertical applications, IMO, are best when they are compiled and controlled. Other things, such as backend databases (MySQL for example) are great open source ideas because people can build vertical applications around those things. There is a place for both, but AFAIK, most PB people write vertical packages which I believe are best compiled and not open source.

Just my opinion of course.

[Mike Stefanik]

... it gives the individual control of their machines and frees them of having to bow to the priests, or even to you and me.

That doesn't make much sense to me. If a scripting/interpreted language is used, then they're dependent on the interpreter. The fact that they can look at the source and modify it doesn't mean squat to 99.9% of the people out there.

How many folks who use Firefox say to themselves, "Finally, I'm free modify the HTML rendering engine to do... whatever"? Not very many. The appeal to the vast majority of users is that open source code is free. They couldn't care less whether it's "free as in freedom", as long as it's "free as in beer".

This notion, that really got going with Stallman's cult of the programmer, that everyone wants to be able to program their computer and have "control" without bowing to the "priesthood" is just completely out-of-whack with reality. Most people see computers as tools that they want to use. They don't care what's going on under the hood, any more than they care about how their microwave works, or how their remote turns their TV on. They just want it to work, and when new versions of the software are released, they want to continue to be able to use the data they've already created. Simple. As for the whole fantasy about controlling the machine and computer priesthoods -- that's just nerd egocentrism ("It's all about me, the programmer! Me!") at its apex.

[Emil Menzel]

I apologize for starting a debate that might be irrelevant. My own personal opinions, a.k.a. biases, reflect the facts that I am an amateur; a retiree with no need or desire to sell my programs for money; an old geezer who learned BASIC when it was probably the only program one really had to know in order to do anything on a personal computer; and also a member of the minority (?) who likes interpreters and would love to see a PB interpreter or its equivalent that incorporates the capabilities of Dan Corbier's parsers.

Perhaps I should add that I recently spent a bundle on Mathematica (full list price $600+, but I fortunately did not have to pay that), precisely because it is so programmable and does use "notebooks" that are even better than an interpreter. If I were young again I'd devote the next year to learning Mathematica and saying goodbye to BASIC, Word, Excel, PowerPoint, and all the other programs whose functions can be matched or surpassed (for my own purposes) by Mathematica.

But on the other hand, I almost surely should not have added that last paragraph. I take it back! I still do love PB dearly and shall always keep her as a member of my harem, I promise.

[Daniel Corbier]

Thanks (everyone so far) for your feedback.

Joe, even though you're not a uCalc LB user, you made some very important observations, and I hope others looking at this thread (non-uCalc users, and uCalc users alike) will also pitch in and say what they think as well.

Others will probably do a better job at explaining why a scripting language might be useful. But one thing that comes to mind is that you may want to give your users the ability to make small changes in the program on their own, without having to ask you to compile a new version of your product for every little change that is needed. Your language does not have to be capable of modifying the core of your program. Nor does your program have to be open-source. You can have a mini-language that supports simple batch-like processes, or use it to let users flexibly configure certain settings, or to pre-process text in a certain way, etc. You can restrict the scripting to a specific part of your program.

Although I have some general ideas of how/why you'd use a scripting language, I need some help from readers here to come up with more specific and compelling examples.

Would you find uCalc LB more appealing if it were free? (I ask about this in the questionnaire as well, and seriously want to know).

The difference between uCalc LB and ThinBasic (as I understand it), is that ThinBasic is a scripting language. It has a particular syntax. uCalc LB on the other hand lets you construct the language of your choice. In this forum, users will naturally probably prefer a BASIC-like syntax (although I haven't posted an online definition file for it like I did for Lisp and Forth, there is one for BASIC in the download). But you can equally define other kinds of scripting languages. Or better yet, if you want, you can have a BASIC-like language that borrows interesting constructs from other languages if you'd like. The thing is that you won't have to wait for me to update the BASIC definition file. You could add to or modify the syntax as you want.

By the way, based on what you pointed out, I have updated the LB page to include sample code for Lisp and Forth so you can see what the syntax looks like. I haven't posted the definition file for BASIC because it looks a little more complicated, and is not quite as complete. However, I can post it anyway. Would that help?

As for uCalc FMP, how much would you pay? (There's no wrong answer; I really want to know). I'm thinking that I may have made a mistake when I increased the price to $600. The previous price didn't seem to cause any problems. But since I've raised it, only a couple of people have paid $600. Meanwhile, whatever you think of the list price, you can't go wrong with the very big discount (which I plan to gradually decrease) currently available. See:

http://www.powerbasic.com/support/pbforums/showthread.php?t=36764

As for uCalc LB, it is still in beta, and a price hasn't been fixed yet. I certainly want to make a profit from it, and at the same time I don't want the price to alienate those who might be interested. What kind of price structure would you suggest?

You observed that you don't know who my market is. In fact, I'm not so sure either, which might explain why I'm not getting the volume of feedback for uCalc LB I've been hoping for. Just like you, probably most others who are mildly curious feel excluded. And this is what I'm trying to change with this discussion. I want to do what it takes so that more people feel that there's something in uCalc LB for them.

[Mike Stefanik]

Others will probably do a better job at explaining why a scripting language might be useful. But one thing that comes to mind is that you may want to give your users the ability to make small changes in the program on their own, without having to ask you to compile a new version of your product for every little change that is needed.

Speaking in general terms, not directly to your software, that can be a real double-edged sword. That kind of extensibility can be a fantastic feature, but it also can create a real testing headache for the developer and introduce security issues because it can be difficult to predict exactly how a particular user will extend or change the behavior of your software. If you look at many of the products out there that have some sort of scripting capability, that's usually the attack vector used to exploit a particular vulnerability in the software. It's like the auto mechanic who leaves the garage door open, the car running inside and his tools easily accessible to anyone walking by. Providing a scriptable interface to your software's core engine is an invitation for folks to "hack" it.

I guess what I'm saying is, what you're talking about can be useful, but it needs to be measured by necessity. Is it a feature that the software really needs to be useful, or would it be just a "gee whiz" feature that's added just because the developer thinks that it would be neat?

[Marco Pontello]

For a series of examples of how a scriptable interface come handy, one can look at the Lua "uses" pages. There are lots of commercial games (so, basically some of the most complex pieces of software around) but also various "serious" applications (like something that run in the Space Shuttle, or WireShark/Ethereal).

http://www.lua.org/uses.html

As for script as a possible source of additional problems, that obviously have to be accounted for in the design phase. But usually one endup creating a sort of sandboxed environment, disabling languages parts/modules/functions not needed/too dangerous, exporting an API that does the needed amount of parameters checking, etc. Also usually the script (usually, for sure for Lua for example, but I don't know for sure for uCalc LB) themselves don't need to be in plain text and/or user accessible: they can be encrypted or embedded in some sort of module; in this case they serve just to ease the developer work, that for example can quickly create a customization in-house and send just a small package to the user, or in many other creatives ways.

As for uCalc LB, Daniel, one thing that I can say is that you probably target a rather small niche. I mean:
1) who have a need for a scriptable interface, AND
2) who have a need for a specific language syntax

Because if 2) isn't really an issue, probably some may find that embedding Lua o Python (the usual suspects, so to speak) may work good enough - while I'm sure that uCalc LB have some features that can come very handy in other specific situations.
Just a tought.

Bye!

[jcfuller]

While I haven't given this a good go over how do you compare ULB to the Gold System?

http://www.devincook.com/goldparser/index.htm

James

[Joe Byrne]

Would you find uCalc LB more appealing if it were free? (I ask about this in the questionnaire as well, and seriously want to know).

Probably not me personally. Like I said, I have a hard time trying to understand a useful purpose for scripting, at least for the business type applications I write. I understand your ideas for making an application more flexible, but to me, that equates to lost revenues. When I sell a package, I try to make it as general for the target market as possible. This gives me the broadest possible base of potential customers. I know going in, and I convey to my customers, that ever business is different so while 95% of the program will work within their existing SOP, there will be 5% that they either need to adjust their processes for, or pay me to adjust the program to fit their methods. Perhaps I'm trying to milk the cow twice, but I also price my applications with that in mind. I suppose I could use the scripting language myself to make these modifications, but if I can do it with a script, I shouldn't have any problem doing it in compiled code either.

The idea that the LB is fluid, not having a ridged structure, actually scares me off. It might just be that my thinking is rooted in 30+ years of procedural code, but structureless concepts like this seems to me nothing more than extra rope to hang myself with. I use MACROs sparingly for pete's sake!

As for uCalc FMP, how much would you pay? (There's no wrong answer; I really want to know). I'm thinking that I may have made a mistake when I increased the price to $600

Again, just for me, I doubt I would purchase it at all. I'm sure its a great package, but I've never written an application that required any more complex math than what I can do with the native compiler. Granted, 99.9% of my apps are fairly straight business verticals, and all, except for some simply utilities, are database systems. I can imagine in an engineering setting, or other market where advanced math was necessary, your program would be very useful, but not for the things I write.

As for the price, I wouldn't jump to the conclusion that its too expensive. When I am faced with a particular problem, I look at all the options. If there is an existing solution, its pretty easy to calculate its cost vs my time to reinvent it. I can't really remember a time when re-inventing was the cost effective solution. So if I'm looking at $2,500 of development time, $600 is a steal.

Chris Boss and a number of us discussed this very thing about his EZGUI system. For a lot of people, the price is too high, but that is mostly (I think) because they are not able to recoup the up-front cost. Either they are writing for an employer, making shareware/freeware, or designing on speculation and if the program doesn't sell, their up front costs can't be recouped. However, those that actually buy it are very serious about using it and getting the most from it. They are more likely to be repeat customers and your support costs won't be nearly as high. So assuming your software is well written (which I have no doubt about), my opinion is that you're better off having fewer customers paying a premium price than more customer's paying a lesser price. Those that buy it are going to be good advertising, and there is exceptional value for you there too.

[Brice Manuel]

it can be difficult to predict exactly how a particular user will extend or change the behavior of your software

Huh? They can only change or do what you allow them to.

ULB is a good product. Unfortunately, there are a lot of scripting solutions out there with good reputations that have been tested on the battlefield. That is not to say there is not room for another.

Potential uses? Game development. Very useful for image processing in "paint" programs. AI modeling, simulation, emulation, prototyping.

[Mike Stefanik]

Huh? They can only change or do what you allow them to.

You seem to be excluding the possibility of unintended consequences; my point was that by providing a scriptable interface to your game engine, you've significantly increased the surface area for someone to do "interesting" things that you didn't envision or intend with your software.

Case in point, the most popular MMO out there, WoW. Their scriptable UI (which uses a C-like language) has been used to do some decidedly unpleasant things ... like automating "spambots" for gold/powerleveling services. Eventually they locked things down to the point where that particular abuse was restricted, but it certainly wasn't a use that they had envisioned for their game.

The "they can only change what you allow them to" kind of thinking tends to presume that a) you can forsee all ways in which the scripting language can be used with your product, b) you won't make any errors in your programming. Both of those are extremely unlikely in anything much more complicated than a "hello, world!" type of program.

I'm not saying that scripting isn't useful. I'm saying that it opens a completely new attack vector for someone to misuse/exploit your software, and therefore shouldn't be something that's added "just because it's neat", but rather when it offers significant value to most users. Whatever methods you expose to the scripting interface in your game engine (or whatever it is you're writing) are now the equivalent of a public API, and so you have to be a lot more defensive in terms of how you handle things like errors, parameter checking and so on.

[Brice Manuel]

Mike: They can only use the syntax you provide and you are the one parsing those commands. If somebody doesn't know how to properly implement a scripting language, then they shouldn't do it.

[Daniel Corbier]

GOLD parser comparison

James, it's been a while since I looked at the GOLD parser site prior to this thread. I have quickly re-visited it again based on your question. However, I don't want to be too authoritative on it for fear that I might misrepresent the way the GOLD parser works. But from what I gather looking at it again, GOLD is a variation (with improvements) on the conventional YACC way of creating a language. One key difference between GOLD and ULB is that with GOLD you must first define the language grammar in a text file, and then have the Builder engine create a grammar table file, which in turn must be combined with special source code from your host compiler in order to complete your language.

uCalc LB uses a different approach that completely eliminates these intermediate steps, and lets you define both syntax and semantics in the same setting. In addition to greatly simplifying things, this allows you to create a language dynamically; even interactively. With uCalc, you can basically take a language definition file in plain text (such as the ones listed at www.ucalc.com/langbuilder.html ), and feed it into uCalc, and you instantly have a working language. There's nothing more to it. With GOLD, after feeding it your language grammar in BNF form, you have more steps before you can have a working language.

GOLD prides itself in supporting many host languages for creating your own language, which is great (I didn't see PB listed on their site though). However, each supported language has its own separate engine. With uCalc LB, there is only one engine, regardless of the compiler you are using. All of the complicated action happens under the hood within uCalc. (You can, however, compile a series of core routines into a DLL, which would be usable from any compiler).

In fact a compiler is not even needed. For instance, although the demo uCalc.Exe interpreter is written in PB, you do not have to compile any PB code before you have a working language. The demo interpreter (uCalc.Exe) automatically transforms itself to any language you load up.

[Daniel Corbier]

I have spent a lot of time working on the technical aspects of uCalc LB. But when it comes to identifying the correct target audience, I haven't been successful in that aspect yet. I kind of hoped that people would figure out on their own what they might be able to do with it. Apparently, it's not going to happen that way. So I guess I'm going to try to figure out the target audience as I go along, hopefully with the help of those reading this thread.

Maybe it's a little bit futile to highlight the aspects of LB that have a lot of competition. The ability to create a full-fledged scripting programming language is just one possibility. However, the thing that you create does not have to be a scripting language per se.

uCalc Fast Math Parser represents one kind of "language" you can develop with uCalc LB. With PB, you can create a program that plots a sine wave for instance. However, if you wanted your program to be able to plot any kind of equation the user wants, you'd need some kind of "language" that interprets math expressions. In this case, the language ability to interpret a math expression is a must, and not a luxury for a plotting program. Math notation is very restrictive as a "language". If your plotting program is restricted to conventional math functions and operators, there is no chance at all that someone could come up with some kind of equation that would create a security risk. Plus, the math expression doesn't modify the core of the program.

Another example is a mini-language that might control Microsoft Agent cartoon-type characters. PB comes with a demo for manipulating a MS Agent character (see the PBWin80\Samples\COM\MSAgent\MSAGENT.BAS file that comes with PB). The PB demo has a series of hard-coded animations. However, how about if you wanted to create some kind of presentation program that lets the user create a script of what the animated character will say, and what kind of movements it will make? It would be strictly limited to the actions supported by MS Agent, along with a few simple constructs like loops, and If/Then. It wouldn't need to be able to write to file, or access the Internet, or anything like that. uCalc LB may be fluid in how it lets you create your language. But once you have the syntax you want, you can lock it from being further modifiable. Your language is not required to allow access to sensitive API routines.

(The following link: http://www.ucalc.com/mathparser/beta/intp0805.zip is an old and obsolete version of the language builder, bit it included an MS Agent language demo. For that one, I wasn't really concerned with restricted the language; but you'll get the idea).

As Marco pointed out, the scripting ability does not have to be accessible to the user at all. In another kind of program, you can make scripting privately accessible only to you as the developer. The language itself can be plain for you, but maybe you'd pass it through a filter so that the text you give the user looks like undecipherable machine code. So there is no loss of revenue for you there. Full control is solely in your hand. For minor updates, you could simply e-mail a small text file with your "machine code" script.

[John Spikowski]

ScriptBasic is really easy to embed in your PowerBASIC programs if your looking for mature Basic interpreter. I will try to post an example here soon.

http://www.scriptbasic.org

Embedding the Interpreter


John

[Elias Montoya]

uCalc is the best i have seen. Currently im very busy with the next upgrade of Egrid32,
but my intentions are to create a full featured spreadsheet engine for Egrid32 using uCalc
as soon as i can. This way i will be able to show my findings with uCalc at the same time
i create a powerful example for Egrid32.

Yeah! PowerBASIC... Power!

[Brice Manuel]

ScriptBasic is really easy to embed in your PowerBASIC programs if your looking for mature Basic interpreter. I will try to post an example here soon.

http://www.scriptbasic.org

Embedding the Interpreter


John

Any particular reason for hijacking Daniel's thread to pimp your product?

[John Spikowski]

Any particular reason for hijacking Daniel's thread to pimp your product?

Brice,

What I got out this thread is that Daniel's offering may not be a good fit as an embedded scripting language. ScriptBasic is a LGPL open source Basic interpreter so I don't get your point saying I'm pimping my product. :thinking:

I have spent a lot of time working on the technical aspects of uCalc LB. But when it comes to identifying the correct target audience, I haven't been successful in that aspect yet. I kind of hoped that people would figure out on their own what they might be able to do with it. Apparently, it's not going to happen that way. So I guess I'm going to try to figure out the target audience as I go along, hopefully with the help of those reading this thread.

I guess I was also pimping the new HP mini-note when I made that post.

Daniel,

I apologize if my post about ScriptBasic as an alternative solution high-jacked your thread. I will remove the content if you wish.

John