Power Basic Benchmark!

[John Guillory]

Wanting to compare the raw speed of Power Basic, I benchmarked darn near every compiler I had working on my laptop that could make a .dll file.
I gotta say, I'm super-impressed with PowerBasic! I created a .dll file of the closest intpretation of Powerbasic's benchmark they put on the main web page. Granted, the benchmark is probably not the best test of compilers, and it doesn't compare all aspects of a language's power, like graphics, file processing, data access, etc. But gosh darn it, I believe this gives a good idea of where the raw speed is.... Here's my results so far:

PB5CC Returned 0.45 (timer) TIX returned 715,645,140
PB8Win Returned 0.41 (timer) TIX returned 657,344,196
HotBasic Returned 2.95 (timer) TIX returned 4,618,767,060
VP 2.1 Returned 3378 ms TIX returned 5,392,704,300
PureBasic (see note) TIX returned 4,833,094,596

note: Pure Basic didn't support all types needed to give a fair result,
and I didn't have time to work out how to return a value in a .dll function,
so I created a procedure. Despite using smaller floating point numbers in
most cases Pure Basic was still significantly slower.

% wise (smaller number is better) are as follows:

PB5/CC 4.4128
PB8/Win 4.0533
Hot Basic 28.4800
Virtual Pascal 2.1 33.2523
Pure Basic 4.3 29.8016

Interesting note: While folks put BASIC down and claim its slow, all BASIC compilers came out faster than Virtual Pascal. I need to look into making a .dll with Free Basic and perhaps Free Pascal and benchmark them.

The reason for the DLL creation is simple. All benchmarking / output is done in PB5/CC, the most basic benchmark is ran as a function in each .DLL and is called from PB5/CC, TIX is called before the function and TIX END is called after the function. All 4 DLL Functions are called exactly the same, such that no favortisim is played.

[Paul Dixon]

John,

Despite using smaller floating point numbers in most cases Pure Basic was still significantly slower.

I think you'll find that if you use "smaller floating point numbers" in Powerbasic, it will also be slower.
Powerbasic uses register variables to gain speed but only for EXT FP data types.

Paul.

[Bob Zale]

Paul-

You should note that the use of automatic register variables is but one of the techniques PowerBASIC uses to achieve excellent execution speed. The fact that it doesn't use register variables with single precision or double precision variables is not a failing of PowerBASIC. Rather, it's the fact that there are no single precision or double precision registers on the FPU! {smile}

Best regards,

Bob Zale
PowerBASIC Inc.

[Michael Mattias]

FWIW, although nobody wants to hear this again...

90% of your application's performance is decided before you write the first line of code... in your design.

MCM

[Henk Spoelstra]

Not impressed by benchmark

John,

I just bought PB/CC5 and did the first test. I have tested the 100.000.000 multiplication loop for it's speed as under the PB/CC5 description on the website.

I ran it on a PC with dual core processor 1.66 GHz.

My results:
PB/CC5 0.67 seconds TIX 1.135.304.470
Delphi 5 (Pascal) 0.75 seconds
My "good old" Quickbasic 4.5 after compilation and stand-alone .EXE 0.60 sec's (!!!)

So where is the speed gain? I am totally not impressed.

Henk Spoelstra

[Edwin Knoppert]

Yawwwn

Same story, same stupid examples.. i am sure.

[Bob Zale]

It's time to be impressed!

John,
So where is the speed gain? I am totally not impressed.
Henk Spoelstra

Thanks for the kind words, Henk. Unfortunately, they're just plain wrong.

PowerBASIC Console Compiler: 0.2003 seconds
microsoft quick basic: 7.687 seconds

PowerBASIC is a little more than 38 times faster. Don't bother being impressed, just post accurate results. {smile}

Bob Zale
PowerBASIC Inc.

[John Montenigro]

Henk,

Would you be willing to post the code you used for comparison?

As MCM pointed out above, proper design has more impact on results than any amount of code tweaking.

Still, my guess is that the experts here would be able to show you inefficiencies that you might not be aware of, and that a proper use of PB's capabilities would indeed give you the impressive results that most of us have experienced.

I'm not a PB employee, investor, etc., just a satisfied customer, but I have to say with enthusiasm: This is PB - there's no need to settle for less than exceptional results. Let's get you impressed!

[Bob Zale]

John --

The code is already posted in the PB/CC Product Section. It's just unfortunate that Henk just didn't measure it accurately. I'm assuming, of course, that it was an inadvertent error and nothing intentional.

Bob Zale
PowerBASIC Inc.

[Henk Spoelstra]

Bob and John M,

The numbers I gave were the numbers I obtained with my programs on my computer. I did not on purpose "trick" the data. Just as a plain, simple user I did a test. Nothing more, nothing less.

Bob is right for the test in Quickbasic when running in the interpreter mode itself. Then I obtain 4 seconds with double precision. When I make an EXE then it's 0.6 seconds.

For the sake of clearity I give the programs below. I hope that clarifies the things I maybe did wrong.

Please correct me where I dove into the pit.
Regards,
Henk

PB/CC5 ->>>
#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
DIM X AS DOUBLE
DIM Y AS DOUBLE
DIM t AS DOUBLE
DIM I AS INTEGER
DIM Cycles AS QUAD
DIM A AS STRING
x = 1
y = 1.000001
t = TIMER
TIX Cycles&&
FOR i = 1 TO 100000000
x = x * y
NEXT
t = TIMER - t
TIX END Cycles&&
PRINT t, Cycles&&
W: A="": A=INKEY$:IF a="" THEN GOTO W
END FUNCTION

QUICKBASIC 4.5->>>
'Speed
CLS
DIM I, J AS INTEGER
DIM X, Y AS DOUBLE
X# = 1#
Y# = 1.000001#
T1 = VAL(MID$(TIME$, 1, 2)) * 3600 + VAL(MID$(TIME$, 4, 5)) * 60 + VAL(MID$(TIME$, 7, 8))
PRINT TIME$, T1
TT1 = TIMER
PRINT TT1
FOR I = 1 TO 10000
FOR J = 1 TO 10000
X# = X# * Y#
NEXT J
NEXT I
T2 = VAL(MID$(TIME$, 1, 2)) * 3600 + VAL(MID$(TIME$, 4, 5)) * 60 + VAL(MID$(TIME$, 7, 8))
PRINT TIME$, T2
TT2 = TIMER
PRINT TT2
PRINT T2 - T1; " Sec"
PRINT TT2 - TT1; " Sec"
PRINT X#
A$ = ""
W:A$ = INKEY$: IF A$ = "" THEN GOTO W

Delphi 5 ->>>
unit TimeUnit1;
interface
uses
Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
StdCtrls;
type
TForm1 = class(TForm)
Button1: TButton;
LabelTijdvertraging: TLabel;
LabelStarttijd: TLabel;
LabelStoptijd: TLabel;
procedure Button1Click(Sender: TObject);
procedure FormCreate(Sender: TObject);
private
{ Private declarations }
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.DFM}
{===========================================================}
FUNCTION VertragingsLoop:Extended;
VAR x,y:Extended;
i :Integer;
pouble;
BEGIN
x:=1;
y:=1.0000001;
FOR i:=1 TO 100000000 DO
BEGIN
x:=x*y
END;
RESULT:=x;
END;
{===========================================================}
FUNCTION VertragingsLoop_alt:Extended;
{alternatief => Deze geeft meer controle over de grote lus waarde }
VAR x,y:Extended;
p :integer;
BEGIN
x:=1;
y:=1.000001;
p:=0;
WHILE p<100000000 DO
BEGIN
p:=p+1;
x:=x*y;
END;
RESULT:=x;
END;
{===========================================================}
procedure TForm1.Button1Click(Sender: TObject);
{Het primaire proces is zo compact mogelijk gehouden. Eerst }
{worden de start en stop tijden bepaalt en daarna afgedrukt }
{Dit om enigszins verwerkings (vertragingstijd te voorkomen }
VAR StartTijd,StopTijd:TTime;
DeltaTijd :TTimeStamp;
Getal :Extended;
begin
StartTijd:=Now;
Getal:=VertragingsLoop_alt; {primair proces}
StopTijd:=Now;
LabelStartTijd.Caption:='Start time = '+TimeToStr(StartTijd);
LabelStopTijd.Caption:='Stop time = '+TimeToStr(StopTijd);
DeltaTijd := DateTimeToTimeStamp(StartTijd-StopTijd);
LabelTijdvertraging.Caption:='Time lapsed = '+IntToStr(DeltaTijd.Time)+' ms'+' Value = '+FloatToStrF(Getal,ffGeneral,15,8);

end;
{===========================================================}
procedure TForm1.FormCreate(Sender: TObject);
{Maakt alle tekst leeg }
begin
LabelStartTijd.Caption:='';
LabelStopTijd.Caption:='';
LabelTijdVertraging.Caption:='';
end;
{===========================================================}
end.

[Bob Zale]

The plot thickens...

The truth is this:

The PowerBASIC code is 38 times faster than QB when compiled to an EXE. It is 61 times faster than QB when run in the interpreter. The PowerBASIC code 8 times faster than DELPHI 5 code (1.728 seconds to 0.2003 seconds). These are extraordinarily good results favoring PowerBASIC. You would not expect to see the same comparison levels in all types of code.

But you can take these numbers to the bank.

Best regards,

Bob Zale
PowerBASIC Inc.

[Kev Peel]

(To Henk):

Even if it was faster in QBDOS it is not a fair comparison. Can you interface with Windows API in QB? DLLs? Threads? .. I thought not

A better comparison would be between PB/WIN/CC and other 32-Bit WINDOWS languages, not DOS.

It's certainly the fastest BASIC in Windows.

[Petr Schreiber jr]

Good evening ( at least here ),

Henk, your code for PB is:

PB/CC5 ->>>
#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
DIM X AS DOUBLE
DIM Y AS DOUBLE
DIM t AS DOUBLE
DIM I AS INTEGER
DIM Cycles AS QUAD
DIM A AS STRING
x = 1
y = 1.000001
t = TIMER
TIX Cycles&&
FOR i = 1 TO 100000000
x = x * y
NEXT
t = TIMER - t
TIX END Cycles&&
PRINT t, Cycles&&
W: A="": A=INKEY$:IF a="" THEN GOTO W
END FUNCTION

Variable "i" should go from 1 to 100 000 000 ... but it will never do, as you declared i as INTEGER (16bit), which can store values from -32,768 to +32,767.
So 100 000 000 is little bit out of range.

I would recommend to use LONG (32bit), and register it on CPU. I do this very often and just using this simple mechanism provides noticeable advantage in real world apps over other compilers and managed solutions I met (Delphi/C#).

Code in products sections also does not use Doubles, but Extended data type.

Also - for such a delicate time interval tests, it is not the best idea to use TIMER or GetTickCount. You can try QueryPerformanceCounter.

I personally do not like this loop benchmarks, there has been cases in the past where various compiler writers optimized output for such a simple cases.


Petr

[Henk Spoelstra]

No more arguing about the speed data

Bob,

I don't argue the figures anymore. I assume that you have tested PB thoroughly against other languages with more whistles and bells than the simple test at the PB-website under the PB/CC5 description by which I was inspired and did tests with. Would be nice though to see the code you did the test with...

Kev & Bob,
I need pure calculation power, just hard data in and hard data out. I don't need DLL's, API's, Threads, Website links or whatsoever from the Windows environment. I agree that can't be done with QB, but I don't need it.

I have a scientific calculation program, which does calculations in QB (.exe mode) for about 2 - 6 hours depending on it's input parameters. After I have converted that program from QB to PB I'll do a comparison test. That will show a more realistic "calculation" power speed comparison for me.

[Kev Peel]

I can understand where you are coming from but the architecture is different between 16 and 32-bit (also 32 and 64-bit, to a lesser degree). A true Win32 program has quite a bit more overhead (clock cycles) than old DOS. If you want to compare DOS code, compare PowerBASIC DOS 3.x and QB/VBDOS - the results will blow you away.

Benchmarking DOS code with Win32 is simply comparing apples with oranges and cannot be taken seriously. It's like comparing a bicycle with a tank, sure the bike goes faster, but.. you see?

[Chris Boss]

Ran tests on Windows XP Home with a 2.5 ghz CPU:

PDS 7.1 (QB's pro version) (ran compiled EXE)

27.84 seconds

PB 9.0

1.36 seconds

PDS 7.1 code:

DIM X AS DOUBLE
DIM Y AS DOUBLE
DIM T AS DOUBLE
DIM I AS LONG
DIM A$
X = 0
Y = 0
T = TIMER
FOR I = 1 TO 100000000
   X = X * Y
NEXT I
T = TIMER - T
CLS
PRINT STR$(T)
DO
   A$ = INKEY$
   IF A$ <> "" THEN EXIT DO
LOOP
END

PB 9.0 code:

#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
     DIM X AS DOUBLE
     DIM Y AS DOUBLE
     DIM t AS DOUBLE
     DIM I AS LONG
     x = 1
     y = 1.000001
     t = TIMER
     FOR i = 1 TO 100000000
          x = x * y
     NEXT
     t = TIMER - t
     MSGBOX STR$(t)
END FUNCTION

[Bob Zale]

Bob,
I don't argue the figures anymore... That will show a more realistic "calculation" power speed comparison for me.

Then it would be very nice if, on your first visit here, you wouldn't broadly announce "I bought PowerBASIC (hours ago)... I'm not impressed.". Please treat us the way you would like to be treated? With accurate timings and normal business courtesy? We would very much appreciate that.

Best regards,

Bob Zale
PowerBASIC Inc.

[Brad D Byrne]

I bought PowerBASIC about (68352 hours ago)... and I'm still impressed!!!.

[Brian Chirgwin]

Try the following.

#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

    DIM X AS EXTENDED
    DIM Y AS EXTENDED
    DIM t AS DOUBLE
    DIM I AS LONG
    DIM Cycles AS QUAD
    DIM a AS STRING
    
    x = 1
    y = 1.000001
    TIX Cycles&&
    t = TIMER
    FOR i = 1 TO 100000000
        x = x * y
    NEXT
    t = TIMER - t
    TIX END Cycles&&
    PRINT t, Cycles&&
 W:
    A="": A=INKEY$:IF a="" THEN GOTO W
END FUNCTION

[Henk Spoelstra]

That helps

Thanks, Brian!

Setting X and Y to EXTENDED gives an average computing time of 0.34 sec's.

So now I gained about a factor of two and now I am impressed.

It is interesting to run the test several times as the speed values are different from run to run around an average value.