Integer evaluated as fixed floating point

[Richard Cann]

Any suggestions on how to tackle this one would be greatly appreciated.

The basic problem is that integers in a small section of a subroutine are displayed and processed as a fixed floating point number, being assigned a value of 9223.37203685478. The variables affected, as evaluated in the Sub seem to change depending on what local code additions I make. I am using PBCC 4.04. There are never any compilation warnings.

Some characteristics are:
1. The integers affected are all Global
2. Changing from Long to Integer has no effect.
3. Placing print statement immediately before the Call shows no problems
4. Placing print statement at the beginning of the Sub shows the problem
5. Placing print statements a dozen lines later after a Select Case shows that the problem has gone away!!!!!

Some of the things I have done to investigate are....

1. Run code analyzer from JF Pro. No anomalies.
2. Run on both Win2000 and Win XP. same result.
3. Run Debug mode and seen the same strange behavior.
4. At the sub routine head dimmed an auxiliary variable and equated the problem variables. No effect
5 Before the Call assigned auxiliary variables and passed the value to the Sub. No effect.
6. Run the code in a 1 sec loop and logged the data to file.
7. Torn all my hair out. No effect.

I hope some one has seen something like that before and can make some recommendations other than a good brand of hair restorer!

Thanks Guys.

Rich

[José Roca]

Are you doing something in your code or calling a third-party DLL that could reduce the precision control bits for the accuracy of the mantissa? If that is the case, you will need to reset the FPU to 64 bits.

[Mel Bishop]

I didn't see anything in your post indicating you trying SINGLE or DOUBLE where numbers are supposed to be processed as fractional.

9223.37203685478, will be processed as a long integer (9223) whereas single or double will process out to the full decimal place.

[John Petty]

Simply put there is no way the compiler could create code to change the precision of a variable (particularly a global one) at run time as it would create horrendous memory management problems. If an integer or long integer is involved in a calculation with a floating point then it may be converted to that whilst storing in an FP register or might even create a hidden temperory variable but the precision of the original variable will always be the same.
I suggest seeing no code has been posted that you look at your 3 print statements, are they identical? if you are using simple STR$ or FORMAT$ without any specified formatting then the print statement will just print the original variable at its original precision. If of course you specify a format that has decimal places or doing a calculation in the PRINT like multiplying by 0.1 then of course the result will be that of an FP number.

[José Roca]

It is not a matter of changing the precision of the variable, but the precision control bits for the accuracy of the mantissa. If the precision is reduced, the translation to a string by the PRINT statement will give erroneous results. I had this problem using DirectX 9 with PBWIN 8 because DirectX reduced the precision to 23 bits unless you specified the flag D3DCREATE_FPU_PRESERVE when calling the CreateDevice method.

[Richard Cann]

Thanks everyone for your quick replies...
Jose:
I am not using any DLL's in my code. Also see below...

Mel:
I am not using Single or Doubles, just Long. I do not convert between forms. So the puzzle is why am I getting an relevant decimal rendering of an integer, particularly when its value is assigned to be an integer of about 1000.

John:
The only operations I do in the code are adding subtracting and Select Case, all with Long integers.

I hope this helps but perhaps it just results in more questions. Thanks every one.

Rich

[Paul Dixon]

Richard,
are you using ASM?
Are you pushing too many values onto the FPU stack? It only takes 4 values unless you use #REGISTER NONE when it'll take 8.

Try adding #REGISTER NONE anyway as a quick test and see if it solves the problem.

Paul.

[Richard Cann]

Found it!

Hi Guys!

Thanks for all your help.
I thought you would like to know I found the answer.

In a FOR/NEXT loop, I was inadvertently taking the LOG10 of zero.
There was no error reported at all. It just corrupted adjacent code.

Learn something every day!!!

Rich

[Michael Mattias]

>It just corrupted adjacent code

Code not shown, but I don't know why an overflow should corrupt other variables... unless you used that "result" in some calculation.

#DEBUG OVERFLOW ON would have caught that. Oh, I forgot, that new feature suggestion has not yet been embraced by the PBPTB.

Maybe more people have to send that suggestion to [email protected] before it stands a fightin' chance of appearing.

[Paul Dixon]

The problem sounds like it's related to this one:
http://www.powerbasic.com/support/pb...3&postcount=13

FPU stack coruption following certain operations which result in an undefined answer.

Paul.