GetStdHandle problems

[Michael Mattias]

I have a problem with the STDOUT code posted a long time ago in Source code forum.

I have copied it inline here to add some debugging message boxes (also to encourage others to try it).

GetStdhandle is not returning what I am expecting under Windows/XP Pro all service packs

#COMPILE  EXE
#DEBUG    ERROR ON
#REGISTER NONE
#DIM      ALL
#TOOLS    OFF
'=====[Windows API Header Files] ============================
'  If you don't need all of the functionality supported by these APIs
'  (and who does?), you can selectively turn off various modules by putting
'  the following constants in your program BEFORE you #include "win32api.inc":
'
'  %NOGDI = 1     ' no GDI (Graphics Device Interface) functions
'  %NOMMIDS = 1   ' no Multimedia ID definitions

%NOMMIDS = 1
#INCLUDE "WIN32API.INC"
'==[End Windows API Header Files]============================


#IF NOT %DEF (%SYSEMT_INC)
  %SYSEMT_INC  = 1
FUNCTION SystemErrorMessageText (BYVAL ECode AS LONG) AS STRING
  LOCAL Buffer AS ASCIIZ * 255
  FormatMessage %FORMAT_MESSAGE_FROM_SYSTEM, BYVAL %NULL, ECode, %NULL, buffer, SIZEOF(buffer), BYVAL %NULL
  FUNCTION = FORMAT$(ECode, "##### ") & Buffer
END FUNCTION

#ENDIF


#IF %DEF(%PB_WIN32)
 FUNCTION STDOUT (Z AS STRING) AS LONG
 ' returns TRUE (non-zero) on success

   LOCAL hStdOut AS LONG, nCharsWritten AS LONG
   LOCAL nStdHandle   AS DWORD   ' casting problems>
   LOCAL w AS STRING
   STATIC CSInitialized AS LONG, CS AS CRITICAL_SECTION
   LOCAL iRET AS LONG, LE AS LONG

   ' ------------------------

   IF ISFALSE CSInitialized THEN
       InitializeCriticalSection CS
       CSInitialized  =  1
       
       
   END IF

   nStdHandle = BITS???(%STD_OUTPUT_HANDLE)   'STD_OUTPUT_HANDLE defined in Win32API.INC as -11&

   EntercriticalSection Cs

   hStdOut         = GetStdHandle (nStdHandle)  ' per Win32API.INC wants DWORD param, returns DWORD
   MSGBOX USING$ ("hStdout  #  & ", hStdout, HEX$(hStdOut, 8))
   SELECT CASE AS LONG hStdOut
       CASE %NULL, -1&
           iRet = AllocConsole        ' OK, it creates the first time
           LE   = GetLAstError
           #IF 0
           IF ISFALSE iRet THEN
               MSGBOX "Could not create Console" & $CRLF & SystemErrorMessageText (LE)
           ELSE
               MSGBOX "Created Console"
           END IF
           #ENDIF

           hStdOut       = GetStdHandle (%STD_OUTPUT_HANDLE)

   END SELECT

   LeaveCriticalSection    CS

   w                     = Z & $CRLF
   iRet                  = WriteFile(hStdOut, BYVAL STRPTR(W), LEN(W),  nCharsWritten, BYVAL %NULL)
   LE                    = GETLastError
   FUNCTION              = iRet
   #IF 1
   IF ISFALSE iREt THEN
       MSGBOX SystemErrorMessageTExt(LE),,"WriteFile Failed"
       ' 2/18/08 its failing on invalid handle. but I am getting a value of 7 which should be valid
       ' I am NOT creating it... I am inheriting it. What If I create my own instead?
       ' See 'fix' above using GetConsoleWindow. This code is not going to work
       ' on Win/98 anymore.
   END IF
   #ENDIF

   FUNCTION = iRet   ' true is good


 END FUNCTION
#ENDIF



#IF NOT %DEF (%INVALID_HANDLE_VALUE_LONG)
   %INVALID_HANDLE_VALUE_LONG    = -1&
#ENDIF


FUNCTION WINMAIN (BYVAL hInstance     AS LONG, _
                  BYVAL hPrevInstance AS LONG, _
                  BYVAL lpCmdLine     AS ASCIIZ PTR, _
                  BYVAL iCmdShow      AS LONG) AS LONG
                  
  ' do exactly one "stdout" just to get the values
  
   LOCAL S  AS STRING
   
   S = "First"
   STDOUT S
   
   ' do the second one to make sure it writes
   S = "Second"
   STDOUT S
   
   ' hold the console open
   
   MSGBOX "Check COnsole"
   
'04/21/08  Working perfectly from within the PB ide. (Compile and execute)
'
' Returns hstdout = 7 when exe is run from command prompt, hStdout=  7 (not zero, not INVALID_HANDLE_VALUE
'    but writefile fails on error 6, "the handle is invalid"
'
' when run from desktop shortcut, returns hStdout = 65537d (0x00010001) and again WriteFile
'  fails on error 6 the handle is invalid.
   
'  Windows/XP Pro, all service packs and updates. PB/Win 8.03


END FUNCTION

As explained in the WinMain function comments, the behavior of GetStdHandle() varies depending on if the program is run from the PB/IDE, from a command prompt, or from a desktop shortcut (shortcut created simply with Explorer, "Send to Desktop (shortcut)")

Can anyone tell me what to look for to get this working? Is this a security thing maybe?

For what it is worth, I DO have a workaround... I call GetConsoleWindow() the first first call to STDOUT; if that returns NULL I do an AllocConsole and after that everything is hunky-dory.

But what is happening here is truly a mystery to me. I looked thru MSDN, both library and knowledge base and can't find anything talking about a difference in behavior.

Thanks

[Michael Mattias]

OH HO, LOOK WHAT I JUST FOUND!

http://support.microsoft.com/kb/105305 (I got there via links from AllocConsole)

In the case of starting the GUI application from the command line without the "start" command, the standard OS handles are NOT correctly zeroed out, but are incorrectly inherited from CMD.EXE [bold mine, but capitalized NOT is Microsoft's] . When the application's CRT initializes, the three low I/O handles 0, 1, and 2 are initialized to use the three handle numbers that the application inherits from CMD.EXE.

And in my 'command-line' test....using "START programname.exe" solves the problem, handle to STDOUT is reported as NULL (still not correct, should ne INVALID_HANDLE_VALUE, but "Null" behavior had previously been reported); When NULL, the function Calls AllocConsole and STDOUT is now available and works just fine.

I can't get the desktop shortcut to accept "start" in the "target." When modifiying the properties the 'apply button it tells me "The name 'START' [or 'start.exe' when I tried that] specified in the target box is not valid. Make sure the the path and filename are valid."

I suppose if worst comes to worst I can make the shortcut launch a CMD file which uses START... let's see if that works...yes, it does work the way it should.

REM test_start.cmd 
REM Echo Testing launch 

Start d:\Software_Development\pbwin80\work\test_getStdhandle_stdout.exe

I did not want to go back to GetConsoleWindow for this test, since STDOUT may have been redirected and the program has no console - and should not have one. But I guess I can test for that, too, it's just going to be a bit more involved.

Anyone with any ideas re what I might do with the shortcut 'properties' or whatever which would result in not needing to create a CMD file to get the START in there... those ideas will be appreciated.

Also: anyone using the current STDOUT code at Simple STDOUT for PB/DLL and PB/Win 2-13-04 would be well-advised to review this info.

MCM

[Marco Pontello]

Anyone with any ideas re what I might do with the shortcut 'properties' or whatever which would result in not needing to create a CMD file to get the START in there... those ideas will be appreciated.

Something like:

%windir%\system32\cmd.exe "/c start notepad"

Bye!

[Michael Mattias]

Excellent!

I tried this in the shortcut target..

C:\WINDOWS\system32\cmd.exe /C Start D:\Software_Development\pbwin80\work\test_getStdHandle_stdout.exe

.. and now the program works correctly when started from the shortcut!