SHFileOperation does not free it's handles

[Michael Zimmer]

Hello.

I thought of using the API function SHFileOperation() to backup folders. I made a little test program with some codefragments i found here in the forum.
When i'am looking at handles with the taskmanager, i see while the copy is in progress handles increase.
So i included the function SHFreeNameMappings(), but handles are still the same.
How can i free those handles?

Here's the testcode, built with PBCC 4.0.4

#COMPILE EXE
#DIM ALL
#INCLUDE "Win32API.inc"


'------------------------------------------------------
FUNCTION DirDelete( BYREF fileorfolder AS STRING) AS LONG

    LOCAL shfo      AS shfileopstruct
    LOCAL lResult   AS LONG
    LOCAL ea        AS LONG

    shfo.wfunc = %fo_delete
    shfo.pfrom = STRPTR(fileorfolder)
    shfo.fflags = %fof_noconfirmation OR %fof_noconfirmmkdir 'OR %fof_silent

    lResult = shfileoperation( shfo )
    ea = GetLastError()
    IF lResult THEN
        FUNCTION = ea
    ELSE
        FUNCTION = lResult
    END IF

END FUNCTION
'----------------------------------------------------------------------------
FUNCTION DirCopy( BYVAL source AS STRING, BYVAL destination AS STRING ) AS LONG

    LOCAL shfo      AS shfileopstruct
    LOCAL lresult   AS LONG
    LOCAL ea        AS LONG
    LOCAL stemp     AS STRING
    LOCAL sourceattributes AS LONG
    LOCAL nMapping  AS DWORD

    shfo.wfunc = %fo_copy 'copy not move.
    shfo.pfrom = STRPTR(source)
    shfo.pto = STRPTR(destination)
    shfo.fflags = %fof_noconfirmmkdir OR %fof_noconfirmation 'or %fof_silent

    lResult = shfileoperation(shfo)
    ea = GetLastError()
    IF lResult THEN
        FUNCTION = ea
    ELSE
        nMapping = shfo.hNameMappings
        SHFreeNameMappings( nMapping )          'does not really help
        FUNCTION = lResult
    END IF

    'set destination attributes to the same as sources
    sourceattributes = GETATTR(source)
    SETATTR destination, sourceattributes

END FUNCTION

'#######################################################
FUNCTION PBMAIN () AS LONG

    LOCAL ret           AS LONG
    LOCAL destination   AS STRING
    LOCAL source        AS STRING

    'change to your needs
    source      = "V:\VMmachines\WinXPFSC"
    destination = "D:\Backup\WinXPFSC"

    ret = DirCopy( source, destination )
    STDOUT "ret="+BIN$(ret)
    STDOUT "now deleting destination"
    WAITKEY$
    ret = DirDelete( destination )
    STDOUT "ret="+BIN$(ret)
    WAITKEY$

    STDOUT "copy again..."
    ret = DirCopy( source, destination )
    STDOUT "ret="+BIN$(ret)
    STDOUT "now deleting destination"
    ret = DirDelete( destination )
    STDOUT "ret="+BIN$(ret)
    WAITKEY$

END FUNCTION

[Michael Mattias]

This apparent behavior (adversely?) affects what?

i.e., So what if task manager is accurate re the handle count?

Are the files copied and deleted?

And you are sure it's ShFileOperation and not SETATTR which is retaining handles? Code shown will not make that distinction.

For that matter, are you sure Task Manager is showing OPEN handles versus total handles used by the process?

[Michael Zimmer]

Yes, beahviour depends on SHFileOperation not SetAttr.
No, i don't know exactly what the taskmanager shows. But compared to a normal copy operation started by a logged on user the explorer.exe (i.e. desktop) increases AND decreses handles.

Now i've copied thousands of files past the last hours on testing this strange behaviour. Still i'am not sure if this will affect the program resources (it's a service and it's running all the times 24 x 7).

[Michael Mattias]

Shot in the dark (Plus you need to do this anyway)...

The pTo and pFrom members must be double-$NUL terminated. With your code they will be only single-$NUL terminated if the string length is an integral multiple of the OLE string allocation granularity minus one; in which case SHFileOperation will fail on a file-not-found error.

Also, I think I'd try canceling the delete step to see if this behavior is related to the copy or the delete. Maybe that will give you a clue as to what is going on.

And as long as you are canceling the delete step, see if you can open one or more of the destination files after the copy for exclusive use. If that fails on file in use, well, you've found your problem... but you'll have to come up with 'another way.'

Didn't someone else have some kind of "many files to copy" application here? IIRC, That was not a problem with handles but with doing copies from multiple threads of execution. I can't remember if the 'final' version used ShFileOperation, but I'm pretty sure at least one of the intermediate versions did.

[Cliff Nichols]

Or even something more simple

Do you by chance have "Spybot Search and Destroy" ??? (Otherwise known as "Spybot S&D") ????

If so, then completely shut down all of its processes, and see if the problem goes away???

One other thing you can do is download SysInternals "Process Viewer" to see what is holding these handles open, that you are not aware that they have open.

[Michael Zimmer]

Shot in the dark (Plus you need to do this anyway)...

The pTo and pFrom members must be double-$NUL terminated.

It makes no difference, if strings are one or double $NUL terminated, same result. I tried also to define source and destination as ASCIIZ * %MAX_PATH, same result.

The copy-function does it's work well (instead of FILECOPY, it raises some access errors, that's why i try SHFileOperation), all files can be copied, all files can be accessed, even if opened in exclusive mode (destination only).

[Michael Zimmer]

(1) Spybot Search and Destroy
(2) SysInternals "Process Viewer"

=>1, i'am working on a W2K8 Server (clean installation), but i'll try if Spybot can be executed here
=>2, think you are talking about "Process Explorer" from Sysinternals/Microsoft; yes i've already watched the service and the test program (posted above) with the Process Explorer. Could not find a reason for this strange behaviour.

[Roger Garstang]

Could be any kind of AV/Firewall, maybe even Bit Defender or whatever M$ has came up with lately.

What happens when you access the same files again with the function, does it reuse the other handles or make more? They are most likely gone when the app goes, so not much to test there, but what if you do your calls from a thread that then ends...I'd think the handles should close then.

[Michael Zimmer]

Copying the same folder for many times does not or only slightly increase handles, e.g.: before first copy 165, after first copy 216, next copy 217,..., and then 216 again.
After the backup thread ends, the handles decrease after a while (2-3 minutes, 183).

Looks like there's nothing mor to do, but i'll keep watching this.

Thanks for your help.

[Knuth Konrad]

What you see could simply be some kind of Windows caching mechanism. It keeps some file handles in reserve for your application in case you open another file (not too unlikely).

[Scott Turchin]

Don't forget the null and double null terminations.

THis is the func I've used now, seems easier to send a parameter such as FO_MOVE etc and use one function for everything.

'¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
'Copy file routine, using shell function (with dialog)
'Returns zero if successful, or nonzero otherwise.
'With two exceptions, you cannot use SHFileOperation to move special folders
'from a local drive to a remote computer by specifying a network path.
'--------------------------------------------------------------------
Function ShellCopyFile(ByVal Source As String, ByVal Destination As String,wFunc As Long,wFlags As Long) As Long
  Local lResult     As Long
  Local shfos       As SHFILEOPSTRUCT
  Local ErrType     As Long
  Local ErMsg       As String

  If Right$(Source,2) <> String$(2,0) Then Source = Source & Chr$(0) & Chr$(0)
  If Right$(Destination,2) <> String$(2,0) Then Destination = Destination & Chr$(0) & Chr$(0)

  shfos.hwnd    = %HWND_DESKTOP
  shfos.wFunc   = wFunc   '%FO_COPY or %FO_MOVE or %FO_DELETE or %FO_RENAME
  shfos.pFrom   = StrPtr(Source)
  shfos.pTo     = StrPtr(Destination)
  shfos.fFlags  = %FOF_WANTMAPPINGHANDLE Or wFlags '  'Or %FOF_NOCONFIRMMKDIR Or %FOF_NOCONFIRMATION or %FOF_SILENT Or %FOF_NOCOPYSECURITYATTRIBS

  lResult = SHFileOperation(shfos)
  If lResult <> %ERROR_SUCCESS Or shfos.fAnyOperationsAborted <> 0 Then
     'user aborted, do whatever is needed..
  End If
  SHFreeNameMappings shfos.hNameMappings
  Function = lResult
End Function