Announcement

Collapse
No announcement yet.

Anyone know call an api to get total file counts for a given extension?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #21
    Hey Paul,
    If I remember well, using a recursive function will slow down a lot as the directory deepness increase.

    Here is another example taken from this forum with some little henencements.
    Sadly, I can't find the original thread to give credit...

    Code is non recursive and disable Wow64 redirection.
    Multi filter, like "*.exe/*.dll" may be used.
    Adaptation of Alessandro Cantatore WildMatch routine which is four times faster than PathMatchSpec().
    Etc, etc...

    Code:
    #COMPILE EXE '#Win 9.07#
    #DIM ALL
    #INCLUDE "Win32Api.inc"
    
    DECLARE FUNCTION Wow64DisableWow64FsRedirectionTemplate(BYREF Wow64RedirectionVal AS DWORD) AS LONG
    DECLARE FUNCTION Wow64RevertWow64FsRedirectionTemplate(BYREF OlValue AS DWORD) AS LONG
    '______________________________________________________________________________
    
    FUNCTION WildMatchAC7(BYVAL sString AS STRING, BYVAL sFilter AS STRING) AS LONG
     '(?) Any single character, (*) None or more character, 4 time faster than PathMatchSpec
     '7th routine. C version by by Alessandro Cantatore. http://xoomer.virgilio.it/acantato/dev/wildcard/wildmatch.html#c_cpp_userjournal_algo
     LOCAL pStringStart AS BYTE POINTER
     LOCAL pStringIndex AS BYTE POINTER
     LOCAL pFilterStart AS BYTE POINTER
     LOCAL pFilterIndex AS BYTE POINTER
     LOCAL Star         AS LONG
    
     pStringStart = STRPTR(sString) : pFilterStart = STRPTR(sFilter)
     CharLowerBuff(BYVAL pStringStart, LEN(sString)) : CharLowerBuff(BYVAL pFilterStart, LEN(sFilter))
    
     pStringIndex = pStringStart : pFilterIndex = pFilterStart
     DO
       IF @pStringIndex = 0 THEN EXIT DO
    
       SELECT CASE @pFilterIndex
    
         CASE 42 '(*)
           Star = %TRUE
           pStringStart = pStringIndex : pFilterStart = pFilterIndex
           WHILE @pFilterStart = 42 '(*) For consecutive (*). While-wend may be removed if multi* is shrinked to single* before calling the function
             INCR pFilterStart
           WEND
           IF @pFilterStart = 0 THEN FUNCTION = %TRUE : EXIT FUNCTION
           pStringIndex = pStringStart : pFilterIndex = pFilterStart
           ITERATE DO
    
         CASE 63 '(?)
           'IF @pStringIndex = 46 THEN '(.) Do not consider dot under (?).
           '  IF star = 0 THEN EXIT FUNCTION 'FUNCTION = %FALSE
           '  INCR pStringStart
           '  pStringIndex = pStringStart : pFilterIndex = pFilterStart
           '  ITERATE DO
           'END IF
    
         CASE ELSE
           IF @pStringIndex <> @pFilterIndex THEN
             IF star = 0 THEN EXIT FUNCTION 'FUNCTION = %FALSE
             INCR pStringStart
             pStringIndex = pStringStart : pFilterIndex = pFilterStart
             ITERATE DO
           END IF
    
       END SELECT
       INCR pStringIndex : INCR pFilterIndex
     LOOP
    
     WHILE @pFilterIndex = 42 '(*)
       INCR pFilterIndex
     WEND
    
     IF @pFilterIndex = 0 THEN FUNCTION = %TRUE
    
    END FUNCTION
    '______________________________________________________________________________
    
    FUNCTION FileEnum(BYVAL sFolder AS STRING, BYVAL sFilter AS STRING, _
                      BYREF FileDataArray() AS WIN32_FIND_DATA) AS LONG
     LOCAL FileData            AS WIN32_FIND_DATA
     LOCAL hFile               AS DWORD
     LOCAL hLib                AS DWORD
     LOCAL pProc               AS DWORD
     LOCAL uBoundSubFolder     AS DWORD
     LOCAL uBoundFileDataArray AS DWORD
     LOCAL Wow64RedirectionVal AS DWORD
     LOCAL SubFolderCount      AS LONG
     LOCAL FileCount           AS LONG
     LOCAL FilterIndex         AS LONG
     LOCAL FilterCount         AS LONG
     LOCAL Looper              AS LONG
    
     REDIM FileDataArray(0 TO 50000) AS WIN32_FIND_DATA
     uBoundFileDataArray = UBOUND(FileDataArray())
    
     DIM zSubFolder(0 TO 1000) AS ASCIIZ * %MAX_PATH
     uBoundSubFolder = UBOUND(zSubFolder)
    
     FilterCount = PARSECOUNT(sFilter, "/")
     DIM sFilterArray(1 TO FilterCount) AS STRING
     PARSE sFilter, sFilterArray(), "/"
     FOR Looper = 1 TO FilterCount
       sFilterArray(Looper) = TRIM$(sFilterArray(Looper))
     NEXT
    
     IF ASC(sFolder, - 1) <> 92 THEN sFolder = sFolder & "\"
    
     'Wow64DisableWow64FsRedirection(Wow64RedirectionVal) 'XPpro64 - Vista - Server2003sp1
     ' The following Wow64DisableWow64FsRedirection disable file system redirection
     'so that a 32-bit application that is running under WOW64 can open
     'the 64-bit version of files in %SystemRoot%\System32
     'instead of being redirected to the 32-bit version in %SystemRoot%\SysWOW64.
     ' Disables file system redirection for the calling thread. File system redirection is enabled by default.
     'IF Wow64DisableWow64FsRedirection(Wow64Value) THEN Wow64DisableWow64FsRedirection(Wow64RedirectionVal)
     hLib = LoadLibrary("Kernel32.dll")
     IF hLib THEN
       pProc = GetProcAddress(hLib, "Wow64DisableWow64FsRedirection")
       IF pProc THEN
         CALL DWORD pProc USING Wow64DisableWow64FsRedirectionTemplate(Wow64RedirectionVal)
       END IF
       'See below FreeLibrary(hLib)
     END IF
    
     DO
       hFile = FindFirstFile(sFolder & "*", FileData)
       IF hFile <> %INVALID_HANDLE_VALUE THEN
         DO
           IF (FileData.dwFileAttributes AND %FILE_ATTRIBUTE_DIRECTORY) THEN
             IF ASC(FileData.cFileName, -1) <> 46 THEN 'Check for "." and "..", only those can end with a dot.
               INCR SubFolderCount
               IF SubFolderCount > uBoundSubFolder THEN
                 uBoundSubFolder = uBoundSubFolder + 1000
                 REDIM PRESERVE zSubFolder(0 TO uBoundSubFolder)
               END IF
               zSubFolder(SubFolderCount) = sFolder & FileData.cFileName & "\"
             END IF
           ELSE 'It's a file.
             FOR FilterIndex = 1 TO FilterCount
               'If .cFilename is ShortFN then .cAlternateFileName will be empty
               'If .cFilename is LongFN  then .cAlternateFileName will contain SFN
               IF WildMatchAC7(FileData.cFileName, sFilterArray(FilterIndex)) THEN 'Much faster than PATHMATCHSPEC
                 'When we copy the temp UDT to the matched file array make sure to include the full path
                 FileDataArray(FileCount) = FileData
                 FileDataArray(FileCount).cFileName = sFolder & FileData.cFileName
                 INCR FileCount
                 IF FileCount > uBoundFileDataArray THEN
                   uBoundFileDataArray = uBoundFileDataArray + 50000
                   REDIM PRESERVE FileDataArray(0 TO uBoundFileDataArray)
                 END IF
                 EXIT FOR
               END IF
             NEXT
           END IF
         LOOP WHILE FindNextFile(hFile, FileData)
         FindClose(hFile)
       END IF
       IF SubFolderCount = 0 THEN EXIT LOOP
       sFolder = zSubFolder(SubFolderCount)
       DECR SubFolderCount
     LOOP
    
     'Wow64RevertWow64FsRedirection(Wow64RedirectionVal) 'XPpro64 - Vista - Server2003sp1
     IF hLib THEN
       pProc = GetProcAddress(hLib, "Wow64RevertWow64FsRedirection")
       IF pProc THEN
         CALL DWORD pProc USING Wow64RevertWow64FsRedirectionTemplate(Wow64RedirectionVal)
       END IF
       FreeLibrary(hLib)
     END IF
    
     REDIM PRESERVE FileDataArray(FileCount)
     FUNCTION = FileCount
    
    END FUNCTION
    '______________________________________________________________________________
    
    FUNCTION PBMAIN() AS LONG
     DIM   FileDataArray2(0 TO 0) AS WIN32_FIND_DATA
     LOCAL sFolder                AS STRING
     LOCAL sFilter                AS STRING
     LOCAL sLog                   AS STRING
     LOCAL TotalByte              AS QUAD
     LOCAL FileCount              AS DWORD
     LOCAL TimeStart              AS DWORD
     LOCAL Looper                 AS LONG
    
     sFolder = "C:\Program Files\"
    
     'Filters are separated by forward slash "/"
     sFilter = "*.exe/*.dll"
    
     TimeStart = GetTickCount()
    
     FileCount = FileEnum(sFolder, sFilter, FileDataArray2())
    
     sLog = FORMAT$((GetTickCount() - TimeStart) / 1000, "###,###.000") & " seconds elapsed" & $CRLF & $CRLF
     sLog &= "Folder: " & $TAB & sFolder & $CRLF & "Filter: " & $TAB & sFilter & $CRLF & $CRLF
    
     FOR Looper = 0 TO MIN(15, FileCount - 1)
       sLog &= FORMAT$(Looper, "0000") & ") " & FileDataArray2(Looper).cFileName & $CRLF
       TotalByte = TotalByte + MAK(QUAD, FileDataArray2(Looper).nFileSizeLow ,FileDataArray2(Looper).nFileSizeHigh)
     NEXT
     sLog &= "..." & $CRLF & $CRLF & "File count " & FORMAT$(FileCount - 1, "#,#") & _
                     $CRLF & $CRLF & "Total byte " & FORMAT$(TotalByte, "#,#") & $CRLF
    
     MessageBox(%HWND_DESKTOP, (sLog), "FINDFIRS/NEXTTFILE", %MB_OK OR %MB_TOPMOST)
    
    END FUNCTION
    '______________________________________________________________________________
    '
    Last edited by Pierre Bellisle; 14 Apr 2021, 05:42 PM.

    Comment


    • #22
      Wildcard matching algorithms by Alessandro Cantatore

      Comment


      • #23
        Using fast recursive file directory listing code from Wayne Diamond.
        Recursion optimised (got rid of subdir array) ... aka. we've got a stack, lets use it
        https://forum.powerbasic.com/forum/u...rce-code/60381


        makes you want to forget about MFT methods of search
        I had used this code earlier today.
        While testing the code, i came across some files that seemed to be actually directories.
        Unremark this line
        IF RIGHT$(sfullfilename,1&)="" THEN STDOUT sfullfilename
        to see questionable results in the file area.
        I had added a line that test for the file length name to be greater than zero in some test code.
        I left those lines in this code because i wanted to bring a little attention to some of my finding.
        I do not want to add comments on this code here.
        Code:
        #COMPILE EXE
        #BREAK ON
        #INCLUDE "win32api.inc"
        
        GLOBAL icount AS LONG
        GLOBAL sfileextensionmatch AS STRING
        
        
        SUB EnumFiles (sPath AS STRING)
          LOCAL hSearch AS LONG, fRes AS LONG, WFD AS WIN32_FIND_DATA
          LOCAL sfullfilename AS STRING
        
        
          hSearch = FindFirstFile(sPath & "\*.*", WFD)
          IF hSearch = %INVALID_HANDLE_VALUE THEN EXIT SUB
          DO
             fRes = WFD.dwFileAttributes
             IF (fRes AND %FILE_ATTRIBUTE_REPARSE_POINT) THEN GOTO FindNext  '// Junction found
             IF (fRes AND %FILE_ATTRIBUTE_DIRECTORY) = 0 THEN GOTO FoundFile '// Now ensure its not "." or ".." ...
             IF (PEEK(LONG, VARPTR(WFD.cFileName)) OR &h00002E2E) = &h00002E2E THEN GOTO FindNext
               '? "PATH=" & sPath & "\" & WFD.cFileName                 '// FOLDER found
              ' stdout sPath & "\" & WFD.cFileName
              ' incr icount
                                '// FOLDER found
                 EnumFiles sPath & "\" & WFD.cFileName
                 GOTO FindNext
             FoundFile:                                                '// FILE found
             '?  " FILE=" & sPath & "\" & WFD.cFileName
                 sfullfilename=sPath & "\" & WFD.cFileName
                 WHILE INSTR(sfullfilename,"\\")
                    REPLACE "\\" WITH "\" IN sfullfilename
                 WEND
                 IF LEN(WFD.cFileName)>LEN(sfileextensionmatch) THEN
                     IF RIGHT$(WFD.cFileName,LEN(sfileextensionmatch))=sfileextensionmatch  THEN
                        ' STDOUT sfullfilename
                         INCR icount
                     END IF
                 END IF
        
                 'IF RIGHT$(sfullfilename,1&)="\" THEN STDOUT sfullfilename
        
           FindNext:
             fRes = FindNextFile(hSearch, WFD)                         '// Find next file/folder
          LOOP WHILE fRes
          fRes = FindClose(hSearch)
        END SUB
        
        
        FUNCTION PBMAIN() AS LONG
        ' stdout "results of files that are directories"
         sfileextensionmatch=UCASE$(".bas")
        
         EnumFiles "C:\"   '// call without a trailing backslash, eg. C:, C:\Windows, etc
        STDOUT STR$(icount)+ " files with extension "+ sfileextensionmatch
        ' ? "DONE":  WAITKEY$
        END FUNCTION
        p purvis

        Comment


        • #24
          from Jose Roca pb apis
          AfxFile.inc

          Code:
          ' Returns the number of files that meet the specified criteria.
          ' Parameters:
          ' - bstrPath = The path of the folder to search.
          ' Can include a drive name and system wildcard characters (* and ?).
          ' - bstrExt = The file extension
          ' - nAttribute = Includes filenames with specific attributes in the search, in addition to normal files.
          ' If the value is negative, only files with the specific attributes are serached.
          ' Usage example: nCount = AfxGetFileCount("C:\Tests\*.bas)
          ' ========================================================================================
          FUNCTION AfxGetFileCount (BYVAL bstrPath AS WSTRING, OPTIONAL BYVAL bstrExt AS WSTRING, OPTIONAL BYVAL nAttribute AS LONG) AS LONG
          
          LOCAL nCount AS LONG
          LOCAL bOnly AS LONG
          LOCAL bstrFile AS WSTRING
          
          bstrPath = TRIM$(bstrPath)
          IF bstrExt = "" THEN bstrExt = "*.*"
          IF bstrPath = "" THEN EXIT FUNCTION
          IF RIGHT$(bstrPath, 1) <> "\" THEN bstrPath += "\"
          bstrExt = TRIM$(bstrExt)
          IF nAttribute < 0 THEN bOnly = 1
          nAttribute = ABS(nAttribute)
          IF bOnly = 0 THEN
          bstrFile = DIR$(bstrPath & bstrExt, nAttribute)
          ELSE
          bstrFile = DIR$(bstrPath & bstrExt, ONLY nAttribute)
          END IF
          WHILE LEN(bstrFile)
          INCR nCount
          bstrFile = DIR$ (NEXT)
          WEND
          
          FUNCTION = nCount
          
          END FUNCTION
          enjoy

          Comment


          • #25
            Perhaps not the solution you want, but check out the thread where I talked about using the command line es.exe to access the Everything database. You can get your answer in literally a couple of seconds with that approach.

            Otherwise, you're faced with API based iterative solutions.

            Comment


            • #26
              not necro enough
              Dale

              Comment


              • #27
                I was thinking there might be a way to directly read the directory / file names as a chunk of data from the disk.
                Hard to believe there is nothing better than FindFirstFile, FindNextFile.




                Comment


                • #28
                  "Everything" has an SDK and command line options.
                  https://www.voidtools.com/support/everything/sdk/

                  Might even ask this question on the voidtools forum.

                  Gary mentioned this in post #6 and #25

                  Comment


                  • #29
                    Originally posted by David Clarke View Post
                    I was thinking there might be a way to directly read the directory / file names as a chunk of data from the disk.
                    Hard to believe there is nothing better than FindFirstFile, FindNextFile.
                    For a start, you'd have to write a different function for each format (FAT32,eFAT,NTFS and maybe HFS+)

                    You might like to look at NTFS's MFT:
                    https://docs.microsoft.com/en-us/win...ter-file-table

                    Comment


                    • #30
                      Anyone know if it is possible to call a windows api to get total file counts for a given extension?
                      Is this just a concocted requirement? Or is there some application which needs to know this? Or, what exactly needs to be known that you believe obtaining a list of all files with a given extension is the best solution? And why the limitation of "call a Windows API function?"

                      MCM
                      Michael Mattias
                      Tal Systems (retired)
                      Port Washington WI USA
                      [email protected]
                      http://www.talsystems.com

                      Comment


                      • #31
                        https://www.andreafortuna.org/2017/0...fs-filesystem/

                        MFT is a special system file that resides on the root of every NTFS partition, named $MFT and not accessible via user mode API’s.
                        However it can been seen when you have raw access to the disk (e.g, forensic image or specific tools).
                        This special file is the heart of NTFS and contains entries for every file and directory on disk.

                        Comment


                        • #32
                          The reason I brought this up is I am working on a program to clean and organize zillions of files. The very first thing I was looking for was total file count for a disk so I could estimate the time required or update the progress bar. I totally expected to find a function named something like GetFileCount(Drive, Extension) .

                          Naturally I got stuck and obsessed with this. There are lots of ways to give the user feedback without knowing the total file count.

                          I spent some time on the MFT thing and got no place. Then I had to stop obsessing and go back to the work that pays!

                          I would have thought the operating system might have maintained some information.

                          So my new obsession is writhing a windows service that keeps track of such things in the background. I figure that is how EVERHTHING.EXE does it.


                          Comment


                          • #33
                            All the source code is there with Everything32.DLL and Everything64.Dll
                            There are also examples in C++ and VisualBasic (not VB6).
                            Scroll to botton for the command line interface, sdk and offline help everything.chm.zip

                            https://www.voidtools.com/downloads/

                            Comment

                            Working...
                            X