Announcement

Collapse

Forum Guidelines

This forum is for finished source code that is working properly. If you have questions about this or any other source code, please post it in one of the Discussion Forums, not here.
See more
See less

Memory Mapped File version of LINE INPUT (to callback procedure)

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Memory Mapped File version of LINE INPUT (to callback procedure)

    include file for use with pb/win or pb/cc

    Code:
    ' file: mmfli.inc
    ' function to use a memory-mapped file in lieu of line input
    ' author  : michael mattias tal systems inc racine wi
    ' placed in public domain by author.
    ' date    : may 31 2004
    ' compiler: pb/windows v 7.02 (full test); pb/cc 3.03 (for clean compile only)
    ' winapi header file win32api.inc: may 9 2002
    ' -------------------------------------------------------
    ' this program demonstrates the use of the function "mmf_li-cb"
    ' quite simply what this function does is parse any file to find records delimted by cr, lf or
    ' any combination of cr and lf, and sends the resulting records to a user-defined callback function
    ' along with an action code and a user parameter value.
    
    ' this function can be used for...
    ' a. parsing large files without the need to load the entire file into a string. this produces a dramatic
    '    reduction in resource requirements. (see: http://www.powerbasic.com/support/pb...ad.php?t=23526 
    ' b. handling unix-delimited files (x'0a') identically to pc-delimited files (x'0d0a') without
    '    knowing or caring which format the file is in
    ' c  avoiding two bugs in the powerbasic line input # function:
    '    1. if file is a unix-delimited file, if the file is too big line input fails to return
    '    2. line input terminates a record on cr, not crlf as documented
    '
    ' note: as written, this software ignores blank lines. the pb line input function will, by design, return a
    '       null string if found in the file.
    '
    '----------------------------------------------------------------------
    ' user-called function:
    declare function mmf_li_cb (szfile as asciiz, byval cbaddr as dword, byval dwuser as dword) as long
    ' --------------------------------------------------------------------------------------------
    ' parameters
    '  szfile  [in]   = file to be processed
    '  cbaddr  [in]   = address of callback function
    '  dwuser  [in]   = user-defined value passed to callback function on each call
    ' returns  :  0      = success, enumeration completed
    '             other  = enumeration never completed, either because of error or user request
    ' input file: records are delimited by  a cr (x0d) or lf (x0a); any mutliple occurrences of
    ' either delimiter is treated as a single delimiter.
    '
    '  callback function prototype and parameters
    ' ----------------------------------------------------
      declare function mmf_li_cbfunction (byval statuscode as long, byval dwoffset as long, sdata as string, byval dwuser as dword) as dword
    ' status codes and dwoffset:
    ' when status code is      dwoffset is                         sdata is
    ' -------------------      -----------                         --------------------
    ' %mmfli_error             an error code                       n/a
    ' %mmfli_filesize          size of file                        n/a
    ' %mmfli_data              offset from zero of the cr or      "the data"
    '                          lf which terminated this record
    '                          i.e., how much of the file has been processed
    ' %mmfli_eof               total number of chars processed      n/a
    
    ' the callback is called
    '   - when the file is successfully mapped, called with %mmfli_filesize
    '   - for each line of data, called with %mmfli_data. sdata will never be null
    '   - at end of file, called with mmfli_eof
    '   - on error, called with mmfli_error. return value is ignored in this case; enumeration stops.
    '
    '  the callback function should return true (nonzero) to continue enumeration or
    '  false (0) to terminate the enumeration.
    ' ------------------------------------------------------------------------------------------------
    
    
    #if not %def(%winapi)
      #include "win32api.inc"
    #endif
    #if not %def (%invalid_handle_value_long)
       %invalid_handle_value_long    = -1&
    #endif
    
    
    ' =====================================================================
    ' file mapping functions used by the main user function
    ' these file mapping functions are available as a 'standalone' #include file at:
    ' http://www.powerbasic.com/support/pb...ad.php?t=24225 
    
    ' memory map a file for input only
    ' returns: true = file was mapped and return value is the base pointer for the mapped object.
    ' false=error  (e.g.,file not found, memory problems)
    ' parameters:
    '  szfilein     [in]  file to be mapped
    '  hsys         [out] control value needed to unmap the file
    '  hfilemapping [out] control value needed to unmap the file
    '  cbmapping    [out] size (number of bytes) of mapped file
    
    function mapthisfile (szfilein as asciiz, hsys as long, hfilemapping as long, cbmapping as long) as dword
     ' note we need the values of hsys, hfilemapping and pifdata to unmap the file
    
      local w32 as win32_find_data, hsearch as long
      local pifdata as dword
    
      ' check that we can find the file, and get its size if we can
      hsearch      = findfirstfile (szfilein, w32)
      if hsearch <> %invalid_handle_value_long then
          cbmapping    =  w32.nfilesizelow
          closehandle     hsearch
      else
          function = 0
          exit function
      end if
    
      ' open the file for reading and get system handle. we could open with pb open and then open handle but
      ' that would just complicate this
      hsys          = createfile (szfilein, %generic_read, %file_share_read, byval %null, %open_existing, %file_attribute_normal, byval %null)
      if istrue hsys  then
         hfilemapping     = createfilemapping (hsys, byval %null, %page_readonly, byval %null, byval %null, byval %null)
         if istrue hfilemapping then
             ' map the file to our memory space and get pointer to data...
             pifdata    = mapviewoffile (hfilemapping, %file_map_read, byval %null, byval %null, byval %null)
             function   = pifdata
         else
             function    = 0
             closehandle hsys
             exit function
         end if
      else
          function = 0
          exit function
      end if
    
    end function
    
    function unmapmappedfile (pifdata as dword, hsys as long, hfilemapping as long) as long
        unmapviewoffile pifdata
        closehandle     hfilemapping
        closehandle     hsys
    end function
    
    ' =============================================
    ' memory-mapped files line input functions
    '             and equates
    '==============================================
    
    ' callback status codes:
    %mmfli_filesize  =  1&
    %mmfli_data      =  2&
    %mmfli_eof       =  3&
    %mmfli_error     = 10&
    
    'parsing constants
    %mmfli_cr        = &h0d?
    %mmfli_lf        = &h0a?
    ' error codes
    %mmfli_cantmapfile  = 101&
    
    ' --------------------------------------------'
    '            user-called function
    ' --------------------------------------------'
    
    function mmf_li_cb (szfile as asciiz, byval cbaddr as dword, byval dwuser as dword) as long
    
     local  pifdata as dword, hsys as long, hfilemapping as long, cbmapping as long
     local  psrc as byte ptr, ncharthisrecord as long, nchardone as long
     local  cbstatus as long, s as string, dwoffset as dword, cbret as long
     local  inrecord as long, dwrecstart as long
    
       ' map the file to memory
       pifdata  = mapthisfile (szfile, hsys, hfilemapping, cbmapping)
    
       ' ---------------------------------------------------------------------------------------
       ' call the callback function, either send filesize if mapping successful or error if not
       ' ---------------------------------------------------------------------------------------
    
       if istrue  pifdata then
           cbstatus  =  %mmfli_filesize
           dwoffset  =  cbmapping
       else
           cbstatus  =  %mmfli_error
           dwoffset  =  %mmfli_cantmapfile
       end if
       call dword       cbaddr using mmf_li_cbfunction (cbstatus, dwoffset, s, dwuser)  to cbret
    
    
       ' if we had an error mapping the file, we exit
       if isfalse pifdata then
           function = %true             ' set non-success return value
           exit function
       end if
    
       ' did user want to terminate the enumeration after seeing the filesize?
       ' if so, free the resources we allocated and exit with 'abort' return value
       if isfalse cbret then
           unmapmappedfile  pifdata, hsys, hfilemapping
           function     = %true           ' set abort code
           exit function
       end if
    
      ' if we get here, we have mapped the file and the user did not ask for the enumeration to stop.
    
      psrc            = pifdata     ' point to start of filedata
      dwoffset        = 0           ' get to zero-based value
      ncharthisrecord = 0           ' well, not yet anyway
      nchardone       = 0
      inrecord        = %false
      cbret           = %true       '
    
    
      do while nchardone < cbmapping    ' for each character of the file
    
          select case as const @psrc
    
              case %mmfli_cr, %mmfli_lf             ' terminates a record if we are in one
                  if istrue inrecord then
                      ' either cr or lf ends a record, and any number of consecutive cr or lf counts as one (1) delimiter
                      ' that is, a null string is never sent to the callback function
                          cbstatus      = %mmfli_data
                          s             = peek$(dwrecstart, ncharthisrecord)
                          call dword      cbaddr using mmf_li_cbfunction (cbstatus, dwoffset, s, dwuser)  to cbret
                          ' does user want to terminate?
                          if isfalse cbret then
                              exit do
                          end if
                          inrecord  = %false
    
                   'else         '  we are not currently building a record, so this is just another occurrence
                                 '  of the delimiter, which we treat as a single record delimiter
                                 ' this is what would have to be changed a bit to send null strings when they occur
    
                    end if
    
              case else                ' a non-delimiter character
    
                  if inrecord then     '
                      incr ncharthisrecord   ' keep track of how many characters we will have to move later
    
                  else                          ' start a new record
                      inrecord         = %true
                      dwrecstart       = psrc
                      ncharthisrecord  = 1      ' because this character is included
                  end if
    
           end select
           ' and increment our offset and other control values
           incr psrc
           incr dwoffset
           incr nchardone
    
    
       loop
       
       ' --------------------------------------------------------------
       ' when we get here, we either have no more characters to process
       ' or the user asked for enumeration to end
       ' --------------------------------------------------------------
    
       if istrue cbret then                ' we were not forced out of loop by user request
         ' if we were building a record but file did not end with a cr or lf, send a data callback
          if inrecord    then
                 cbstatus      = %mmfli_data
                 s             = peek$(dwrecstart, ncharthisrecord)
                 call dword      cbaddr using mmf_li_cbfunction (cbstatus, dwoffset, s, dwuser)  to cbret
          end if
          ' send the end of file message:
          cbstatus      = %mmfli_eof
          s             = "
          call dword      cbaddr using mmf_li_cbfunction (cbstatus, dwoffset, s, dwuser)  to cbret
         ' we just ignore the return value here, since we won't be sending any more messages anyway
       end if
    
      ' and, whenever we get this far, we've mapped the file so we must clean up
       unmapmappedfile  pifdata, hsys, hfilemapping
    
       ' and set function return value; 0 if enumeration completed, nonzero otherwise
       function =  isfalse (cbret)  ' if cbret is true function returns zero (success)
    
    end function
    
    ' ====================================================================================================
    ' simple callback function provided for reference. dwuser (user supplied value) is a long passed by
    ' reference and simply counts records. pass to main user function as "byval varptr(nrec)"
    ' or the equivalent.
    ' this function writes each record to a pc-delimited (crlf) text file. this output may then be
    ' used safely by the powerbasic "line input #" functions.
    ' ====================================================================================================
    
     function mmfli_prototype_callback_function (byval statuscode as long, byval dwoffset as long, sdata as string, nrec as long) as dword
    
      static hfile as long
    
        select case as const statuscode
          case %mmfli_filesize
              hfile          = freefile
              open             "mmfli_pc_file.txt" for output as hfile
              function       = %true       ' continue enumeration
           case %mmfli_data
               print           #hfile, sdata
               incr             nrec
               function      = %true
            case %mmfli_eof
               close            hfile
               hfile         = 0                   ' reset static
               function      = %true
            case %mmfli_error
              function = 0    ' false to terminate true to continue enumeration
              ' not really necessary to process this message, as the enumeration function
              ' itself will exit immediately on error
        end select
    
     end function
    
    
    ' *** end of file mmfli.inc
    ------------------
    michael mattias
    tal systems inc.
    racine wi usa
    mailto:[email protected][email protected]</a>
    www.talsystems.com
    Michael Mattias
    Tal Systems (retired)
    Port Washington WI USA
    [email protected]
    http://www.talsystems.com

  • #2
    PB/WIN 7.02 Demo Program
    Code:
    ' FILE: MMFLIDEMO.BAS
    ' Using a memory-mapped file in lieu of LINE INPUT
    ' Author  : Michael Mattias Tal Systems Inc Racine WI
    ' Placed in public domain by author.
    ' Date    : May 31 2004
    ' Compiler: PB/Windows v 7.02
    ' WinAPI Header file Win32API.INC: May 9 2002
    ' Use     : Select file, click one of the 'do it' buttons.
    ' -------------------------------------------------------
    ' This program demonstrates the use of the function "mmf_li_cb" found in
    ' the #INCLUDE file "mmfli.inc"
    '-------------------------------------------------------
    
    #COMPILE  EXE
    #DEBUG    ERROR ON
    #REGISTER NONE
    #DIM      ALL
    #TOOLS    OFF
    '=====[Windows API Header Files] ============================
    '  If you don't need all of the functionality supported by these APIs
    '  (and who does?), you can selectively turn off various modules by putting
    '  the following constants in your program BEFORE you #include "win32api.inc":
    '
    '  %NOGDI = 1     ' no GDI (Graphics Device Interface) functions
    '  %NOMMIDS = 1   ' no Multimedia ID definitions
    
    %NOMMIDS = 1
    #INCLUDE "WIN32API.INC"
    '==[End Windows API Header Files]============================
    
    #IF NOT %DEF (%INVALID_HANDLE_VALUE_LONG)
       %INVALID_HANDLE_VALUE_LONG    = -1&
    #ENDIF
    
    ' we will use the Powerbasic supplied "openfiledialog" so we need the #INCLUDE:
    ' The DECLARE is just here for convenience
    #INCLUDE "COMDLG32.INC"
     DECLARE _
       FUNCTION OpenFileDialog (BYVAL hWnd AS DWORD, _           ' parent window
                             BYVAL Caption AS STRING, _       ' caption
                             Filespec AS STRING, _            ' filename
                             BYVAL InitialDir AS STRING, _    ' start directory
                             BYVAL Filter AS STRING, _        ' filename filter
                             BYVAL DefExtension AS STRING, _  ' default extension
                             Flags AS DWORD _                 ' flags
                            ) AS LONG
    
    ' returns selected file in filespec when TRUE
    
    ' ===============================================================================================
    '                          FILE MAPPING ENCAPSULATION FUNCTIONS
    ' ===============================================================================================
    #INCLUDE "MMFLI.INC"
    
    ' ==========================
    ' DEMO PROGRAM STARTS HERE
    ' ==========================
    
    ' -----------------------------------------------
    ' DIALOG CONSTANTS
    
    %ID_FILENAME         =     101
    %ID_FILENAME_BROWSE  =     102
    %ID_MESSAGE          =     103
    %ID_PROCESS_1        =     104     ' simple message processing with echo to screen
    %ID_PROCESS_2        =     105
    %ID_PROCESS_3        =     106
    %ID_SHAMELESS        =     998     ' you'll see how I picked this name for an equate
    %ID_QUIT             =     999
    
    ' UDT used in both dialog and MMFLI callback functions
    TYPE MMFLIType
        hDlg      AS LONG
        nRec      AS LONG
        nChar     AS LONG
        pdwS      AS STRING PTR            ' used in process #2
        nMod      AS LONG                  ' you'll see how used
        hFile     AS LONG                  ' another test value
    END TYPE
    
    %PROCESS_1_NUMBER_UPDATES = 20         ' number of times we update the screen during  process # 1
    ' ==========================================================
    ' callback for process #1, dwUser = Address of mmfli UDT
    ' All this function does is update the dialog message %PROCESS_1_NUMBER_UPDATES
    ' times during the course of processing the file. (Updating all could take forever)
    ' ===========================================================
    
     FUNCTION Process_1_Function (BYVAL StatusCode AS LONG, BYVAL dwOffset AS LONG, sData AS STRING, mmfli AS MMFLIType) AS DWORD
    
      STATIC lFileSIze AS LONG '
      LOCAL  BlockNo   AS LONG, s AS STRING
    
        SELECT CASE AS CONST statuscode
          CASE %MMFLI_FILESIZE
              lFileSize  = dwOffset   ' remember the filesize, we'll need it
              FUNCTION   = %TRUE      ' yes, continue please
    
          CASE %MMFLI_DATA
              ' in which block of data are we? We want to update the screen when we get to a new block
              BlockNo  = (dwOffset \ (lFilesize\ %PROCESS_1_NUMBER_UPDATES)) + 1
              IF BlockNo <> mmfli.nMod THEN
                  s =  USING$ ("Block ###   ###,###,### of ####,###,### characters", BlockNo, dwOffset, lFileSize)
                  s =  s & $CRLF & sData
                  CONTROL SET TEXT  mmfli.hdlg, %ID_MESSAGE, S
                  DIALOG DOEVENTS
                  mmfli.nmod  = BlockNo
              END IF
    
              INCR  mmfli.nrec   ' increment the counter we will show at end
    
              FUNCTION = 1       ' true to continue enumeration
    
            CASE %MMFLI_EOF
               FUNCTION = 1    ' true to continue enumeration (Not that we would expect any more calls)
               ' save the total number of characters processed
               mmfli.nchar    = dwOffset
    
    
            CASE %MMFLI_ERROR
               ' terminate enum.. handled by the function itself, which will refuse to continue
               ' However, user may want to do something special here
                FUNCTION = 0    ' false to terminate true to continue enumeration
    
        END SELECT
    
     END FUNCTION
    
    ' ====================================================================================================
    ' callback for process #2, dwUser = Address of mmfli UDT
    ' This function will cease the enumeration when a line containing the string in [email protected] is found
    ' ====================================================================================================
    
     FUNCTION Process_2_Function (BYVAL StatusCode AS LONG, BYVAL dwOffset AS LONG, sData AS STRING, mmfli AS MMFLIType) AS DWORD
    
      STATIC lFileSIze AS LONG '
      LOCAL  BlockNo   AS LONG, s AS STRING
      LOCAL  iLoc      AS LONG
    
        SELECT CASE AS CONST statuscode
          CASE %MMFLI_FILESIZE
              lFileSize  = dwOffset   ' remember the filesize, we'll need it
    
              'MSGBOX "Search String is " & [email protected]   ' it has this just fine!
    
              FUNCTION  = %TRUE       ' continue enumeration
    
          CASE %MMFLI_DATA
    
              ' Update the screen...
              ' in which block of data are we? We want to update the screen when we get to a new block
              BlockNo         = (dwOffset \ (lFilesize\ %PROCESS_1_NUMBER_UPDATES)) + 1
              IF BlockNo <> mmfli.nMod THEN
                  s       =  USING$ ("Block ###   ###,###,### of ####,###,### characters", BlockNo, dwOffset, lFileSize)
                  s       =  s & $CRLF & sData
                  CONTROL SET TEXT  mmfli.hdlg, %ID_MESSAGE, S
                  DIALOG  DOEVENTS
                  ' SLEEP here will make this a little easier to follow...if using smaller files
                  mmfli.nmod  = BlockNo
              END IF
    
              INCR  mmfli.nrec   ' increment the counter we will show at end
    
              ' Does this line contain  (non-case-Sensitive) our target string? If so, terminate our enumeration now
              ' and find offset in file where it was found
              ' Because dwoffset when this is called is the end-of record delimiter, we have to go backward
              ' in the file to find where the record starts.
    
              iLoc             = INSTR(LCASE$(sData), LCASE$([email protected]))
              IF ISTRUE iLoc THEN
                  mmfli.nchar  =  dwOffset - LEN(sData) + iloc -1   ' where was string found?
                  FUNCTION     =  %FALSE                            ' terminate the enum now
              ELSE
                  FUNCTION     =  %TRUE
              END IF
    
    
            CASE %MMFLI_EOF
    
               mmfli.nchar     = dwOffset    ' should agree with filesize
               FUNCTION = 1    ' true to continue enumeration (Not that we would expect any more calls)
    
    
            CASE %MMFLI_ERROR
               ' terminate enum.. handled by the function itself, which will refuse to continue!
               ' However, user may want to do something special here
                FUNCTION = 0    ' false to terminate true to continue enumeration
    
        END SELECT
    
     END FUNCTION
    
    
    ' -------------------------
    ' DIALOG CALLBACK FUNCTION
    ' -------------------------
    CALLBACK FUNCTION DlgCallback()
    
      LOCAL mmFLI AS mmflitype, dwUser AS DWORD, dwAddr AS DWORD, szFile AS ASCIIZ * %MAX_PATH
      LOCAL s AS STRING, lret AS LONG, stopwhenFound AS STRING
      LOCAL flags AS DWORD
    
        SELECT CASE AS LONG CBMSG
    
           CASE %WM_CTLCOLORSTATIC
               IF GetDlgCtrlId (CBLPARAM) = %ID_MESSAGE  THEN
                  FUNCTION           = GetStockObject (%WHITE_BRUSH)
               ELSEIF  GetDlgCtrlId (CBLPARAM) = %ID_SHAMELESS THEN
                   SetTextColor         CBWPARAM, %RED
                   FUNCTION           = GetStockObject (%WHITE_BRUSH)
               ELSE
                   FUNCTION = 0  ' take default
               END IF
               EXIT FUNCTION
    
           CASE  %WM_COMMAND
              SELECT CASE AS LONG CBCTL
                 CASE %ID_PROCESS_1
                     ' set up our callback parameters
                     MMFLI.hDlg  =  CBHNDL
                     MMFLI.nRec  =  0
                     dwUSer      =  VARPTR (mmfli)  ' pass address of our parameter block
                     ' set callback address:
                     dwAddr      =  CODEPTR (Process_1_Function)
                     ' get the file name into ASCIIZ
                     CONTROL GET TEXT CBHNDL, %ID_FILENAME TO s
                     szFile     = s
                     ' call the mmf line input function
                     lRet       =  mmf_li_cb (szFile, dwAddr, dwUser)
                     ' show the completion status in the text message area
                     s          =     "Process #1: Simple record counter with screen update" & $CRLF
                     s          = s & "Return code  " & STR$(lRet)  & $CRLF
                     ' if success, show the number of records, else show error message
                     s          =  S & USING$("Processed  #,###,### Bytes and sent #,###,### records to callback", mmfli.nChar, mmfli.nrec) & $CRLF
                     s          =  s & IIF$(lret, "Enumeration Terminated", "Enumeration Completed")
                     CONTROL      SET TEXT  CBHNDL, %ID_MESSAGE, s
    
                     EXIT FUNCTION
    
                 CASE %ID_PROCESS_2
                     ' Send file to the callback procedure which will terminate the enumeration as soon a Target String is found.
    
                     ' Get the string on which we terminate
                     StopWhenFound  = INPUTBOX$ ("Search file for text string ", "MMF Line Input")
                     IF LEN (StopWhenFound) THEN
                         MMFLI.hDlg     =  CBHNDL
                         MMFLI.nRec     =  0
                         MMFLI.pdws     =  VARPTR (StopWhenFound)   ' pass string pointer
                         MMFLI.nRec     =  0
                         dwUser         =  VARPTR (mmfli)           ' pass address of our parameter block
                         dwAddr         =  CODEPTR (Process_2_Function)
                         ' get the file name into an ASCIIZ
                         CONTROL GET TEXT CBHNDL, %ID_FILENAME TO s
                         szFile     = s
    
                         ' call the data enumerator
                         lRet       =  mmf_li_cb (szFile, dwAddr, dwUser)
                         ' show the completion status in the text message area
                         s          =       "Process #2: Scan records for text string '" & StopWhenFound & "'" & $CRLF
                         s          =   s & "Return code  " & STR$(lRet)  & $CRLF
                         IF lret THEN      ' enumeration was terminated because our string was found
                             s          =   s &  "string found at offset" & STR$(mmfli.nchar)
                         ELSE
                             s          =   s & USING$("String not found in #,###,### Bytes and #,###,### records", mmfli.nchar, mmfli.nrec)
                        END IF
                     ELSE
                         s = "Can't very well search for nothing, can I?"
                         MessageBeep   %MB_ICONHAND
                     END IF
    
                     CONTROL      SET TEXT  CBHNDL, %ID_MESSAGE, s
    
                     EXIT FUNCTION
    
                 CASE %ID_PROCESS_3
                     MSGBOX "Come on. Show a little imagination: Write a callback yourself!", %MB_ICONEXCLAMATION OR %MB_TASKMODAL, "MMF LINE INPUT"
    
    
                 CASE %ID_QUIT
                     DIALOG END CBHNDL, 0
    
                 CASE %ID_FILENAME_BROWSE
                    Flags      = %NULL
                    IF ISTRUE OpenFileDialog (CBHNDL, "Select file to process", s, "","*.*", "", Flags) THEN
                         ' put the file name into the filename control
                         CONTROL SET TEXT  CBHNDL, %ID_FILENAME, s
                    END IF
    
    
              END SELECT
    
        END SELECT
    
    END FUNCTION
    
    ' =========================================='
    '          PROGRAM ENTRY POINT
    ' =========================================='
    
    
    FUNCTION PBMAIN () AS LONG
    
      LOCAL  szFile AS ASCIIZ * %MAX_PATH  ' the file we are mapping
      LOCAL  hDlg AS LONG, lresult AS LONG
    
      ' create a dialog, with space to show filename, action, filesize,
    
        DIALOG NEW  0, "Memory-Mapped File Replacement for LINE INPUT", 0, 0, 300,180, _
                %DS_CENTER OR %WS_CAPTION OR %WS_SYSMENU, 0 TO hDlg
    
    
        ' Abort if the dialog could not be created
         IF hDlg = 0 THEN
              EXIT FUNCTION
         END IF
    
         ' filename label and textbox
         CONTROL ADD LABEL   , hDlg, -1&, "File",  4 ,4, 40,14
         CONTROL ADD TEXTBOX , hDlg, %ID_FILENAME, "", 50,4, 200, 14
    
         ' browse button for file name
         CONTROL ADD BUTTON  , hDlg, %ID_FILENAME_BROWSE, "Browse", 256, 4, 40,14
    
         ' buttons to process the current file
         CONTROL ADD BUTTON  , hDlg, %ID_PROCESS_1, "Record Counter"   ,  20, 24, 60, 14
         CONTROL ADD BUTTON  , hDlg, %ID_PROCESS_2, "Find Text in file",  90, 24, 60, 14
         CONTROL ADD BUTTON  , hDlg, %ID_PROCESS_3, "Surprise!"        , 160, 24, 60, 14
    
         ' text message area to show progress
         CONTROL ADD LABEL   , hDlg, %ID_MESSAGE, "We haven't done anything yet", 4, 40, 292, 60
    
         ' quit button
         CONTROL ADD BUTTON  , hDlg, %ID_QUIT, "Quit",  120, 110, 40,14
    
         ' shameless
         CONTROL ADD LABEL   , hDlg, %ID_SHAMELESS, "Demo Courtesy Michael Mattias Racine WI", 4, 162, 292, 12, %Ws_VISIBLE OR %WS_CHILD OR %SS_CENTER
    
    
         ' just for testing
         ' CONTROL Set Text hDlg, %ID_FILENAME, "S:\pbwin70\winapi\win32api.inc"
         ' you can search for "updatewindow" which is near the end of the file
    
        ' show the dialog:
        DIALOG SHOW MODAL hDlg, CALL DlgCallback TO lResult
    
        ' The return value of success! (Demos always succeed).
        FUNCTION = 0
    
    END FUNCTION
    
    ' ********* END OF FILE MMFLIDEMO.BAS *******

    ------------------
    Michael Mattias
    Tal Systems Inc.
    Racine WI USA
    mailto:[email protected][email protected]</A>
    www.talsystems.com
    Michael Mattias
    Tal Systems (retired)
    Port Washington WI USA
    [email protected]
    http://www.talsystems.com

    Comment


    • #3
      Updated version: adds mapping of part of file as well as I-O capability.

      Code:
      ' TSI_MapFile.INC
      ' created 4-08-04 for use as #INCLUDE file
      ' Usage:
      ' #INCLUDE "TSI_MAPFILE.INC"
      '    LOCAL pIFData AS DWORD, hSys AS LONG, hFileMapping AS LONG, cbMapping AS LONG, IO AS LONG
      '    pIFData = MapThisFile (szFile, hSys, hFileMapping, cbMapping, IO)  ' IO: 0= input only, TRUE=I-O
      '    piFdata = pointer to start of data
      '    cbmapping is returned as file size (number of bytes to do)
      '    save hSys, hFileMapping
      '    DO REAL WORK HERE
      '    UnmapMappedFIle  pIFData, hSys, hFileMapping
      ' ---------------------------------------------------
      ' Added 7/30/05 ability to map only part of a file using MapPartOfFile function
      ' ----------------------------------------------------
      
      ' DECLARES
      DECLARE FUNCTION MapThisFile (szFileIn AS ASCIIZ, hSys AS LONG, hFileMapping AS LONG, cbMapping AS LONG, BYVAL IO AS LONG) AS DWORD
       ' piFDATA, hSys ANd hFilemapping are what is returned by the MapThisFile Function
      DECLARE FUNCTION UnmapMappedFile (pIfData AS DWORD, hSys AS LONG, hFileMapping AS LONG) AS LONG
      ' usage: MapThis file, save pifdata hSys, Filemapping for call to Unmap mapped file
      
      DECLARE FUNCTION MapPartOfFile (szFileIn AS ASCIIZ, BYVAL iMapOffset AS LONG, BYVAL iMapLen AS LONG, _
                         hSys AS LONG, hFileMapping AS LONG, cbMapping AS LONG, BYVAL IO AS LONG) AS DWORD
      ' usage: MappartOfFile, save pifData (return value), hSys, hFilemapping for call to UnmapPartOfFile
      DECLARE FUNCTION UnmapPartOfFile (pIfData AS DWORD, hSys AS LONG, hFileMapping AS LONG) AS LONG
      
      ' ===============================================================================================
      '                          FILE MAPPING ENCAPSULATION FUNCTIONS
      ' ===============================================================================================
      ' -----------------------------------------------------------------------------
      '  MEMORY-MAPPED FILE FUNCTIONS ADDED VERSION 1.3.0 (from personal library)
      '  LOCAL pIFdata AS DWORD, hSys AS LONG, hFileMapping AS LONG, cbmapping AS LONG, szFile AS ASCIIZ * %MAX_PATH
      ' ---------------------------------------------------------------------------------------------------
      ' returns: TRUE = success, pointer to file data in piFDATA, FALSE=Error
      ' BYVAL IO : If true, file is mapped for both read and write; else is mapped for READONLY
      FUNCTION MapThisFile (szFileIn AS ASCIIZ, hSys AS LONG, hFileMapping AS LONG, cbMapping AS LONG, BYVAL IO AS LONG) AS DWORD
      
        LOCAL w32 AS WIN32_FIND_DATA, hSearch AS LONG
        LOCAL pIFdata AS DWORD
        LOCAL dwShareMode AS DWORD, dwDesiredAccess AS DWORD, _  ' createfile flags
              dwCreationDisposition AS DWORD
        LOCAL fProtect    AS DWORD                               ' createFileMapping flasd
        LOCAL dwmvDesiredAccess AS DWORD                         ' mapviewoffileflags
      
        hSearch      = FindFirstFile (szFileIn, W32)
        IF hSearch <> -1& THEN
            cbMapping =  w32.nFileSizeLow
            CloseHandle hSearch
        ELSE
            FUNCTION = 0
            EXIT FUNCTION
        END IF
      
        IF ISTRUE IO THEN
            dwShareMode             =  %NULL     ' exclusive use
            dwDesiredAccess         =  %GENERIC_READ OR %GENERIC_WRITE
            dwCreationDisposition   =  %OPEN_ALWAYS
            fProtect                =  %PAGE_READWRITE
            dwMvDesiredAccess       =  %FILE_MAP_WRITE          ' creates read-write access
      
        ELSE
            dwShareMode             = %FILE_SHARE_READ  ' ms recommends exclusive use for mapping, I choose this
            dwDesiredAccess         = %GENERIC_READ
            dwCreationDisposition   = %OPEN_EXISTING
            fProtect                = %PAGE_READONLY
            dwMvDesiredAccess       = %FILE_MAP_READ
      
        END IF
      
        ' Open the file for reading and get a system handle
        hSys          = CreateFile (szFileIn, dwDesiredAccess, dwShareMode, BYVAL %NULL, DwCreationDisposition, %FILE_ATTRIBUTE_NORMAL, BYVAL %NULL)
        IF ISTRUE hSys  THEN
           hFileMapping     = CreateFileMapping (hSys, BYVAL %NULL, fProtect, BYVAL %NULL, BYVAL %NULL, BYVAL %NULL)
           IF ISTRUE hFileMapping THEN
               ' map the file to our memory space, returning the pointer..
               ' NOTE WE NEED THE VALUE of pIFData TO UNMAP THE FILE
      
               pIFData    = MapViewOfFile (hFileMapping, dwMvDesiredAccess, BYVAL %NULL, BYVAL %NULL, BYVAL %NULL)
               FUNCTION   = pIFDATA
           ELSE
               FUNCTION    = 0
               Closehandle hSys
               EXIT FUNCTION
           END IF
        ELSE
            FUNCTION = 0
            EXIT FUNCTION
        END IF
      
      END FUNCTION
      
      FUNCTION UnmapMappedFile (pIfData AS DWORD, hSys AS LONG, hFileMapping AS LONG) AS LONG
          UnmapViewOfFile pIFData
          CloseHandle     hFileMapping
          CloseHandle     hSys
      END FUNCTION
      
      ' ===========================================================================
      '        FUNCTIONS TO MAP AND UNMAP PART OF A FILE ADDED JULY 30 2005
      ' ===========================================================================
      '=======================================================================
      ' == FILE MAP PART OF A FILE==
      '  szFileIn     [in]  File to be mapped
      '  iMapOffset   [in]  Offset in szFileIn at whci to begin mapping
      '  iMapLen      [in]  Length to map. Error (No mapping created) if offset + len > filesize
      '  hSys         [out] Control value needed to unmap the file
      '  hFilemapping [out] Control value needed to unmap the file
      '  cbMapping    [out] Size (number of bytes) of mapped file
      '  Returns:     True ==> Pointer to data
      
      ' Edits we do:
      ' Filesize < 2 GB,
      ' offset+Maplen > filesize
      
      ' Must unmap using function UnmapPartOfFile, passing returned value from this function, hSys and hfilemapping
      ' also returned by this function
      '=======================================================================
      FUNCTION MapPartOfFile (szFileIn AS ASCIIZ, BYVAL iMapOffset AS LONG, BYVAL iMapLen AS LONG, _
                            hSys AS LONG, hFileMapping AS LONG, cbMapping AS LONG, BYVAL IO AS LONG) AS DWORD
      
        LOCAL w32 AS WIN32_FIND_DATA, hSearch AS LONG
        LOCAL pIFdata AS DWORD
        LOCAL dwShareMode AS DWORD, dwDesiredAccess AS DWORD, _  ' createfile flags
              dwCreationDisposition AS DWORD
        LOCAL fProtect    AS DWORD                               ' createFileMapping flasd
        LOCAL dwmvDesiredAccess AS DWORD                         ' mapviewoffileflags
        ' to support partial file mapping
        LOCAL iGranularity AS LONG
        LOCAL iViewOffset  AS LONG, iViewLen AS LONG
        LOCAL SysInfo AS SYSTEM_INFO, ESys AS LONG
      
        hSearch      = FindFirstFile (szFileIn, W32)
        IF hSearch <> -1& THEN
            cbMapping =  w32.nFileSizeLow
            CloseHandle hSearch
            IF ISTRUE w32.nFilesizeHigh THEN   ' file is greater than 2 Gb
                FUNCTION = 0
                'STDOUT "File > 2 Gb"          ' unable to test on my Win/98 O/S
                EXIT FUNCTION
            ELSEIF cbMapping < iMapoffset + iMapLen THEN
                FUNCTION = 0
                'STDOUT "offset + len > filesize"  ' << WORKING OK 7/30/05
                EXIT FUNCTION
            END IF
        ELSE
            FUNCTION = 0
            EXIT FUNCTION
        END IF
      
        IF ISTRUE IO THEN
            dwShareMode             =  %NULL     ' exclusive use
            dwDesiredAccess         =  %GENERIC_READ OR %GENERIC_WRITE
            dwCreationDisposition   =  %OPEN_ALWAYS
            fProtect                =  %PAGE_READWRITE
            dwMvDesiredAccess       =  %FILE_MAP_WRITE          ' creates read-write access
      
        ELSE
            dwShareMode             = %FILE_SHARE_READ  ' ms recommends exclusive use for mapping,
                                                        ' I choose this when IO=False
            dwDesiredAccess         = %GENERIC_READ
            dwCreationDisposition   = %OPEN_EXISTING
            fProtect                = %PAGE_READONLY
            dwMvDesiredAccess       = %FILE_MAP_READ
      
        END IF
      
        ' Open the file for reading or read/write and get a system handle
        hSys          = CreateFile (szFileIn, dwDesiredAccess, dwShareMode, BYVAL %NULL, DwCreationDisposition, %FILE_ATTRIBUTE_NORMAL, BYVAL %NULL)
        IF ISTRUE hSys  THEN
           hFileMapping     = CreateFileMapping (hSys, BYVAL %NULL, fProtect, BYVAL %NULL, BYVAL %NULL, BYVAL %NULL)
           ' we are creating a mapping object the size of the file so we can get at
           ' any portion of it with our view; and also in case other processes map the file we
           ' need to keep our view coherent regardless of activity in that other process.
           IF ISTRUE hFileMapping THEN
               ' map the file to our memory space, returning the pointer..
               ' NOTE WE NEED THE VALUE of pIFData TO UNMAP THE FILE
               ' to map parti
               ' get the granulariity of page allocation on this system
               GetSystemInfo   SysInfo    ' dwPage
               iGranularity   = Sysinfo.dwAllocationGranularity
              'STDOUT "Granularity IS " & STR$(iGranularity) & "  hex " & HEX$(iGranularity)
              ' on my system it's 65536  7/30/05
              ' since we must map at a page boundary, find next lowest page boundary..
               iViewOffset          = (iMapOffset \ iGranularity) * iGranularity
              ' map the length the user wanted, plus the part BEFORE the offset we need to
              ' map because the mapped view must start on an allocation boundary:
               iViewLen            = iMapLen         +  (iMapOffset MOD iGranularity)
               '                     whatcaller wanted   Extra we needed to map ahead of file
               ' here is would be good to show: Passed offset, len, mapped offset, len
              ' STDOUT USING$("passed offset #, Len #,",iMapOffset, iMapLen)
      
              ' STDOUT "viewOffset=" & STR$(iViewOffset) & " hex " & HEX$(iViewOffset)
              ' STDOUT "viewLen   =" & STR$(iViewLen)    & " hex " & HEX$(iViewLen)
              ' get a pointer to the start of the view, which is the page boundary:
               pIFData    = MapViewOfFile (hFileMapping, dwMvDesiredAccess, BYVAL %NULL, BYVAL iViewOffset, BYVAL iViewLen)
      
               IF ISFALSE pIFDATA THEN
                   ESys   = GetLastError
                  ' STDOUT  "MapView of File failed"
                  ' STDOUT      SystemErrorMessageText (ESys)
                   CloseHandle  hFileMapping    ' close the underlying handles
                   CloseHandle  hSys
                   FUNCTION =  0
                   EXIT FUNCTION
               ELSE
                  ' above this pointer will be to the start of the page, to allow for the extra we mapped:
                    'show RAW PifData, which is the value we will need to unmap the view
                   ' STDOUT USING$("Raw piFDATA #,", pifData)
                    piFData    = piFData  + (iMapOffset MOD iGranularity)
               END IF
               FUNCTION   = pIFDATA
           ELSE                     ' could not create file mapping object
               FUNCTION    = 0
               Closehandle hSys     ' close underlying file handle
               EXIT FUNCTION
           END IF
        ELSE
            FUNCTION = 0
            EXIT FUNCTION
        END IF
      
      END FUNCTION ' mapPartOfFile
      
      FUNCTION UnmapPartOfFile (pIfData AS DWORD, hSys AS LONG, hFileMapping AS LONG) AS LONG
      ' pifdata is original pointer to data, which is offset when necessary
        LOCAL SysInfo AS SYSTEM_INFO
        LOCAL iViewStart AS DWORD
        LOCAL iStat AS LONG, ESys AS LONG
        LOCAL offsetFrompage AS DWORD
      
          ESys          = %NULL
          GetSystemInfo  SysInfo
         ' the 'real' pointer to the mapped view is the first page boundary preceding pifData
         ' because we mapped to the next lower page and added the (offset mod granularity.)
         ' the 'odd' piece, which must be the offset from the mapped view start must be:
          OffsetFromPage = pifdata MOD Sysinfo.dwAllocationGranularity
          'STDOUT USING$("UNMAPPING GRANULARITY #,", Sysinfo.dwAllocationGranularity)
          'STDOUT USING$("UNMAPPING pifIn #,",  pifData)
          'STDOUT USING$("UNMAPPING OFFSET FROM PAGE #," ,OffsetFromPage)
          ' so the view start is at:
          iViewStart =  pifData - offsetFromPage
          ' STDOUT USING$ ("UNMAPPING  offset #, pifData  #,   iViewStart #,", offsetfrompage, pifData, iViewStart)
          ' Unmap it...
          iStat         =  UnmapViewOfFile(iViewStart)  ' returns true on success
          eSys          = GetLasterror
          IF ISFALSE IStat THEN
            ' STDOUT "Unmap view of file failed " & SystemErrorMessageText (eSys)
          'ELSE
            ' STDOUT "Unmap View of file returns " & FORMAT$(iStat) & " (TRUE=SUCCESS)"
          END IF
          CloseHandle     hFileMapping
          CloseHandle     hSys
      
          FUNCTION = ISFALSE(iStat)   ' return Zero on success.
      END FUNCTION
      
      ' *** END OF FILE TSI_MAPFILE.INC ****
      MCM
      Michael Mattias
      Tal Systems (retired)
      Port Washington WI USA
      [email protected]
      http://www.talsystems.com

      Comment


      • #4
        I've been distracted for the last week on another app, so I've not thought as much about the LTF viewer issues. Another day or two and I'll be able to pay more attention to this thread again and will want to compare the approaches in my own gbLTFViewer to the other suggestions here, especially now that code is beginning to be posted. With code in hand, speed testing/comparison becomes possible. What fun!

        Comment

        Working...
        X