Announcement

Collapse
No announcement yet.

#Resource RcData Size Limit?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #Resource RcData Size Limit?

    I was trying to put a 785MB file into an EXE as a resource, but got a "Resource File Error 478". Rebooting didn't seem to help.

    I was able to put a 441MB file in the EXE with no problem.

    Anyone aware of a size limitation on the #Resource statement?

    I've seen that error before, usually when the folder containing the resource was open. So I don't think the 785MB is an issue, but thought I'd ask just in case.

  • #2
    Hey Gary,

    You should be able to get at least 2gb,
    even more depending on the partition type and OS,
    a compiler could impose some limit.

    This one use a 778,371,072 bytes file with no problem.
    Could you describe how you did it?

    Code:
    ' MyBigRes.rc...
    ' #include "resource.h"
    ' 123 RCDATA  "Some742gbFile.iso" //778,371,072 bytes file
    
    #COMPILE EXE '#CC 5.07#
    #DIM ALL
    #INCLUDE "Win32Api.inc"
    #RESOURCE "MyBigRes.pbr"
    '_____________________________________________________________________________
    
    FUNCTION PBMAIN() AS LONG
     LOCAL hInst           AS DWORD
     LOCAL hRes            AS DWORD
     LOCAL hData           AS DWORD
     LOCAL pData           AS DWORD
     LOCAL sData           AS STRING
    
     hInst = GetModuleHandle("")
     hRes  = FindResource(hInst, "#123", BYVAL %RT_RCDATA)
     IF hRes THEN
       hData    = LoadResource(hInst, hRes)
       IF hData THEN
         pData = LockResource(hData)
         IF pData THEN
           STDOUT FORMAT$(SizeofResource(hInst, hRes), "0,")
         END IF
       END IF
     END IF
    
     PRINT : PRINT "Type or click..." : MOUSE ON : MOUSE 3, UP : WAITKEY$
    
    END FUNCTION
    '_____________________________________________________________________________
    ' 

    Comment


    • #3
      [QUOTE=Pierre Bellisle;n771044]Hey Gary,

      You should be able to get at least 2gb,
      even more depending on the partition type and OS,
      a compiler could impose some limit.

      This one use a 778,371,072 bytes file with no problem.


      Your code apparently uses an external resource compiler to geberate your .pbr file.

      I suspect Gary is using
      #RESOURCE RCDATA ........

      I just built a 1,000,000 byte executable using #BLOAT ( just displays a Msgbox)

      Then complied another file with

      #RESOURCE RCDATA BigExe "BigExe.exe"

      It worked fine.


      Edit: Oops, that was only 1,000,000 bytes. Let me try again!

      Comment


      • #4
        Tried twice with a 1GB file and gave up after PB sat using 25% CPU resources for over half an hour each time.

        No error though.

        Comment


        • #5
          was trying to put a 785MB file into an EXE as a resource,
          Why? Do you really need to do that? (I don't know but I just want to be sure you have thought about it).

          Ok, maybe helpful...

          1. Split the resource into, say, three or more chunks and assemble at runtime.

          2. Make the resource either a RES or PBR file; do NOT use the inline resource definition statement, instead you will compile in the *.RES or *.PBR file. Note this is the way Pierre got this to work in Post #2 (Note also: Under PB now all resources have to be in the same format (RES or PBR), but we all know how to do that anyway).

          3. Use a third-party tool to compile a *.RES and then use a third-party linker to link that *.RES into your EXE.

          MCM
          Michael Mattias
          Tal Systems Inc. (retired)
          Racine WI USA
          [email protected]
          http://www.talsystems.com

          Comment


          • #6
            Hi Pierre,

            I did this, where bigfile.zip is 785MB ...

            Code:
            #Resource RcData, 1000, "bigfile.zip"
            And, Stuart,
            Yes, I finally got the compiler to start cranking ... but it chugged along for quite a while before stopping/crashing with no error message.

            And, MCM,
            I've no problem going to another approach, but would like to know why #Resource approach doesn't work and perhaps know what the actual limitation is.

            Comment


            • #7
              Gary,

              You are using Windows 10, try to compile under Windows 7 if possible, if it works then it is a PowerBASIC / Windows 10 issue.

              I'm sure that if you use an external .rc file under any OS, it will work all-wight and the ~2gb limit will prevail .

              Comment


              • #8
                Originally posted by Pierre Bellisle View Post
                Gary,

                You are using Windows 10, try to compile under Windows 7 if possible, if it works then it is a PowerBASIC / Windows 10 issue.

                I'm sure that if you use an external .rc file under any OS, it will work all-wight and the ~2gb limit will prevail .
                My test which I abandoned after 1/2 an hour or so, was under Win 7.

                Comment


                • #9
                  I tested it with a 1.5GB iso file. PBWin 10.03 reads the entire file into memory in ~2MB chunks, and then exits with

                  Error 478 in c:\Test\PB\test.bas(4:001): Resource file error
                  Line 4: #RESOURCE RcData, 123, "H:\os\6001.18000.080118-1840_x86fre_Server_en-
                  us-KRMSFRE_EN_DVD.iso"

                  It didn't take half an hour for me though, took 90 seconds. A 550MB file actually produced an exe, but required 1.2GB of memory, in 42 seconds.

                  Since it reads the specified file completely into memory, PBWin would seem to need to be able to allocate 2x the size of the file + have space for all the Windows dlls and whatever else in the 2GB of address space as (my version at least) isn't LargeAddressAware.

                  I'm obviously guessing, but there's probably no hard maximum file size where the compiler outright rejects your rcdata files, but implementation wise, it seems to roughly be around 900MB-ish depending on how much code is generated, what other resources there are, etc.

                  Comment


                  • #10
                    Hmm.... it's been a while since I created a PBR file. So, I loaded this one-line .RC file and compiled it (BigFile.zip is 785MB) ...

                    Code:
                    123  RCDATA  "BigFile.zip"
                    and, with the compiler set to "Create a .PBR file when compiling a .RC files", the compiler created a 785MB RES file, but a PBR file of only 100 bytes. I don't think that's how it's supposed to work!

                    From Help ...

                    When the "Create a .PBR when compiling .RC files" option on the compiler options tab is selected the IDE will also produce a .PBR file from the .RES file. This is achieved in one simple step: simply load the .RC file into the IDE and select Compile.

                    Comment


                    • #11
                      But, Help does say that #Resource supports this syntax ...

                      Code:
                      #RESOURCE RES, "filespec.RES"
                      But when I use that syntax with the 785MB RES file that resulted from my compilation as mentioned above, the result is a 77KB EXE file.




                      Comment


                      • #12
                        Gary, could you post all the code from the offending program before any functions or procedures?
                        Rod
                        I want not 'not', not Knot, not Knott, not Nott, not knot, not naught, not nought, but aught.

                        Comment


                        • #13
                          Hi Rod,
                          This is some test code I used. Runs OK in Windows 7 but I get the same failed .exe when compiled in WIndows 10 as Gary sees..!
                          (Though my failed .exe is only 13K)

                          Code:
                          #Compile Exe
                          #Dim All
                          #Include "win32api.inc"
                          #Resource Res, "RCDataZip.res"
                          
                          Function MakeFileFromRes(sPath As String, ResId As String) As Long
                           Local datasize, filehandle As Long
                           Local buffer As String
                          
                              buffer   = Resource$(RcData, ResID)
                              datasize = Len(buffer)
                              ? ResID,,"ResID"+Str$(datasize)
                          
                             If datasize > 0 Then
                               winbeep 800, 200
                                filehandle = FreeFile
                                Open spath For  Binary  As filehandle
                                  Put$ filehandle, buffer
                                Close filehandle
                                Function = %TRUE
                             Else
                                Function = %false
                             End If
                          End Function
                          '------------------/MakeFileFromRes
                          
                          Function PBMain () As Long
                           Local lRes As Long
                          
                            lRes = MakeFileFromRes("RCDataExtracted.zip", "RESID")
                            ? "Done",,Str$(lRes)
                          
                          End Function
                          '------------------/PBMain
                          
                          #If 0
                          //rc file contents - saved As RCDataZip.rc
                          
                          ResId RcData "ResData.zip"     // 835+MB archive. Compiled To .\RCDataZip.res
                          #EndIf
                          explorer listing
                          Rgds, Dave

                          Comment


                          • #14
                            Hey Gary,
                            What if you try manually?

                            REM Create a .res
                            X:\Basic\PB-Compiler\bin\rc.exe "X:\Basic\Bas\SRC\MyFile.rc"

                            REM Create a .pbr
                            X:\Basic\PB-Compiler\bin\pbRes.exe "X:\Basic\Bas\SRC\MyFile.res"

                            REM In the bas file have
                            #RESOURCE "MyFile.pbr"

                            REM Create a .exe
                            X:\Basic\PB-Compiler\bin\pbWin.exe "X:\Basic\Bas\SRC\MyFile.bas" /L /I"X:\Basic\PB-Compiler\\WinApi"

                            Comment


                            • #15
                              Dave, I don't have a Win 10 system but I couldn't get your code to run on Win 7 with my 435,000+kb file. I did change #RESOURCE RES to #RESOURCE RCDATA, 1000, filename and changed "lRes = MakeFileFromRes("RCDataExtracted.zip", "RESID")" to "lRes = MakeFileFromRes("RCDataExtracted.zip", "#1000")" and it ran like a charm with the #IF block commented out. I get the Error 478 resource file error with your code on my machine.
                              Rod
                              I want not 'not', not Knot, not Knott, not Nott, not knot, not naught, not nought, but aught.

                              Comment


                              • #16
                                Using the method above, I did a manual compile under Windows 10.
                                pbRes failed to create a valid .pbr from a valid 760mb .res file. The result was a 100 bytes file.
                                Same procedure worked fine under Windows 7 though. [Added: Windows 8.1 worked fine also.]
                                Last edited by Pierre Bellisle; 23 Mar 2018, 09:12 PM.

                                Comment


                                • #17
                                  Hi Rod,
                                  You need to create a .RC file using the line inside the #IF block. Then compile it with RC.EXE to generate the .RES file.

                                  (Otherwise..)
                                  Error 478 in C:\PBWin10\Samples\RCData\RCDataExe.bas(4:001): Resource file error
                                  Line 4: #Resource Res, "RCDataZip.res"
                                  Rgds, Dave

                                  Comment


                                  • #18
                                    Right, Dave. I've been spoiled, never gave thought to the multi step process.All is good here.
                                    Rod
                                    I want not 'not', not Knot, not Knott, not Nott, not knot, not naught, not nought, but aught.

                                    Comment


                                    • #19

                                      Did some more testing - using the #Resource RCDATA, RESID, "ResData.zip" method - no pre compiling to RES / external resource file.
                                      In Windows 10 things go awry on my PC when the .zip file exceeds 660MB or so.

                                      Above the limit, a small .exe file (~13k) is written to disk but PBWin.exe sits in memory eating up 25% of my CPU until killed with Task Manager's 'End Task'.

                                      Code:
                                      #Compile Exe
                                      #Dim All
                                      #Include "win32api.inc"
                                      '#Resource Res, "RCDataZip.res"
                                      #RESOURCE RCDATA, RESID, "ResData.zip"
                                      
                                      Function MakeFileFromRes(sPath As String, ResId As String) As Long
                                       Local datasize, filehandle As Long
                                       Local buffer As String
                                      
                                          buffer   = Resource$(RcData, ResID)
                                          datasize = Len(buffer)
                                          ? ResID,,"ResID"+Str$(datasize)
                                      
                                         If datasize > 0 Then
                                           winbeep 800, 200
                                            filehandle = FreeFile
                                            Open spath For  Binary  As filehandle
                                              Put$ filehandle, buffer
                                            Close filehandle
                                            Function = %TRUE
                                         Else
                                            Function = %false
                                         End If
                                      End Function
                                      '------------------/MakeFileFromRes
                                      
                                      Function PBMain () As Long
                                       Local lRes As Long
                                      
                                        lRes = MakeFileFromRes("RCDataExtracted.zip", "RESID")
                                        ? "Done",,Str$(lRes)
                                      
                                      End Function
                                      '------------------/PBMain
                                      Rgds, Dave

                                      Comment


                                      • #20
                                        Originally posted by Dave Biggs View Post

                                        Did some more testing - using the #Resource RCDATA, RESID, "ResData.zip" method - no pre compiling to RES / external resource file.
                                        In Windows 10 things go awry on my PC when the .zip file exceeds 660MB or so.

                                        Above the limit, a small .exe file (~13k) is written to disk but PBWin.exe sits in memory eating up 25% of my CPU until killed with Task Manager's 'End Task'.
                                        That's what I found on Win 7 too.

                                        Comment

                                        Working...
                                        X