Append binary file to EXE file

[Steve Hutchesson]

A while ago, there was a thread about using large binary resources within
an EXE file and through testing done by Semen, it was determined that at
least some versions of 32 bit windows do not function correctly with the
resource load#### API functions once the resources are over a certain
size.

Below is an alternative approach that is free of the limitations imposed
by the operating system in this instance. There are two (2) seperate
pieces of code here, one is a complete utility writen for PBCC version 2
and the other is a function that is used in the EXE that has a file
appended to the end of it.

Here is how it works, the utility takes 2 command line parameters, the
first is the name of the EXE file that is to have another file appended to
the end of it, the second is the name of the file that is appended to it.

The utility reads both files into memory and gets their length, writes
both lengths into an unused part of the DOS header at the start of the EXE
file in memory and then deletes the original EXE file on disk and writes
the new one with the same name and the appended file to disk as one disk
file. The utility doe not modify the PE header or image at all.

To get the appended file from within the EXE file that has the file
appended to it, the function below reads the two (2) DWORD values from the
DOS header where the utility wrote them and loads the appended file into a
normal basic string ready to write to disk.

This process has been tested with files over 12 meg in size and it does
not have any problems and all of the normal resource load#### function
work normally. It has also been tested with AVP to ensure that the
appended file does not trigger the heuristic virus alert in the smarter
end of anti-viral software.

One limitation imposed by this technique is that the file should not
be modified with either a compressor or encryption after it has been made.
If you need to compress or encrypt the EXE file, do it before it has the
append operation done to it. Otherwise the function will give the wrong
values for the appended file length.

Regards,

[email protected]

  --------------------------------------------------------------------------
  
  The function to use in the EXE file to get the appended file as a basic
  string.
  
  ' #########################################################################
  
  ' To use with APPENDIT.EXE. This function reads the two (2) DWORD
  ' values written at 30h in the DOS header of the running EXE file
  ' and then reads the appended file into the return string.
  
  ' #########################################################################
  
  FUNCTION GetAppendFile() as String
  
  '   DECLARE FUNCTION GetAppendFile() as String
  
      #REGISTER NONE
  
      LOCAL fLen1    as LONG
      LOCAL fLen2    as LONG
      LOCAL dump     as LONG
      LOCAL hFile    as LONG
      LOCAL src      as LONG
      LOCAL fs       as LONG
  
      LOCAL szbuffer as ASCIIZ * 128
  
      GetModuleFileName ByVal %NULL,szbuffer,128
  
      hFile = CreateFile(ByVal VarPtr(szbuffer),%GENERIC_READ, _
                         %FILE_SHARE_READ,ByVal %NULL,%OPEN_EXISTING, _
                         %FILE_ATTRIBUTE_NORMAL,ByVal %NULL)
  
      fs = GetFileSize(hFile,%NULL)
  
      Buffer$ = space$(fs)
  
      rv& = ReadFile(hFile,ByVal StrPtr(Buffer$),fs, _
                     ByVal VarPtr(dump),ByVal %NULL)
  
      src = StrPtr(Buffer$)
  
      ! mov esi, src
      ! add esi, &H30     ; offset in DOS header
      ! lodsd
      ! mov fLen1, eax
      ! lodsd
      ! mov fLen2, eax
  
      CloseHandle hFile
  
      FUNCTION = right$(Buffer$,fLen2)
  
  END FUNCTION
  
  '##########################################################################
  
  
  
  The PBCC 2 application that appends the file and writes to the DOS header.

  
  
  
  ' #########################################################################
  
      DECLARE FUNCTION Exist&(fname$)
  
  ' #########################################################################
  
  FUNCTION PbMain()
  
      #REGISTER NONE
  
      If command$ = "" Then
        StdOut "No command line supplied"
        Exit Function
      End If
  
      cmd$ = command$
      spp& = instr(cmd$," ")  ' space position
  
      exename$ = trim$(left$(cmd$,spp& - 1))
  
      If Exist&(exename$) = 0 Then
        StdOut "Cannot find > " + exename$
        Exit Function
      End If
  
      addfile$ = trim$(right$(cmd$,len(cmd$)-spp&))
  
      If Exist&(addfile$) = 0 Then
        StdOut "Cannot find > " + addfile$
        Exit Function
      End If
  
      Open exename$ for Binary as #1
        ln1& = lof(1)
        Get$ #1, ln1&, a$
      Close #1
  
      Open addfile$ for Binary as #1
        ln2& = lof(1)
        Get$ #1, ln2&, b$
      Close #1
  
      kill exename$
  
      c$ = a$ + b$
  
      src& = StrPtr(c$)
  
      ! mov edi, src&
      ! add edi, &H30
      ! mov eax, ln1&     ; write 1st length at 30h
      ! stosd
      ! mov eax, ln2&     ; write 2nd length after it
      ! stosd
  
      Open exename$ for OutPut as #1
        Print #1, c$;
      Close #1
  
      msg1$ = "exe file length    = "+str$(ln1&)+" bytes"
      msg2$ = "append file length = "+str$(ln2&)+" bytes"
      msg3$ = exename$ + " written at"+str$(ln1& + ln2&)+" bytes"
  
      StdOut msg1$
      StdOut msg2$
      StdOut msg3$
  
      FUNCTION = 0
  
  END FUNCTION
  
  ' #########################################################################
  
  FUNCTION Exist&(fname$)
    FUNCTION = Len( Dir$(fname$, 17) ) > 0
  END FUNCTION
  
  ' #########################################################################

[E B Knoppert]

Steve, of course but i did that for about 2 years now.
I did use VB3 and some alg. to calculate the multiple appended files, if you are one byte misaligned, you could start all over.
(Very time consuming for our updates)

Finally we used PB and had the opportunity to use resources.
Now we suffer from a problem... (somewhere)
To find a sollution to my cursor and icon problems, i tried the createiconfromresource API, unsuccesfull yet..

until now we have files (compressed) wich are not larger than 1Mb (lucky)

Thanks,


------------------

[Steve Hutchesson]

Edwin,

I actually needed this code for an installation I have to do shortly and
the approach I am taking with it is to use a winrar EXE archive which will
be written to disk and run to expand up the directories and files.

To do a number of seperate files is a bit more messing around to get the
correct starting offsets but the basic string instructions will handle
this reasonably easily once you have the appended file.

Regards,

[email protected]

------------------

[jcfuller]

Steve,
Would you shed some light on how windows loads an exe?
I believe I read somewhere about loading a packed exe defeats the practice of just loading the exe when resources are part of the exe. When compressed the whole exe is loaded into, and then expanded in memory, taking up a much larger memory foot print than a non-compressed (packed) file would.

James


------------------

[Steve Hutchesson]

James,

While trying to produce a description of how an operating system loader
works is a task that I would not like to take on without a 6 month lead
time, generally a non-modified PE exe pages required parts on demand.

When an EXE file is compressed, the page on demand capacity is defeated and
the whole image is loaded into memory. This used to be a problem in 16 bit
Windows as memory was usually a lot more limited than todays machines where
you can generally assume that the machine has at least 16 meg of memory.

Depending on the size of the original EXE file, this can be a problem when
multiple instances are run. A characteristic of 32 bit PE files is that the
disk image is generally larger than the memory image which is the reverse
of DOS EXE files where the memory image was almost exclusively larger than
the disk image.

Later EXE compressors like the design that Jeremy Collake has written
correct this problem to a very large degree by bringing the disk image down
closer to the memory image size.

Most of the work I do involves tools and similar styles of programs that
are aimed at having a small memory image so an EXE compressor does not
cause any real problem, even with multiple instances loaded. Where it can
become a problem is when a program that is designed to have multiple
instances running has a large memory image to start with.

The solution for large memory model applications under 16 bit windows was
to limit the applications to single instances. 16 bit Word was a good
example of this approach.

Program design can solve this to a reasonably high degree by using properly
designed DLLs where you literally control the paging of program components
by dynamically loading and unloading DLL code on requirement.

This means using LoadLibrary(), GetProcAddress() and FreeLibrary() but it
not difficult code to write and it give the programmer a lot of control
over the memory usage associated with the running program. This capacity is
one of the good things left over from 16 bit windows where memory usage was
critical to keep it from falling over too often.

The utility that I posted in this thread was originally aimed at solving a
problem with using binary resources over a certain size where the resource
Load####() function were disabled. It does that OK but it has an extra
advantage in that the PE format EXE that the extra data is appended to will
load the image size associated with the EXE file in the PE header, not the
actual length of the disk image so the appended data does not effect the
memory image unless it is read from disk and loaded into memory.

This is in fact more efficient in terms of memory usage than using a large
binary resource if the EXE file has been compressed. The EXE that has the
data appended to it can be compressed before the data is appended without
and problems and the data depending on what it is can also be compressed
and reconstructed when loaded.

Regards,

[email protected]

------------------

[jcfuller]

Steve,
Thanks for the insight. I have a few more specific compression
questions that I'll post in the 3rd party forum.

James


------------------