Designing an auto-update feature

[Bern Ertl]

I would like to implement a feature in an application I'm developing will automatically download any updates to the software or associated data files from our company website (from a password protected folder) and install them (as long as the user is current with their maintenance agreement of course).

Has anyone done something similar to this already? I've never fooled around with programming internet access, so I'm not sure what is involved in accessing a file from a password protected folder (Apache web server).

It would be nice if I could update a log file on the server as well showing who has accessed/downloaded an update.

Also, how could a program update itself (replacing the .EXE file when it's already running)?

[Scott Slater]

Also, how could a program update itself (replacing the .EXE file when it's already running)?

Most auto-updating programs have an auto-updater that does the download and replace. It is a separate ap that would run regardless of the main up running. Usually a tray ap or a service.

Awhile back someone posed an HTTP utility that would get a file from a website. You would need one that handled whatever type of security that the web server was using. Usually BASIC or DIGEST. DIGEST uses a more complex system of a secure authentication by exchanging tokens. I used the very same code and modded it for DIGEST authentication for an ap that I wrote that has to retrieve logs from a HTTP device. Search the forums for HTTP GET, and you should come across it. This would be a good starting place.

[Michael Mattias]

Also, how could a program update itself (replacing the .EXE file when it's already running)?

Cannot be done. The best you can do is either ..

Use MoveFileEx() with MOVEFILE_DELAY_UNTIL_REBOOT flag and prompt user to do so.
OR
Your update program can detect that the 'program of interest' is already running (mutex probably most straightforward way), and prompt the user to 'end' before continuing.

MCM

[John Petty]

Bern
I would suggest getting totally away from http and port 80 for many reasons including the security settings of most browsers. If you have a server running a version of Windows then you can do the whole thing in PB. Create an entry on your DNS server for something like "update.domain_name.com". In your clients program at start up do a TCP connection to that address on some uncommon port number you pick ie 6543. The client then sends their relevant user data with whatever level of encryption you choose. If updates are available then the server notifies this and the original client program then starts a seperate download program (possibly passing it a authorising key)and quits.
The download program then connects to a server (it can be at a different IP address and on a different port again), downloads the new programs/data (compressed?) to temp files. After verifying the downloads it then does the updates, restarts the client program and itself quits (if it needs to make changes to Windows secure folders then just like Windows updates it puts the requested file copies in the registry and advises the client to restart).
John

[Edwin Knoppert]

The old app could call a hidden bat file using EXIST and GOTO and DEL.
Once the bat is executed the old app may terminate, the bat takes over..
Simple and i have tested this a few years ago.

(PwrDev users can download an example:

http://www.hellobasic.com/cgi-bin/forum/YaBB.pl?board=VDExamples;action=display;num=1146778328

Sorry no example here...)

[Michael Mattias]

The old app could call a hidden bat file using EXIST and GOTO and DEL.
Once the bat is executed the old app may terminate, the bat takes over..
Simple and i have tested this a few years ago.

Edwin, this may "work" but it relies on timing issues.. specifically Windows freeing its open handle to the executable file on disk before the DEL is attempted.

Call me a purist, but seems to me if you want to publish (internally or externally) an application which includes an "automatic updates from the internet" feature, the author may as well "do it right" and not rely on cheap tricks.

Using a batch file here would be like using a Yugo or Trabant interior in a Cadillac.


MCM

[Edwin Knoppert]

As long EXIST = true it will remain in the loop.
it works..

Oh and you can call the new exe to copy itself, alternatives enough

Trabant?
The bat file can be executed hidden..

[Michael Mattias]

> As long EXIST = true it will remain in the loop.
> ...
> The bat file can be executed hidden

There is no elevator to success; there are only stairs.

(Philistines!)

[Edwin Knoppert]

Yawwwn

[Michael Mattias]

>Trabant?

http://en.wikipedia.org/wiki/GDR_jokes

[Mike Doty]

'Original.bas
$UpdateProgram = "Updater.exe"
#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
   CALL UpdateMySelf
END FUNCTION
SUB UpdateMySelf
  LOCAL pid AS DWORD, Result AS LONG
  Result = MSGBOX("Update myself? " + EXE.FULL$, %MB_YESNO, EXE.FULL$)
  IF Result = %IDYES THEN
    pid =  SHELL ($UpdateProgram + " "  + EXE.FULL$)  'optionally pass our name
  END IF
END SUB

'updater.bas
#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
  LOCAL OldName AS STRING
  LOCAL NewName AS STRING
  LOCAL txt     AS STRING
 
  REM Rename program that was running
  OldName$ = COMMAND$
  NewName$ =  OldName$ + ".backup"
  KILL NewName$:ERRCLEAR     'get rid of old backup
  'SLEEP 500 'might be needed
  NAME OldName$ AS NewName$  'create backup
  IF ERR THEN
     MSGBOX "Error" + STR$(ERRCLEAR) + " unable to rename " + _
             $DQ + OldName$ + $DQ + _
             " to " + $DQ + NewName$ + $DQ,,EXE.FULL$
     EXIT FUNCTION
  END IF
  MSGBOX "Renamed " + OldName$ + " to " + Newname$,,EXE.FULL$
 
  '--- SUCCESS   TCP FILES,  OR BETTER YET INSTALLATION PROGRAM TO OURSELF ---------------------------------
  FILECOPY NewName$, OldName$   'Simulate update (use old file)
  IF ERR THEN ? "FILECOPY error" + STR$(ERRCLEAR)
 
  SHELL OldName$                'optional, restart
END FUNCTION

There is nothing wrong with shelling to a batch file to do everything.
The batch file can be created on the fly to do different things.
Many programs need to reboot and call a batch file on restart.

This does not do the internet part (original question.)

[Kev Peel]

I can verify the BAT deletion of the EXE works fine on every system I tried it (all versions of Windows, multiple users). The biggest challenge here is that a limited account user on Windows XP, Vista and future Windows will not be able to change protected files.

In your update installer stub it would be best to include an XML resource which defines the installer as "Administrator", then limited account users will be shown a dialog asking them to log in as admin so the update can proceed. You should also only have the installer download the update files (zip, etc) to the app's data directory, NOT the program folder - this can also fail. Then after downloading, have your app SHELL to the installer, exit, and it will perform the relevant updates and delete itself with the batch file.

I've had this feature in my programs for over 4 years and my end-users love it. That's not to say it was an "easy" feature to add, but it's definitely worth the work. If I discover a bug in my app, then it's just a case of FTP'ing a new update archive, change the version log and all Internet-enabled installations will get the update.

[Bern Ertl]

... Awhile back someone posed an HTTP utility that would get a file from a website. You would need one that handled whatever type of security that the web server was using. Usually BASIC or DIGEST. DIGEST uses a more complex system of a secure authentication by exchanging tokens. I used the very same code and modded it for DIGEST authentication for an ap that I wrote that has to retrieve logs from a HTTP device. Search the forums for HTTP GET, and you should come across it. This would be a good starting place.

Thanks Scott. A search turned up 500 results (the max before truncation I assume). Any idea what the thread was called or who posted the code? Was it this one?

Unattended FTP file download

I would suggest getting totally away from http and port 80 for many reasons including the security settings of most browsers.

Is the sample code in the link I posted above using HTTP and port 80? I only skimmed it, but it appears to be using FTP and port 21.

If you have a server running a version of Windows then you can do the whole thing in PB.

Our web server is Linux based.

In your update installer stub it would be best to include an XML resource which defines the installer as "Administrator", then limited account users will be shown a dialog asking them to log in as admin so the update can proceed.

Could you possibly post a sample of what this XML resource looks like (or a link to an explanation on how to make one)?

I've had this feature in my programs for over 4 years and my end-users love it. That's not to say it was an "easy" feature to add, but it's definitely worth the work. If I discover a bug in my app, then it's just a case of FTP'ing a new update archive, change the version log and all Internet-enabled installations will get the update.

Thanks (for the feedback) Kev.

I'm thinking I'd like for my application to check (on start-up or user command) the web server for an "update instruction file" and if it references a newer update than what's already installed, download the associated update and a batch file with updating commands and then shell to it.

[Kev Peel]

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <assemblyIdentity version="1.0.0.0" processorArchitecture="X86" name="AUTHOR.PROGRAMNAME" type="win32"/>
  <description>PROGRAM_DESC_HERE</description>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity type="win32" name="Microsoft.Windows.Common-Controls" version="6.0.0.0" processorArchitecture="X86" publicKeyToken="6595b64144ccf1df" language="*"/>
    </dependentAssembly>
  </dependency>
  <!-- WARNING: This application requires ADMINISTRATOR privileges. -->
  <ms_asmv2:trustInfo xmlns:ms_asmv2="urn:schemas-microsoft-com:asm.v2">
    <ms_asmv2:security>
      <ms_asmv2:requestedPrivileges>
        <ms_asmv2:requestedExecutionLevel
          level="requireAdministrator"
          uiAccess="false"/>
        </ms_asmv2:requestedPrivileges>
       </ms_asmv2:security>
  </ms_asmv2:trustInfo>
</assembly>

Replace "AUTHOR.PROGRAMNAME" and "PROGRAM_DESC" with the relevant info.

Note: you can't just create a BAT file and do the update just using that, the purpose of the BAT file is to delete the installer program after it's finished replacing the program EXE and other files.

[Bern Ertl]

In your update installer stub ...

What do you mean by a stub?

[Kev Peel]

It just means the update program that extracts/copies the files and deletes itself.

Here's an interesting thread on the subject of auto-updating your app:

http://discuss.fogcreek.com/joelonsoftware5/default.asp?cmd=show&ixPost=150473&ixReplies=14

[John Petty]

Bern
I wasn't referring to that particular sample, rather your comment about having an Apache web server. Actually my point was staying away from using any intermediate programs or protocols like HTTP or FTP, thus my comments about using a Windows based server as to date PB only writes compilers for that platform. The term Windows server was probably misleading as its power would only be a consideration based on the number of concurrent connections required ie XP Pro handles 10 or go back to 98SE which handles many more. I would expect your company has a NAT/PAT firewall so no need to even have a different DNS listing if your Apache server is in house as the firewall will direct the packets to the correct computer based on the port you choose.
By writing the Update "server" yourself totally in PB I see many advantages ie if you use HTTP and Apache then the Apache server would need to call a cgi program to verify the users credentials and payment status. FTP can't verify the users payment status, you have to be activating/deactivating allowed users. By writing the entire system yourself you have much easier control of all these features as well as logging, passing other messages to the client like "update available but your payments not current" etc
Personally I would include the installer program in the initial package rather than downloading as it could then have an appropriate manifest to overcome security issues. The permanent installer can then also perform the other important function if you main program is distrubuted on a LAN of keeping all the workstations versions synchronised.
John

[Bern Ertl]

Note: you can't just create a BAT file and do the update just using that, the purpose of the BAT file is to delete the installer program after it's finished replacing the program EXE and other files.

OK. I could write an update program that reads simple scripts (in case the update is more complicated than just replacing a single file). I don't think it's necessary to delete the update application (which could be installed with the initial distribution).

If the update application (which downloads an update from the web server and then replaces the main application EXE file) still need an XML resource if it is installed with the initial software package and never deleted? If so, what exactly am I supposed to do with the XML resource - just save it as an <update application name>.XML file in the application folder? If not, how could the installer application in this scenario check/ensure that it has permission to delete/write files in the main application installation folder?

... if your Apache server is in house ...

It's not. We contract with a web hosting company.

By writing the Update "server" yourself totally in PB I see many advantages ie if you use HTTP and Apache then the Apache server would need to call a cgi program to verify the users credentials and payment status. FTP can't verify the users payment status, you have to be activating/deactivating allowed users. By writing the entire system yourself you have much easier control of all these features as well as logging, passing other messages to the client like "update available but your payments not current" etc

I'm handling the payment/access differently. It's not necessary to process this on the web server.

The client application will only download the update if they are allowed to do so. If not, it will alert them that they need to update their maintenance license to get the update.

I'm not terribly concerned with hackers gaining unauthorized access to an update.

[Michael Mattias]

I do PPPS updates from my web site, just as you describe: if the user has current maintenance, he can download the update; if not, he can't.

My web hosting company provided a database for me to use, where I can maintain serial numbers and the date maintenance expires for each serial number; this is checked when the user logs in and either allows user to open the update file (or download if they don't want to run directly from the internet) or gives them a message that either serial number is invalid or maintenance is expired.

Ok, so it's not a "pure PB solution:"

The database and maintenance are provided by the web hosting company. It cost me $150.00 one time to have it set up and the user screens created. I email them update files when needed.

I create the "update.exe" using Inno Setup.

The software does not automatically check for updates, the user must go to the web site.

But I could fairly easily add "automatic check for updates" from the application and transmit the serial number when doing so; or (what I prefer), add a "check for updates" function to the application. (Personally I hate software which checks for me and then interrupts whatever I am doing to tell me an update is available, but that's just personal preference).

While this thread has wandered into "replace running EXE" ( Can Not! Con So! Can Not! Can So!) and an alphabet soup of communications protocols and equipment (FTP HTTP NAT/PAT CBS/NBC/CNN), the original question was, "has anybody done something like this?"

The point being here... why not ask your web hosting service if they might have some ideas for you, and if so, what would you have to do on the 'application side' to implement one of those ideas?

Surely a web hosting service has fielded the same question from their other clients and would like to expand the services they provide and/or remain competive with other hosting services.

MCM

[Bern Ertl]

While this thread has wandered into "replace running EXE" ( Can Not! Con So! Can Not! Can So!) and an alphabet soup of communications protocols and equipment (FTP HTTP NAT/PAT CBS/NBC/CNN), the original question was, "has anybody done something like this?"

They are the two essential problems that need to be solved to achieve the objective.