AaronLS
18th October 2008 05:56 UTC

Incremental Updates
I have a installer compiled with NSIS that is roughly 300 mb. Each new version of the installer we release is designed to work for both new installs and updates.

The issue we have is that in some cases updates need to be deployed to a large number of users who are on low bandwidth connections or are on mobile broadband. After the initial install, there are many cases where the amount of data needed for an update may be minimal. I could make seperate installers that only include what's different from version to version, but I don't think I could maintain seperate reduced size installers designed to do the udpates, and also be compatible with all our previous versions. I'd also increase the amount of testing I'd have to do significantly to try all the combinations.

The installer I have now is wonderful, thanks in a large part to the NSIS community, and I think the same scripts could be used for an updater.

If there was a way to modify the bahevior of the File command, such that instead of trying to extract a file, it attempts to download it from some configured location. It would function something like this:
1. Compile installer as a "remote updater". Instead of the File command causing files to be compressed and included in the installer, only a hash/checksum of the file is stored. Thus the installer would be significantly smaller.
2. When running the installer, it would expect that script instructions provide some information about where the actual files should be retrieved from. I.e.
SetRemoteFileSource "\\someServer\networkshare\MyInstallerFiles\"

This at the programmers option this could be set dynamically at runtime. In my case I'd probably make it a command line or a ini file accompaning the installer, so that each user's site would have a different configuration for where the files are stored.

3. As the installer is running and encounters a File command, it uses the checksum/hash of the files to compare against the local files that would have been overwritten had this been a real File command. If the files are different, then the new file is retrieved from the remote location, it's integrity is verified(using the has again), and then the existing logic used for overwriting the old file with the new is applied.

This would minimize the amount of data that must be pushed down to clients before the installer is run, and only transfer the files that are neccessary. My hope would be that implementing this feature using the File command in this way will abstract the file transfer to the extent that existing scripts could be recompiled with minimal modification and take advantage of this feature.

Bah, I just drunkenly misclicked and added "neccessary" to my firefox dictionary.

Unfortunately the only C/C++ experience I had was in college, and I have used it next to never since then, so I don't think I'd be capable.

If we write our own updater for the ground up using another language, then it will surely suck because I have put a lot of effort into our existing NSIS installer. Thanks to poor design of some of the third party components and software our installation depends on, there is a lot of conditional logic and wacky things we have to do to deploy our software. Now that we have completely redone our installer using NSIS and manage the build with some NANT scripts, I am very happy with it.

If we roll our own autoupdater using something like WCF services, but then it will surely suck in comparison to our installer, it will be duplicate management of the same logic that we already put into our isntaller(and I would not want to replace the NSIS installer, because even with an updater, it would still be the ideal choice for new installs), and it probably won't be as easy to write as the NSIS installer was due to the wonderful features NSIS provides.

Any opinions on this?

AaronLS
18th October 2008 06:09 UTC

Maybe since some people would like the files pulled via HTTP, rather form a network share, it should be plugable. I.e the file command does its normal logic, but at the point where it would normally extract the file, it instead calls some function of a plugin which then handles the retrieval of the file, then once that function returns, the file command continues with it's normal logic. There probably also need to be plugable logic for compile time, because the files that are referenced by File commands should probably be gathered into a folder structure that can easily be copied to the network share or http site.

Of course I could do all this stuff without the file command using existing plugins, but it seems it would be tedious and difficult to manage. There are lots of places in our installer that a file command is handling dozens upon dozens of files. In our build environment, most changes to the installer involve just dropping files in appropriate subdirectories, and the recersive File commands do the rest. I love the File command, and I'd hate to replace it.

QOAL
18th October 2008 14:12 UTC

Hrmm, well this may be of some help to you:

http://forums.winamp.com/showthread....hreadid=297533

AaronLS
21st October 2008 15:21 UTC

I will take a look at it. It seems extending the functionality of the File command would be ideal since that wouldn't require additional scripting. I could then still have my traditional "install from CD/thumbdrive/download" that is ideal for new installs, and just recompile with the network functionality and provide that version of the isntall to large agencies wanting to push it out over their network.

I have the source code for NSIS. It looks like the file related operations are on lines 446+:

"#ifdef NSIS_SUPPORT_FILE
case EW_EXTRACTFILE:
{"

My hope is to set it up where you can plugin DLLs so that in the future the File command can be extended however someone wants.

Edit: that's in exec.c

AaronLS
21st October 2008 17:23 UTC

That block of code in exec.c looks like it is what gets run by an installer, correct?

And then it looks like in script.cpp on line 4472 this is the code that is run when compiling a script, correct?

case TOK_FILE:
#ifdef NSIS_SUPPORT_FILE
{...

Just trying to get my head wrapped around the code.

AaronLS
22nd October 2008 04:32 UTC

So just fleshing out the design of this. I'm working on getting NSIS to compile on my machine right now.

User's of this feature would provide two functions. The first is called at compile time for each File command, and it decides what to do with the source file.

The second function is called at runtime, and handles fetching the file.

-Commands at compile time would define what functions handle these things. Perhaps:
SetCompressor /PLUGIN=MyPluginName

Other parameters to that command such as the algorithm type could still be used by the plugin if it itself will be doing any type of compression. But in essence it will handle grabbing the file, so it replaces the step of compressing the file into the installer.

-Of course if you are the implementer of this plugin, you need some info at runtime about where these files should be retrieved from. Depending on how the plugin retrieves the files, the plugin will probably need additional information. I haven't figured out how this will be communicated yet. It would probably be some commands or macros that would set values to be used by the plugin.