vbgunz
9th December 2003 14:20 UTC

Request: A perfect idea for a patch system or am I re-inventing the wheel?
I think this is a great idea. Not sure if it has been mentioned though a search for "patch" on the boards revealed nothing of it.

If the installer can do what I mention here I believe this can make a great patch system. Don't knock this down. NSIS is a superiour installer and I would recommend it to anyone who needs one. Just build with me here :)


1. When an installation has been created, get the exact date and time it was created.

2. Instead of a build button have a patch button on MakeNSIS which will only includes files *after* the last builds date.

The only complaint I can see come from this is, wouldn't it take too long? My answer is better to automate long than to have do it manually.

What do the devs think about this? I currently use another patch system which works great and I can make patches quite fast with it but am just hoping I keep all the build stuff within NSIS.

I'd really like to hear any ideas about this.

Afrow UK
9th December 2003 18:24 UTC

1. Compile date and time are the following:
${__DATE__} and ${__TIME__}

2. Would this be to add extra files rather than recompile the whole installer?

-Stu

vbgunz
9th December 2003 18:46 UTC

Hello Afrow,

In my head I see the simplest way of making NSIS as good a patcher as it is a full blown installer, is to write a command to itself with the exact date and time of its last full version build.

A user can choose full version at any time by doing what they do now. They simply click the build button. This will compile a full version release. This is standard.

The method I have in mind will add a new button to the installer. It will be the patch button. Maybe it can read a certain formatted commented-out date and time within the script and then *only* include files which are newer than the date.

It will still build directory structures exactly the same way it did for the full version except patches will *exclude* files with a date and time older than the last full version build.

Does it make sense? It sounds easy but I wouldn't really say that. I believe if it was it would probably have been implemented a long time ago. Without a system like this (which only in theory is quite simple) builders are left up to using other alternatives for creating patches.

The two greatest elements of any install is either it's first initial full component install or the patch. Not sure how hard it would be to implement it the way I see it but to be honest I think it would be a world simpler than using a file diff tool.


IMHO, this system would be the easiest and quickest method for any builder to easily create patches.

What do you think?

vbgunz
9th December 2003 18:53 UTC

I just came up with another idea. What would be really cool, is if some how NSIS can get mathematic about it. This would require a genius greater than I.

(btw, we're not talking about merging, simply overwriting, NSIS has the tools to merge, etc). This new idea that I got would work like this. Exactly as I mentioned before, it will patch an install by only including files newer than the last full version build *but* it can keep a list of dates.

A user can then choose either through a full install or a patch install, from what version they would like to update from. Some users miss the last few patches, it would be nice to still keep the full version size down by only including patches from 3 full versions ago.

Just brainstorming, would really like to know what others think about this. By the way, this is the genius approach... I think if a simply patch system like my first idea was to be introduced it will be better than the non-existent patch system NSIS has now :)

all just imho.

lewellyn
9th December 2003 20:15 UTC

Maybe I'm missing the point, but if it's for distribution to end-users, what's wrong with VPatch?

I can see adding new files to the end of an uncompressed installer for "in-house" testing builds, but I personally like to test on installers that are built the same way the final will be...

Again, if I'm missing something, please fill me in.

vbgunz
9th December 2003 20:40 UTC

I believe I tried vpatch once and it failed to work for me. This was a while back and things have probably changed since then.

I have both a file diff tool in which I use to make patches and a really cool patch setup program but I would simply like to see such a feature incorporated into NSIS.

There wouldn't be more than one working directory of installation files with my idea so no file differing. This is a benefit as it can save me both space and time.

Another benefit is a centralized work environment for builders. This alone will save a builders time rather than having them bounce from application to application to get an installation type job done.

Another benefit about the method I propose is tiny to no over-head in learning the NSIS installer. If a user can make one build a user can easily make patches there after...

The kind of patch system I propose should be fairly simple. I am not talking about overwrite if newer but *include* into the build *only* if newer. Theres got to be someway to incorporate this kind of feature.

1. NSIS makes the first full version build (the script now knows at run time a full version build was made on N date.

2. Next build a user can choose to make another full version build or opt to make both a full and a patch or just a patch.

3. NSIS will read the build date on the script and then if a builder chooses full NSIS will work as it would normally do, spider the working directory and include all files.

4. If a user chooses to patch only, NSIS will follow step 3 *but* this time only include files with a newer timestamp.

Theres obviously a flaw to this method if one thinks about it. Sometimes a file will be edited and saved and then reverted. This will not be any good in a patch as it is the same file but with a newer time stamp...

I guess at this moment either a file diff tool would be cool at this point or no file diff tool can still make this patch system widely acceptable because files just don't change their timestamp for nothing...

Just brainstorming...

Joost Verburg
9th December 2003 21:25 UTC

VPatch works perfectly fine and is available as NSIS 2 plug-in. It's even included in recent distributions. You can find detailed instructions in the Contrib\VPatch folder. When using VPatch, only differences between files can be included, so it saves lots of space.

If you want to include only changed files, like you described in your post, you can easily write an application (you can execute it using !system) that checks the file times and writes an include file.

kichik
10th December 2003 15:32 UTC

Though I agree this would be a really cool feature, I don't believe it can be done well in makensis itself. I can't think of any good way to do this without limiting options. For example, how would the UI of the patch look like? What pages will it show? Suppose it should show the same pages as the real installer, should the text be changed automatically? What if the user wants to add some custom steps to the patcher? Should it has another part of the script for specifying how the patcher will act but still follow the same script for extracting files?

Also, if we're already making a patch system, why include the entire files? Why not just include the differences like VPatch does? Assuming we do include just the differences, where would makensis get the old files for the patch? That too should be in the IDE. The IDE should take care of all of the files that the project needs and where they are at.

If not the IDE, then a nice set of macros can do the job too. For example, a macro can call a program that compares dates/CRCs in an external database against the current date/CRC and writes the needed commands into a file which will be included by the macro, depending on the current build method defined somewhere.

I would love to see a solution for this, but I just don't see any good way this could be implemented in makensis without automating too much.

vbgunz
11th December 2003 02:27 UTC

Hello Kichik,

I think the best approach without having to go crazy with too many commands...

1. The only one title which will most likely change throughout the entire installer is "Product name" to "Product name patch". I believe everything else can stay the same without a big deal or

2. Add a few simple new switches to a few visual and some non-visual properties... e.g.

-p means include in any patch
-px means exclude from any patch ;Same as no switch
-po means include *only* in patch

The above can be applied liked this

Section-p
This section will be included and shown in the components section. This section will show for both full version and patch installations.
SectionEnd

Section-px ;Same as no switch
This section will not be visible in the components section of a patch version build and it will not be installed on patch versions. This switch is for full versions only.
SectionEnd

Section-po
This section will not show in any full versions and will show-up in patches only.
SectionEnd

At first I didn't think of switches or many options. I thought to myself, try to keep it as simple as possible while offering the advantage of trying to make a good patch system even if it is necessary to apply a psuedo patch option.

The above switches can and should also be applied to functions-p. I believe this is definitely outlining a patch installatin by about 90%. The actual implementation is left up to the NSIS devs and more options can always be added. This is all just a suggestion.

Again, I am only building here. It's just a suggestion and it would be nice to see in NSIS. The two greatest properites of any installations creation is it's full version and patch. NSIS in my eye's has already mastered the full version compilation. It would be nice to see it begin on a patch builder of some sort.

Thank you for your time. I hope others simply come along and build, I believe together we can move quicker to coming up with an idea that not only works but is actually feasible all through NSIS.

Vytautas
11th December 2003 02:50 UTC

I might have the wrong end of the stick here, but can't you achieve that already by using a Custom Page to select if you wan't to patch, I assume you mean upgrade, or just install. Then you can just use a second instfiles page to apply the patches to the system.

I suppose it could be simpler to implement a patch system the way you are proposing, but is it really nessesary?

Vytautas :blah:

vbgunz
11th December 2003 08:19 UTC

Hello Vytautas,

Ok, I need to clarify on my idea a little. I did some reading and noticed I might have confused some of the readers and followers of the thread. I apologize.

There is *no* one final build which a user can launch that will prompt the user (full installation or patch). No one build will do it all, *although* a full version build can easily be managed to patch an installation.

My idea is this. On Makensis.exe itself will be two buttons. One to build, one to patch. A full version build is standard and is what users have now as the only option. I am suggesting a new button (patch build) within Makensis.exe which will compile the standard installation but exclude files with time stamps *older* than the last previous full version build.

My idea is based on removing all third party solutions and the needs to create more than one installation script for one package. I am talking about Makensis.exe being able to build a full version or a patch version based on one script.

At first thought in my head, I figured everything about the installer can stay the same. The only difference would be a patch build and it will exclude all files which have an older time stamp. So the same, introduction, license, custom screens, etc, will always be included in both builds (full and patch).

The only true difference between both builds is the internal file structure and the size of the installer. e.g. A full version build weighing in at 26MB weighs in at only 500kb with a patch build. Follow me :)

The beauty of this is no need to work with more than NSIS and a script editor and no need to work on more than one script.

Kichik brought up a very interesting idea. What if a user needs to or wants to add or remove something and simply make something unique within each build they create... e.g. a patch build will have a new custom screen in which the full version does not display... Well, this makes a lot of sense, and when I thought about it, my main angle is simply being able to keep the workflow as simple as possible.

My idea, is to not involve third party scripts, applications or work on the same script twice (two install scripts). I then thought the best way to achieve such a behaviour with one script and Makensis alone is to simply add switchs to the NSIS directives.

-p means include in any patch
-px means exclude from any patch ;Same as no switch
-po means include *only* in patch

I am not sure if this would be easy so I cannot say it would be easy for the devs to do. I know though it should be easy for users if such an option existed.

While some users might complain but this patch system isn't all too dynamic, etc, the honest to god truth is, some kind of patch system is always better than no kind of patch system at all. Even if all screens on both installers stay the same as my original idea, I think the patch system gets off on a great start if the patch system excludes files based on the last full version build.

Also, while some might argue but this can all be achieved with 3rd party applications and more than one install script, I say you're absolutely right. I do it myself at the moment and this is only a suggestion. NSIS is already full grown at making beautiful installations. Maybe a little time to focus on how to patch would be a good idea. This is why I brought up this thread. I believe a way to patch within NSIS would be a good thing and am only asking for others to help build with me here.

So in the end, Makensis will have two buttons. One will create a full final version build and the other will create a simple patch.

Only including files with a later time stamp is a good idea. Adding to it of course would be better but this is how I start this thread off. Ideas are always welcomed.

Vytautas
11th December 2003 11:37 UTC

OK, I did have the wrong end of the stick. :o You mean a patch system as in generating a service release to a program after the main installer has been shipped. Hey that's a good idea. :up:

Vytautas :D

kichik
11th December 2003 15:44 UTC

Section-* could allow the user to create both patch and installer in one script but there are still some problems. How will the compiler know which date it should "patch" against? What if the user first used build, build and then patch? Should it not write the time stamp on the second build and only write one after a patch or the first compile? What if the user wants to build a patch from an earlier date? Should we strict it to v1->v2, v2->v3, v3->v4 instead of allowing him/her to create v1->v4?

Also, what about files that should no longer be included in the installer? How will makensis know of them? What about changing registry keys, INI files or normal files from one version to another? All these can't be done by makensis without keeping a detailed database of changes to the script and data files.

I don't see anyway this could be done well without restricting, automating or turning makensis into a complete IDE. The only way I can think of that will be somewhat acceptable and could help you achieve your goal is a command called something like IgnoreFileOlderThan <date>.

vbgunz
11th December 2003 16:49 UTC

Hello Kichik,

I love your programming brain. I can tell you've got your hamsters spinning the wheel on overdrive. I like this, and because of it, you make my hamsters spin a little harder ;)

I do not have an answer for everything. I need you to know this, though it will seem like it, truth is, I only have ideas in which I only hope can be applied in a sensible manner. I'll try to rebuttal some of your negative comments (can't) with some idea's about how.

Please, I am no know-it-all, just an idealist.

How will the compiler know which date it should "patch" against? What if the user first used build, build and then patch? Should it not write the time stamp on the second build and only write one after a patch or the first compile?

Also, what about files that should no longer be included in the installer? How will makensis know of them? What about changing registry keys, INI files or normal files from one version to another?

Comm@nder21
12th December 2003 14:33 UTC

but isnt that "IgnoreFilesOlderThan" command included yet in parts by this command:
SetOverwrite ifnewer
?

vbgunz
12th December 2003 14:40 UTC

Hello Comm@nder21,

SetOverwrite ifnewer is a run time command and not a compile time command. SetOverwrite ifnewer will always include all files in a build. If IgnoreFilesOlderThan was a command, devs can opt to create much lighter installations in some instances.

kichik
12th December 2003 23:01 UTC

I don't expect you to have to answers to all of my questions, I'm just brainstorming too ;)

I forgot to mention that all that expect the selective date sensitive inclusion of files everything can already be done using the preprocessor. To write to a header file you can use !system "echo bla bla > file". Inclusion of certain patch or installer specific parts can be controlled by a define, PATCH for example. I think that because it uses existing methods it will require a lot less work and time thinking about so many small details and will eliminate a lot of redundancy that could have been caused by adding new patch specific commands to makensis.

Here's an example script that demonstrates what I mean:

!ifndef PATCH
!system  "echo ${__TIMESTAMP__} >> builds.lst"
!endif

!define uninst_key Software\Microsoft\Windows\CurrentVersion\Uninstall

Name bla
!ifdef PATCH
Caption "$(^Name) patcher"
OutFile bla-patch.exe
!else
OutFile bla-setup.exe
!endif

Section
File bla.exe
File bla.txt
!ifndef patch
WriteRegStr HKLM "${uninst_key}\bla" "UninstallString" '"$INSTDIR\uninst.exe"'
WriteRegStr HKLM "${uninst_key}\bla" "InstallLocation" "$INSTDIR"
!endif
SectionEnd

vbgunz
13th December 2003 00:58 UTC

Hello Kichik,

I'll leave the technicalities and programmatic thinking in your court. I am certain you and the NSIS team can make this happen. My angle is an idea and all of the feedback you might need. I believe merge the two with more feedback from both sides and we'll take steps others will be sure to follow.

When I had the conception of a patch I didn't think complex file differing or merging of binaries, etc. I thought, if only one difference mattered most, what would it be? I didn't think byte for byte, line for line or crc checksum. I thought date and time would be as simple as black and white. Exlcude the old, include the new. If it has already been included, no need to add it again...

Some might argue about doing it the CRC, byte for byte, line for line way but personally, if it's older than N date exlcude it. Keep it simple for now. In due time it will grow and those who can help make it grow, will. I personally, will help in any way I can so feel free to post here or contact me for testing. Simple is better than null any day and so we move ahead with one more post.

I would really like to hear what others got to say because personally I have some really great patch systems but still would rather have a simpler NSIS option than a complexed do-it-all third party app. Well, I am done, what do others think? Well, let's find out, 1, 2, 3, 3 licks to the center of a tootie roll pop.

Kichik, I'll test anything. Thank you for your interest and time.

kichik
13th December 2003 14:55 UTC

Expect to see this in NSIS 2.1. Thanks for the nice idea.

vbgunz
13th December 2003 14:59 UTC

Hello Kichik,

I expect for NSIS to be a full blown operating system by 2.1. If it isn't I'll settle for the patch system :)

Thank you again Kichik

eccles
13th December 2003 15:47 UTC

Might I suggest that any new compiler flag such as "IgnoreFilesOlderThan" mentioned above behave like "SetOverwrite" and friends in that it can appear multiple times and affect the script from that point onwards only.

Suggested name: FileFilterDateRange from[ to]

Alternatively make it an optional parameter to the File command.

I agree that the preprocessor should be used for whatever can be got away with, at this stage at least.

vbgunz
13th December 2003 17:30 UTC

Hello Eccles,

Wouldn't this effect sections only? I believe the switch might be a little handier.

The top of the nsi script can hold all the dates. If not, an nsh file can hold all the dates. Then one date gets the switch "-p".

Perhaps the -p switch can signify one thing for the *patch build only*. Include files only from that date onward. Then, to not have to turn a template into two different scripts in one body, seperate content and action with more switches.

Anything which starts with a -p immediately means "patch".

Section-p
SectionEnd

The above section is included both in the full version and patch build.

Section-po
SectionEnd

A full version build will skip the above section as it is a patch only switch.

Section-px
SectionEnd

A patch build will skip the above section as it is for a full version build only. Also, we can duplicate the -px switch by not adding it at all. We just have to get the patch builder to understand, no switch means no adding to patch.

I apologize if I in any way sound repetitive. I am thinking in the sense of trying to be clean while being able to do everything within one script.

If I followed you wrong please advise. I am interested in what you think.

Thank you

eccles
13th December 2003 18:25 UTC

Hi Victor,

Wouldn't this effect sections only?

The top of the nsi script can hold all the dates. If not, an nsh file can hold all the dates. Then one date gets the switch "-p".

Section-p
SectionEnd

The above section is included both in the full version and patch build.

Section-po
SectionEnd

A full version build will skip the above section as it is a patch only switch.

Section-px
SectionEnd

A patch build will skip the above section as it is for a full version build only.

vbgunz
13th December 2003 19:34 UTC

Hello Eccles,

I know you and Kichik are on the same team but I thought you meant defining "IgnoreFilesOlderThan" the same way we define "SetOverWrite". Either way it get's implemented I am sure it will be done right. You've guys done great so far so I would expect nothing less than a well thought out patch system.

Thanks Eccles

Koen van de Sande
19th December 2003 19:37 UTC

It makes sense to make "IgnoreFilesOlderThan" a command which only applies to a certain section, that way you can control sections so only changed files from a certain section are installed (but certain other sections' files are always installed, even in the patch).
Of course, one could have a global version as well which affects all sections... but that is slightly less powerful.

My thoughts on a patching system: most of the time, you have a 'file tree' for your application, which contains all files for a certain release. Now, if you keep a copy of this file tree (which you'd include in NSIS with File /r *.*) for every release you make, it would be nice to have an automatic system to generate patches between file trees of different versions (which installs new files, patches changed files with VPatch). Only problem, as mentioned above: removed files aren't removed (too dangerous).
One could make an external tool which generates a script for this patch between trees and then include it using !include?
This system has problems of it's own of course, but it's a possibility.

Related suggestion

A suggestion related to this:
Often, a script has a number of defines (like PATCH above) and it would be nice to have some kind of 'define profile' in MakeNSIS when building your script.
E.g. you can select "Full" with defines INCLUDE_RUNTIMES and INCLUDE_SOURCE and you can select "Minimal distro" which doesn't define anything or "Patch" which defines PATCH, and after that you can select "Build" to make that particular profile. Note that this would be even more useful if you could specify the values for these defines (not sure if that's possible through the command-line /D?), like PATCH_ORG=1.1 and PATCH_DEST=1.2 to generate a patch tree between 1.1 and 1.2 using the system with an external tool above.
Not sure if it's very useful but just an idea I had ;)

VPatch updated

Since VPatch has been mentioned a few times in this thread: I've send an updated version to Kichik which fixes crashes when given invalid arguments and fixes a bug when patching a very small file (< 64 bytes) into a file larger than 64 bytes. It should appear in a development snapshot soon I think.

HG-Owen
25th December 2003 08:31 UTC

I hate to raise old threads, but maybe you would specify things like this:

file data\info.dat
file bin\exe.exe