I have a fairly complex and huge installer I'm working on right now, and have an issue where on Vista (not XP), it takes forever to start up on some machines.
For an idea of the size of this thing, it's about 6000 lines (without counting all the includes...), about 12000 files (for a total of about 1.25 GB), and with SetCompressor lzma weighs in at about 700 MB.
As it needs to write to \Program Files, it has RequestExecutionLevel admin. The installer is also Authenticode-signed.
I have tried usiing the UAC plugin instead of RequestExecutionLevel admin (that causes... additional issues...), but that did nothing to speed up what I assume is a verification phase.
I've ruled out antivirus and such, and disk performance is certainly not the issue. And by slow, I mean: "It takes longer to launch the installer than to build it." I've also tried all variants of compression. (That should rule out ReserveFile... But, if not, how would I determine *which* file was causing this? All my functions are in an include at the very top of the script...)
Has anyone encountered something similar? It's unreasonable to release a product which can conceivably take > 15 minutes to "do something" at install time (the customer's first impression, too!), and plopping up a splash screen that says "now loading" just won't cut it...
Extremely slow installer startup on Vista
47 posts
have you tried using processmonitor to see what your installer's doing during this log wait?
It seems that Windows is copying the entire shebang to %TEMP%... This, also, seems "not right". :P
are you using solid compression?
I've tried both solid and non-solid compression, as well as with compression off. At this point, it seems my only option is to include the bulk of the data in an external file, uncompressing it during the point I'd normally copy files. 🙁
Have you tried without signing? Signing involves storing a hash of the signed executable, and this could take quite a while to verify. If the source media is slow, that could slow the verification down considerably, in which case a different method might be preferable (such as storing the files seperately, or perhaps only signing a stub .exe that requests permissions and launches the real installer).
Yes, I've tried signing and not, RequestExecutionLevel admin and undefined, compression on and off, solid and not, zlib, bzip2, and lzma. All seem to cause the installer to be copied to %TEMP%.
The only way I seem to be able to avoid getting it copied to the temp dir is by requesting user-level permissions. Of course, this doesn't work terribly well for installing system-wide. 🙁
Also, the source media is irrelevant; I can duplicate this from local hard disk, network share, CD-ROM, and USB flash drive.
Hopefully, I'll get a chance to whip together a small test-case in a little bit.
The only way I seem to be able to avoid getting it copied to the temp dir is by requesting user-level permissions. Of course, this doesn't work terribly well for installing system-wide. 🙁
Also, the source media is irrelevant; I can duplicate this from local hard disk, network share, CD-ROM, and USB flash drive.
Hopefully, I'll get a chance to whip together a small test-case in a little bit.
I hate bumping threads, but I was curious if anyone's come up with any ideas. I'm not using ReserveFile anywhere, all my functions are defined at the top of my script, it doesn't happen on XP (nothing earlier is supported for this app, so remain untested), it happens on both Vista and Windows 7 (meaning this isn't a "passing Vista issue" or something that may disappear in a Service Pack), it happens regardless of RequestExecutionLevel being admin or none (or user + the UAC plugin), signing doesn't affect the behavior positively nor negatively, compression set to anything, solid or not...
I was going to come up with a smallish testcase, but I decided there's no great way to do so that I can think of. Certainly, if I figure out a test case that's not a few hundred megs in size that's likely to work, I'll post one. 🙂
As it stands, it looks like we're going to have to go with a 30-ish megabyte "mini installer" (about the largest which doesn't "feel slow" while copying to %TEMP%) which does the install of the bulk of the files (again, more than ten thousand files weighing in at well over a gig) from a compressed archive, external of the installer.
The real problem with this is that it's a second file, of course. A second file means that distribution methods are slightly more limited. (I don't make the distribution decisions, but I would rather not have to revisit this issue in some months.)
Also, there's been testing done on a fair sampling of our large-ish installers from the past 5+ years, and it seems that they all behave this way on Vista. 🙁 But this is the first one that I recall doing with such a large installed footprint, personally.
In any case, it would be awesome if someone could give some pointers on how to make the installer not copy to the temp dir. 🙂
I was going to come up with a smallish testcase, but I decided there's no great way to do so that I can think of. Certainly, if I figure out a test case that's not a few hundred megs in size that's likely to work, I'll post one. 🙂
As it stands, it looks like we're going to have to go with a 30-ish megabyte "mini installer" (about the largest which doesn't "feel slow" while copying to %TEMP%) which does the install of the bulk of the files (again, more than ten thousand files weighing in at well over a gig) from a compressed archive, external of the installer.
The real problem with this is that it's a second file, of course. A second file means that distribution methods are slightly more limited. (I don't make the distribution decisions, but I would rather not have to revisit this issue in some months.)
Also, there's been testing done on a fair sampling of our large-ish installers from the past 5+ years, and it seems that they all behave this way on Vista. 🙁 But this is the first one that I recall doing with such a large installed footprint, personally.
In any case, it would be awesome if someone could give some pointers on how to make the installer not copy to the temp dir. 🙂
It has to do with UAC elevation somehow... Are you starting the installer from a shared folder or something?
Thanks Anders for the quick reply again. 🙂
I figured it had to do with UAC; but I can't quite finger what... And, as stated before, I can replicate this from across the network, on optical storage, USB flash, USB hard disk, as well as various local hard disk locations. It really seems to not care where it's running from. 🙁
I'm having a test run right now where they have 3 installers: no compression (SetCompress off), bzip2 (SetCompressor bzip2), and LZMA Solid (SetCompressor /FINAL /SOLID lzma). They will be testing from various install media with the installers both signed and unsigned across various machines. We're hoping that there's a result of "That particular combination seemed to not take forever!" for some combination, though my hopes are not high.
I have been searching the forum, as I thought I posted something about this in 2003 or so. (I seem to recall my first multi-hundred-megabyte installer having a slowness-while-launching-from-CD issue on 2000... Though that was probably around the introduction of /SOLID and me not knowing how to use it yet or something... So I wanted to see if it was potentially related to this issue, at the least.) But, all I could find were a couple of scattered posts of people complaining about the issue and the "solution" being to just not have such large installers.
I recall that there's a couple of fairly large installs out there which use NSIS... How are they coping with this, as it appears to not be isolated to me?
(And I am still trying to figure out how to get a minimal test case up... And, sadly, no, I can't post my "real" .nsi for various reasons [not least would be "permission", followed by "altogether, we're looking at a megabyte of installer script"], so if you can't duplicate it there, let's see if my brain kicks in something smart.)
I figured it had to do with UAC; but I can't quite finger what... And, as stated before, I can replicate this from across the network, on optical storage, USB flash, USB hard disk, as well as various local hard disk locations. It really seems to not care where it's running from. 🙁
I'm having a test run right now where they have 3 installers: no compression (SetCompress off), bzip2 (SetCompressor bzip2), and LZMA Solid (SetCompressor /FINAL /SOLID lzma). They will be testing from various install media with the installers both signed and unsigned across various machines. We're hoping that there's a result of "That particular combination seemed to not take forever!" for some combination, though my hopes are not high.
I have been searching the forum, as I thought I posted something about this in 2003 or so. (I seem to recall my first multi-hundred-megabyte installer having a slowness-while-launching-from-CD issue on 2000... Though that was probably around the introduction of /SOLID and me not knowing how to use it yet or something... So I wanted to see if it was potentially related to this issue, at the least.) But, all I could find were a couple of scattered posts of people complaining about the issue and the "solution" being to just not have such large installers.
I recall that there's a couple of fairly large installs out there which use NSIS... How are they coping with this, as it appears to not be isolated to me?
(And I am still trying to figure out how to get a minimal test case up... And, sadly, no, I can't post my "real" .nsi for various reasons [not least would be "permission", followed by "altogether, we're looking at a megabyte of installer script"], so if you can't duplicate it there, let's see if my brain kicks in something smart.)
when you say its copied to temp, are you talking about the .exe? what is the filename? the same as the "original" exe? Check in Process Explorer, are there 1 or 2 processes etc
I just ran a check again, from two approaches.
1) Process Explorer shows one process.
2) Watching from Process Monitor, it appears that Windows Explorer does its normal "launching an exe" reads. Then, svchost.exe takes over and copies to C:\Windows\Temp. In the case I'm watching, it's being copied as C:\Windows\Temp\20,208D.tmp from a folder in my %USERPROFILE%. Then, the .tmp file executes and consent.exe takes over.
Anything else I can do to shed light on this? Thanks.
1) Process Explorer shows one process.
2) Watching from Process Monitor, it appears that Windows Explorer does its normal "launching an exe" reads. Then, svchost.exe takes over and copies to C:\Windows\Temp. In the case I'm watching, it's being copied as C:\Windows\Temp\20,208D.tmp from a folder in my %USERPROFILE%. Then, the .tmp file executes and consent.exe takes over.
Anything else I can do to shed light on this? Thanks.
Test Case!
OK, I figured out how to get a test case. 😁
Attached is a 7-zip file (couldn't get the 1 gig file compressed to under 200 KB with .zip) with a 1 gig test file and a very minimal script. Yes, you'll need about 3-4 gigabytes of free disk space to test this. :P (At least: 1 gig to extract the archive, 1 gig to build it, 1 gig to run it.)
If someone could verify that they see the described behavior, it'd be awesome. 🙂 I'm tired of feeling like I've gone insane... 🙁
Note that the versioning keys are irrelevant to whether it does this or not. I included them out of habit.
N.B. The installation portion of this thing is untested. I've only tested its behavior up to the consent prompt. 🙂
OK, I figured out how to get a test case. 😁
Attached is a 7-zip file (couldn't get the 1 gig file compressed to under 200 KB with .zip) with a 1 gig test file and a very minimal script. Yes, you'll need about 3-4 gigabytes of free disk space to test this. :P (At least: 1 gig to extract the archive, 1 gig to build it, 1 gig to run it.)
If someone could verify that they see the described behavior, it'd be awesome. 🙂 I'm tired of feeling like I've gone insane... 🙁
Note that the versioning keys are irrelevant to whether it does this or not. I included them out of habit.
N.B. The installation portion of this thing is untested. I've only tested its behavior up to the consent prompt. 🙂
Well, I can certainly verify it's being copied and that the UAC prompt takes ages to appear.
I do not see the issue if RequestExecutionLevel is set to user, but if the issue pops up once the UAC plugin is used, that doesn't do you much good.
I also do not see the issue if the installer is launched from a process that is already elevated (Command Prompt, in my testcase).
This is a workaround, but if you really want to stick to a single file, you could use a form of bootstrapping. You make a bootstrapper that requires admin, and only serves to launch the real installer with these privileges. This bootstrapper, and the installer, is then packaged into a single file that has RequestExecutionLevel user, and extracts the two files to %TEMP% and launches the bootstrapper.
It'll take a while to extract, and it's going to require more disk space on the user's computer - but at least it won't look like nothing's happening.
I do not see the issue if RequestExecutionLevel is set to user, but if the issue pops up once the UAC plugin is used, that doesn't do you much good.
I also do not see the issue if the installer is launched from a process that is already elevated (Command Prompt, in my testcase).
This is a workaround, but if you really want to stick to a single file, you could use a form of bootstrapping. You make a bootstrapper that requires admin, and only serves to launch the real installer with these privileges. This bootstrapper, and the installer, is then packaged into a single file that has RequestExecutionLevel user, and extracts the two files to %TEMP% and launches the bootstrapper.
It'll take a while to extract, and it's going to require more disk space on the user's computer - but at least it won't look like nothing's happening.
But, still, the extraction will take a while. :P And people expect to see the welcome page of an installer pretty quickly. With the bulk of the stuff in an external data file (using Nsis7z, for the compression and the fancy ExtractWithDetails), I have a 5-8 MB installer (for things that shouldn't go into $INSTDIR for whatever reason). That, at least, comes up reasonably fast.
And, yes, the UAC plugin doesn't change the issue. Well, it does from a watching-the-processes standpoint, but nothing a user can see. It seems Windows really wants the thing in %TEMP% before elevation. 🙁
I suppose, now that someone has verified it, the question is: "Who's at fault? NSIS or Windows?" This would be followed by the corollary of: "Can it be fixed within NSIS?"
And, yes, the UAC plugin doesn't change the issue. Well, it does from a watching-the-processes standpoint, but nothing a user can see. It seems Windows really wants the thing in %TEMP% before elevation. 🙁
I suppose, now that someone has verified it, the question is: "Who's at fault? NSIS or Windows?" This would be followed by the corollary of: "Can it be fixed within NSIS?"
I don't really have any workaround suggestions other than testing with the nightly build. And you should file a bug report so this does not get lost on the forum
I verified that it's an issue in the nightlies, and I verified with a circa-2002 installer I had in my bookcase. (Is there a way to determine version of an NSIS installer?) So, this is definitely a long-standing issue.
A quick poke around shows this to be a much-reported issue on the tracker. As far as I can tell, kichik marks them all as dupes, with a trail leading to a bug for making multi-archive installers. (Which, really, is the polar opposite of what is desired here.) Hopefully, he won't do so with [url=http://sourceforge.net/tracker/?func=detail&aid=2785629&group_id=22049&atid=373085]my latest bug[url]. 😎
As I mentioned in the bug, this makes NSIS unusable for many use cases where a single-file installer is necessary. Which means not only having the same program with an NSIS multi-part installer and a "solid" installer in another system (or just skipping the NSIS version altogether...), but having to keep people trained on both and writing all support functions for both. This is actually an expensive proposition (if not in money, in manpower and training) which can make NSIS a liability for some organizations and projects. This could potentially affect NSIS's popularity as XP dies off and people stop hating UAC, while software also continues its trend of growing larger.
I really hope this post doesn't come off as bashing NSIS or kichik or anyone. I wouldn't still be using NSIS after all these years if I didn't feel it was the appropriate tool. I simply want to be able to keep using it in the future, without being pushed away to another product simply due to user perception. 🙂
A quick poke around shows this to be a much-reported issue on the tracker. As far as I can tell, kichik marks them all as dupes, with a trail leading to a bug for making multi-archive installers. (Which, really, is the polar opposite of what is desired here.) Hopefully, he won't do so with [url=http://sourceforge.net/tracker/?func=detail&aid=2785629&group_id=22049&atid=373085]my latest bug[url]. 😎
As I mentioned in the bug, this makes NSIS unusable for many use cases where a single-file installer is necessary. Which means not only having the same program with an NSIS multi-part installer and a "solid" installer in another system (or just skipping the NSIS version altogether...), but having to keep people trained on both and writing all support functions for both. This is actually an expensive proposition (if not in money, in manpower and training) which can make NSIS a liability for some organizations and projects. This could potentially affect NSIS's popularity as XP dies off and people stop hating UAC, while software also continues its trend of growing larger.
I really hope this post doesn't come off as bashing NSIS or kichik or anyone. I wouldn't still be using NSIS after all these years if I didn't feel it was the appropriate tool. I simply want to be able to keep using it in the future, without being pushed away to another product simply due to user perception. 🙂
Extremely slow installer startup on Vista
Blame Vista, not NSIS!
It seems that Vista scans the whole exe if it requests admin privileges (proper Authenticode signature does not help). Last year I had to make a big (1.2GB) patch for a game and had the same problem. Here is the solution:
1. The installer DOES NOT request admin level.
2. The installer silently copies a small launcher to the temp dir, executes it and quits.
3. The launcher requests admin level (UAC::RunElevated). Vista scans it quickly, as it's very small.
4. The launcher launches the main installer (again), this time use /NOCRC to save time. The installer inherits admin privileges granted to launcher, but Vista does not scan it, so it launches immediately.
The whole trick takes almost no time (just unpacking 75 kilobytes).
Notes:
- Obviously, the installer must "know" if it's launched by the user or by the launcher, and the launcher must know the path of the installer. Hint: commandline arguments ;-).
- Both installer and launcher should be signed, otherwise UAC will display 3 ugly warnings before the installer GUI is shown.
- To be precise: I used UAC plugin to elevate user rights (both the installer and the launcher request USER execution level). It's wise to test if UAC::RunElevated worked (should return 1223) before starting the actual installation.
Blame Vista, not NSIS!
It seems that Vista scans the whole exe if it requests admin privileges (proper Authenticode signature does not help). Last year I had to make a big (1.2GB) patch for a game and had the same problem. Here is the solution:
1. The installer DOES NOT request admin level.
2. The installer silently copies a small launcher to the temp dir, executes it and quits.
3. The launcher requests admin level (UAC::RunElevated). Vista scans it quickly, as it's very small.
4. The launcher launches the main installer (again), this time use /NOCRC to save time. The installer inherits admin privileges granted to launcher, but Vista does not scan it, so it launches immediately.
The whole trick takes almost no time (just unpacking 75 kilobytes).
Notes:
- Obviously, the installer must "know" if it's launched by the user or by the launcher, and the launcher must know the path of the installer. Hint: commandline arguments ;-).
- Both installer and launcher should be signed, otherwise UAC will display 3 ugly warnings before the installer GUI is shown.
- To be precise: I used UAC plugin to elevate user rights (both the installer and the launcher request USER execution level). It's wise to test if UAC::RunElevated worked (should return 1223) before starting the actual installation.
Wow, I am curious to try that approach. But I'm afraid to see the process tree as that starts up. 😁
And, I'm not sure one can claim it's really Vista's fault... It seems that if you have a Vista manifest on an app requesting elevation, you don't see this behavior. But, if it only has an XP manifest, you do. And, NSIS only has an XP manifest. 🙁 It seems some other install systems now use Vista (maybe even Windows 7) manifests, so they avoid the "slow startup" issue.
I haven't updated the bug with this information, as I'm still verifying it. (Yay huge drives, for things like this, eh?) But, hopefully, I'll be able to post some source soon to demonstrate this.
And, I'm not sure one can claim it's really Vista's fault... It seems that if you have a Vista manifest on an app requesting elevation, you don't see this behavior. But, if it only has an XP manifest, you do. And, NSIS only has an XP manifest. 🙁 It seems some other install systems now use Vista (maybe even Windows 7) manifests, so they avoid the "slow startup" issue.
I haven't updated the bug with this information, as I'm still verifying it. (Yay huge drives, for things like this, eh?) But, hopefully, I'll be able to post some source soon to demonstrate this.
In the process tree you will see just 3 processes, and only if you are quick enough ;-)
I doubt if changing anything in the manifest would help (even Authenticode have not helped, and anyone can write anything in the manifest).
But I suppose your idea can be tested by simply changing the manifest with a resource editor.
I am curious myself but I don't have time to test it now (not to mention I don't know much about the manifests :-) ).
I doubt if changing anything in the manifest would help (even Authenticode have not helped, and anyone can write anything in the manifest).
But I suppose your idea can be tested by simply changing the manifest with a resource editor.
I am curious myself but I don't have time to test it now (not to mention I don't know much about the manifests :-) ).
Well, the bug tracker has a patch for an updated manifest. Perhaps I'll apply that and rebuild in a few hours to do some testing. 🙂
It'd be kinda neat if that one patch could kill both bugs.
It'd be kinda neat if that one patch could kill both bugs.
AFAIK, that Vista manifest GUID was added in Win7, Vista itself does not know about it
@Xutzl: UAC::RunElevated returning 1223 means user pressed cancel IIRC, only 0 means success
@Xutzl: UAC::RunElevated returning 1223 means user pressed cancel IIRC, only 0 means success
Hm, I guess that means it's time to look closer at what the large non-NSIS installers that don't have this issue are doing. :P
MS could be looking for nsis installers and thinking they need this, but why I don't know. I don't understand the point of running it from $temp. Maybe you could ask on the MSDN forum
My experience with the MSDN forum is unpleasant. So, I think I'll first see if I can figure out what's different between NSIS and other things. 🙂
And I'm already fairly sure it sees NSIS and does "special" stuff. I'm not entirely sure what signature it's looking for though, yet. :P I'd like to defeat that check, to see if it changes things. 🙂
And I'm already fairly sure it sees NSIS and does "special" stuff. I'm not entirely sure what signature it's looking for though, yet. :P I'd like to defeat that check, to see if it changes things. 🙂
Vista may recognize installers (NSIS, WISE etc.) without proper manifest and grant them admin rights (with the usual UAC warning window), but I doubt it does something nasty just because a file is compiled with NSIS.
Besides, I have seen also a Wise installer suffering the same long start issue (but it was created with some older version of Wise, I mean before it supported Windows Installer).
Edit: More here, search for "Installer Detection Technology"
Besides, I have seen also a Wise installer suffering the same long start issue (but it was created with some older version of Wise, I mean before it supported Windows Installer).
Edit: More here, search for "Installer Detection Technology"
http://technet.microsoft.com/en-us/library/cc709628.aspxAnyway, big files requesting only USER rights launch fast on Vista. Files that inherit admin rights from their parents also start quickly (like on XP), that's why my "trick" works.
So... It looks like I've managed to "solve" the issue of the slow startup... Apparently Vista is seeing the name of the exehead in the manifest and applying this special-for-NSIS behavior.
Simply changing <assemblyIdentity name=> to anything else did the trick. So, my test installers currently have manifests where it is set to "Nullsoft.NSIS.win32.exehead". Et voila, instant elevation prompt.
I've not done extensive testing yet to see what compatibility stuff is now being "turned off" by Vista, but I've got a sneaking suspicion that whatever this was working around is probably fixed. If not, it will now be visible (hopefully) enough to fix. 🙂
Can anyone else verify that a simple manifest update does the trick for them? (Use RequestExecutionLevel, not the UAC plugin for testing this, btw. This causes you to need /NCRC on the installer.) If so, I'll update the bug and write a script to update my manifests for now.
Simply changing <assemblyIdentity name=> to anything else did the trick. So, my test installers currently have manifests where it is set to "Nullsoft.NSIS.win32.exehead". Et voila, instant elevation prompt.
I've not done extensive testing yet to see what compatibility stuff is now being "turned off" by Vista, but I've got a sneaking suspicion that whatever this was working around is probably fixed. If not, it will now be visible (hopefully) enough to fix. 🙂
Can anyone else verify that a simple manifest update does the trick for them? (Use RequestExecutionLevel, not the UAC plugin for testing this, btw. This causes you to need /NCRC on the installer.) If so, I'll update the bug and write a script to update my manifests for now.
Originally posted by XutzlI meant to reply to this the other day and forgot.
Edit: More here, search for "Installer Detection Technology"
From the URL you Linked to:
Installer Detection only applies to:#1 and #3 certainly apply. #2 does not, in our case. So, it's not this particular technology being triggered. And, as I said, I've not seen this with any other modern installers.
1. 32 bit executables
2. Applications without a requestedExecutionLevel
3. Interactive processes running as a Standard User with LUA enabled
So, something needs to be done on NSIS's side, probably. As my previous post shows, embedding a slightly different manifest appears to do the trick. I'm currently rebuilding my NSIS sources with a slight change to see if I can submit a patch (likely to be rejected in current form, but it'll be on the right track, I hope) to make these large installers work better. Let me just tell you that on a 1.2 GB installer, it's the difference between night and day.
(If this is true)
I fucking hate MS
Probably some installer out there that they tested required running from $temp for whatever insane reason and they decided that this was a smart thing to do for every NSIS installer without even filing a bug report or a simple forum post
If a manifest change is required, I guess that is not too bad, but what happens during Win8 testing and they add a *NSIS* check?
I fucking hate MS
Probably some installer out there that they tested required running from $temp for whatever insane reason and they decided that this was a smart thing to do for every NSIS installer without even filing a bug report or a simple forum post
If a manifest change is required, I guess that is not too bad, but what happens during Win8 testing and they add a *NSIS* check?
Originally posted by lewellynThat's great 😁. I wish I knew that last year (I wouldn't have to make up my weird way of getting admin rights...).
As my previous post shows, embedding a slightly different manifest appears to do the trick.
Could you post your full manifest here? Then anyone struggling with the problem will be able to change the manifest in their installers without waiting for fixed NSIS.