random system.dll crash on installer shutdown

Animaether
26th February 2009 23:42 UTC

random system.dll crash on installer shutdown
Hi there,

I'm getting a random crash from system.dll ; I realize that this 'could be anything', but maybe you guys have some thoughts.

Basically system.dll is used to create buffers, structs, make API calls, etc. in various places. As far as I can tell, all of the code is 'safe' - i.e. buffers get cleaned up where they should be.. but I'm still not an expert.

The crash only occurs when the installer shuts down, and is highly intermittent. In fact, I'm trying to see if I can get some details on the crash but I can't get the thing to crash now.

Is there something happening at installer shutdown where, presumably, system.dll gets unloaded or so and something I've forgotten could cause these crashes?

If you need code, I'll try and share as much as I can, but the all of the system calls used are the below (with minor modifications; e.g. Get_Local_Time adjusted to return a specific format instead of the hours/minutes/etc. on the stack; no changes to the system calls themselves).

http://nsis.sourceforge.net/Get_Local_Time
http://nsis.sourceforge.net/Refresh_shell_icons
http://nsis.sourceforge.net/Dump_log_to_file
http://nsis.sourceforge.net/NsDialogs_SetImageOLE
http://nsis.sourceforge.net/Allow_on...aller_instance

And...
Refresh a control:

System::Call "user32::InvalidateRect(i,i,i)i (r0, 0, 1)"

Sadly I can't rule any of these out too easily given that the crash *is* intermittent.

Thanks in advance for any pointers.

Anders
27th February 2009 00:06 UTC

I would suggest running the installer in a loop (batchfile or whatever) and comment things out untill it stops crashing (the installer could write a special file to $temp at the end, then the batchfile could check for this file, it it exists, it just runs it again, it its not there, there was a crash etc)

Animaether
27th February 2009 00:38 UTC

yeah, I've had to move on to making the installer do what it's supposed to do (when not crashing; which has been the last 30 manual runs; I hate heisenbugs), after which I'm likely to do something like that; I hope that there's no conceivable way that timing of the installer closing could have something to do with it.. e.g. a user closing the installer after 1.5s > crash, batch closing it after 0.000xs > no crash, etc.

Animaether
2nd March 2009 13:37 UTC

Well, never got the crashes again (set up a small scripted test), but I'll be keeping an eye out. For now I'll just rule this as Windows having been in a dirty state from some earlier tests, although I'm not sure how system.dll would be affected .

folklore
2nd March 2009 14:01 UTC

Originally posted by Animaether
Well, never got the crashes again (set up a small scripted test), but I'll be keeping an eye out. For now I'll just rule this as Windows having been in a dirty state from some earlier tests, although I'm not sure how system.dll would be affected .

The crash only occurs when the installer shuts down, and is highly intermittent. In fact, I'm trying to see if I can get some details on the crash but I can't get the thing to crash now.

It allways means that there are a memory error:check if you allocated a memory resource and then forget to free it.Or access the memory without a allocation .The installer's script is build in a language like assembly.and I sugest that if you make a stack error:I means thant the stack if out of order.It will be lead you to failure at shudown time.
All above,It like a memory error.Good luck.

Animaether
2nd March 2009 14:25 UTC

well, that's the thing... as far as I can tell, the NSIS stack is always clean by the end of a function call / macro / etc (I always code them with comments that indicate what the stack should look like at a specific line of the code, so that I always know where I should be popping things off and into what variables etc.).

I don't think the System plugin itself cares much about the NSIS stack, short of using it for input / output, but none of the system calls I use grab things from / place things on the stack (without popping right after), that I can tell - using the variables ($0-$9,$R0-$R9) instead...

Animaether
3rd March 2009 16:02 UTC

update: found the culprit

Although it never seemed to crash again in the main installer, a derived (much smaller) installer once again showed the crash. The problem was that the finish page was based off of the welcome page. The welcome page uses the OLE image function to set a JPG file to a Bitmap control. To do so, a buffer has to be created for the image data, which must be cleared once done with it.

However, the finish page did not set this image.. but it was still clearing the buffer which was presumed to be in $0. The value of $0 could, of course, be just about anything at the time of installer end and the fact that I never saw the crash again is more likely to be 'luck' than anything else. The smaller installer always had $0 set to a value of 0, 1, 2 or 3 (depending on some test results) which I'm guessing the call in question would -always- cause to throw an error later.

Looks like this was a case of not seeing the forest for the trees; focusing on the stack and the integrity of the System calls themselves and completely missing the fact that System was called where (on something) it never should have been, no matter the integrity of the call itself.