Yathosho
21st September 2011 16:53 UTC

Debugging advice
i'm suffering over a problem occuring with my script and i'm having a difficult time pinpointing it. what this script does is locating and copying some files and using textreplace. in most cases it works without problems, but when a large number of files is involved (in my case 2530) the installer will freeze. to make things worse this does not happen always, just occasionally.

i'm looking for some hints on how to find the bug. i've already disabled various parts of the script, but without gaining any knowledge. all i can say is, this happens when working with a lot of files and i believe i did not have this problem when using other (slower) functions to replace text or to locate files. is there anything wrong with my textreplace calls? can they be optimized? any other tips on how to narrowing down the problem?

any help is greatly appreciated.

{_trueparuex^}
21st September 2011 20:44 UTC

In cases like this I usually just add some messgeboxes to the code to find the point where things go wrong, but in this case Nsisdbg plug-in is probably much better choice.

With Nsisdbg plug-in use the nsisdbg::sendtolog call to track the progress of your script.

Edit: I just realized that the Nsisdbg plug-in download link is is dead. I uploaded the plug-in on my site.

Yathosho
22nd September 2011 08:25 UTC

i shall also note that i only get those problems on windows 7 (both 32 and 64 bit), not on windows xp.

Yathosho
22nd September 2011 17:02 UTC

thanks {_trueparuex^}, i actually tried all of this before - to no success. as i suspect the number of files to be responsible, i wonder if this could be a memory issue. i do have 8gb ram and enough space on all drives, so that should not be the case. i'm also not exceeding the string-length (i'm using the large string build), so i wonder where else i can look. what puzzles me, why does it work on xp in a virtual machine (with 2gb ram)

Yathosho
30th September 2011 10:24 UTC

it appears i could find the problem in my script, so i wanted to resolve it in case someone runs into similar problems.

the problem appeared (occasionally, only on windows vista or later) when setting the outdir when the path existed already

SetOutPath "$MyTempDir\fonts"

IfFileExists "$MyTempDir\fonts\*.*" +2
CreateDirectory "$MyTempDir\fonts"

Yathosho
30th September 2011 20:03 UTC

i continue to have this bug at a later point. again it's caused by a simple copy operation, just this time i can't figure out when and why it's happening (again it happens occasionally and only on vista/win7.) i have a test scenario which helps me testing this on 312 examples, but the result continue to be entirely random, though certain test-files tend to trigger the bug more often.

the line in question is this:

CopyFiles /SILENT "$EXEDIR\ape\addborder.ape" "$MyTempDir\ape\addborder.ape"

Yathosho
1st October 2011 19:01 UTC

here are a couple of things i tried

1. extract the file from the installer rather than copying it -> works
2. copying the file using the full path as on my system -> works
3. checking if the source-file and the target-folder exist -> yes
4. checking if $EXEDIR is correct -> yes

$EXEDIR = C:\Program Files (x86)\whyEye.org\PimpBot\ape

CopyFile "$EXEDIR\ape\addborder.ape" "$MyTempDir\ape\addborder" -> crashes installer
CopyFile "C:\Program Files (x86)\whyEye.org\PimpBot\ape\addborder.ape" "$MyTempDir\ape\addborder" -> works

i'm clueless, please please help me!

MSG
1st October 2011 20:45 UTC

What if you call API directly, instead of CopyFiles?

Yathosho
1st October 2011 22:22 UTC

i tried StdUtils, which also results in a crash

MSG
2nd October 2011 07:46 UTC

Just a random thought: Have you tried this on other PCs? Win Vista/7 are very clunky with file copying. It's possible that bad RAM or a bad HDD causes it to blow.

Yathosho
2nd October 2011 09:44 UTC

i asked some friends to test it, but they didn't get any problems (well, i'd love to test it myself). i replaced my ram since this first came up and i tried various disks including ssd, harddrives and ramdisk (hence i'm using $MyTempDir instead of $PLUGINSDIR). i have another windows 7 (32-bit) running in a virtual machine (same problem) and also windows xp (where i NEVER get this problem.) i'm going to install windows 7 on vmware on my laptop and see what happens..

further, i've tested various nsis builds (standard and longstr)

it's interesting that there have been reports regarding CopyFiles and windows 7 on this forum, though i didn't find a case that really crashes the installer.

having found the exact line where this problem occurs, i wonder if it could still be caused by something else. not sure what i should be looking for.

Yathosho
2nd October 2011 11:35 UTC

same thing on the other computer :(

MSG
3rd October 2011 05:51 UTC

Originally posted by Yathosho
$EXEDIR = C:\Program Files (x86)\whyEye.org\PimpBot\ape

CopyFile "$EXEDIR\ape\addborder.ape" "$MyTempDir\ape\addborder" -> crashes installer
CopyFile "C:\Program Files (x86)\whyEye.org\PimpBot\ape\addborder.ape" "$MyTempDir\ape\addborder" -> works

Yathosho
3rd October 2011 09:07 UTC

Originally posted by MSG
It shouldn't lead to a crash, but if $EXEDIR already has a trailing 'ape', your copyfile command is wrong.

{_trueparuex^}
3rd October 2011 20:06 UTC

I think you have a heap corruption issue here. The crash.log shows it crashes in ntdll.dll with memory access violation which I think is probably result of heap corruption. If this is the case you probably also get crashes with exception code 0xc0000374 (STATUS_HEAP_CORRUPTION). And the cause for heap corruption would be most likely a third party plug-in. :blah:

PaR

Yathosho
3rd October 2011 22:09 UTC

Originally posted by {_trueparuex^}
I think you have a heap corruption issue here. The crash.log shows it crashes in ntdll.dll with memory access violation which I think is probably result of heap corruption. If this is the case you probably also get crashes with exception code 0xc0000374 (STATUS_HEAP_CORRUPTION). And the cause for heap corruption would be most likely a third party plug-in. :blah:

PaR

Yathosho
5th October 2011 13:06 UTC

i commented out the two plugins in question each at a time (TextReplace and Locate) and it appears that either the locate plugin is buggy or my code using it produces the bug (will look into it!) in either case, the installer didn't crash once in 2,500 test runs.

the only let down is this: what happens if it's to blame on the plugin. it appears it's no longer maintained by the author :(

DrO
5th October 2011 13:18 UTC

Originally posted by Yathosho
the only let down is this: what happens if it's to blame on the plugin. it appears it's no longer maintained by the author :(

Yathosho
5th October 2011 13:36 UTC

Originally posted by DrO
as the source is provided, you'd need to find someone to update / fix it for you or learn how to do it :)

DrO
5th October 2011 13:37 UTC

something like that though i'd prefer a milkdrop 3 which supports avs presets :p

-daz