"Out of memory';." message

sebarnolds
12th February 2007 15:47 UTC

"Out of memory';." message
Hi.

I'm using a custom program I wrote to generate a NSH file which contains instructions to install all the files of a directory. These instructions include check for file existence, backup if needed and add to log. So there are several script lines for each file. The script is later included by my installer script.

Here is my problem : I try to build an installer for a directory which contains 11821 files and 872 folders. The resulting script file takes almost 11Mo on my hard drive ! Then when I try to compile my script to build an installer, through HM NIS Edit 2.0.3 it reports me an "Out of memory';." error message halfway through.

Anybody has any idea about that and how I could solve the problem ?

I could zip everything and use a custom plugin I wrote to do that but it would be a bit messy as the progress bar wouldn't update while the unzip/check/backup/log process.

Thanks,
Sebastien

Red Wine
12th February 2007 17:59 UTC

Installers can be as large as 2GB

The resulting script file takes almost 11Mo on my hard drive !

11 mb if I got it right is too big for a script I guess.

I've made and successfully compiled a complete script with NSIS Quick Setup Script Generator which contains almost 4000 files and 500 directories (added both in install and uninstall section assumes 8000 files and 1000 directories) and the script.nsi is about 700 kb.

sebarnolds
13th February 2007 08:20 UTC

Hi.

Thanks for your answer.

The following code is replicated for each file:

File "Installation Files\\PROG\FILE_NAME"

>PLUGIN::checkFilesDates "$0\\FILE_NAME" "$INSTDIR\\FILE_NAME"

>Pop $R0

StrCmp $R0"1" +8

StrCmp $R0"0" +5

IfFileExists"$INSTDIR\\Backups\\FILE_NAME" +4

Rename "$INSTDIR\\FILE_NAME" "$INSTDIR\\Backups\\FILE_NAME"

>FileWrite $1 "b,$INSTDIR\\FILE_NAME$\\r$\\n"

>GoTo +2

FileWrite$1 "f,$INSTDIR\\FILE_NAME$\\r$\\n"

>IfFileExists "$INSTDIR\\FILE_NAME" 0 +2

Delete "$INSTDIR\\FILE_NAME"

>Rename "$0\\FILE_NAME" "$INSTDIR\\FILE_NAME"

>GoTo +2

Delete "$0\\FILE_NAME"

So all these lines duplicated for 11821 files make a script file of 11Mo. I need this code because it checks for file existence, backup it if needed (and if no backup is already present), log the operation and the copy it to its destination. I guess I'll need to do this from my plugin after having extracted the directory content from a zip file.

Anyway, thanks. If anybody has an idea, don't hesitate.

Sebastien

onad
13th February 2007 14:38 UTC

IDEA:

Use a separate INI file in a ZIP which contains all file names, then USE NSIS plugin to unzip the INI to HD temp and read the INI line by line to test for file existence.

Be creative ;)

{_trueparuex^}
13th February 2007 15:29 UTC

Have you simply tried to compile the script with just makensis.exe?

I remember when I tried to compile very large script with HM NIS Edit, I got the same error. But when I compiled the script with just makensis it worked.

I think the 'Out of memory' error occurs simply because the length of NSIS output in the HM NIS Edit console exceed certain length.

So you can probably compile the script with HM NIS Edit simply by adding this '!verbose 3' in the beginning of your script. I will reduce the output from NSIS. :)

sebarnolds
13th February 2007 16:51 UTC

Yeah, that's a good idea. Now that you say it, I remember that the lat output of the editor was quite weird. I should have thought about it.

Anyway, I tried your suggestion "!verbose 3" and it seems to correct the problem. Now there is the problem that I need more than one hour (it is still running and it has for a long time) to build it :-)

I guess I will stick to the zip version, it is not as clean but a lot faster.

Thanks a lot for your answers and your suggestions.

Sebastien

Boyito
13th February 2007 17:28 UTC

HI the size of my installer is 60Mb and I dont have any problem.

{_trueparuex^}
13th February 2007 20:12 UTC

Originally posted by sebarnolds
Anyway, I tried your suggestion "!verbose 3" and it seems to correct the problem. Now there is the problem that I need more than one hour (it is still running and it has for a long time) to build it :-)

I guess I will stick to the zip version, it is not as clean but a lot faster.

Great. :) If you are using solid compression you could try non-solid (E.g non-solid zlib). It should speed things up significantly, but if your zip is already bigger than 2GB, don't bother. ;)

@Boyito
??? ... lol

sebarnolds
14th February 2007 08:01 UTC

Hi and thanks for your suggestions.

I left the compiler when I left yesterday and it has finished the compilation process. From the modification time of the generated installer I can say that it took about 1 hour, 1 hour and a half to compile it.

The problem is not the size of the installer (~86Mo). As I said, there are a lot of files (more than 10000) and a few lines for each file results in a 10Mo script file. From the last log lines of the compiler, I can see that there are 266289 instructions in total. So I suppose I understand the tile to compile all of this :-)

I will try your suggestions (non-solid compression) but I don't think it should help.

Thanks,
Sebastien

sebarnolds
14th February 2007 09:29 UTC

Hi again !

I was using bzip2 compression, not solid. I tried zip compression, supposed to be faster, but it took it exactly one hour to build the installer. Anyway, it seems it is a bit overwhelmed by the huge amount of lines of code in the script.

I will revert to my custom zip solution : extracting my zip file to a temp directory and then doing all the file check, backup and log from my plugin. I will have a progress bar stalling during that operation but it will be much faster (building the installer and running it).

Anyway, thanks for all your help.

Sebastien