rtg
6th April 2001 15:17 UTC

I'm trying to basically create a very simple installer, similar to a self extracting zip. The contents are the contents of a cdrom (subdirectories, installshield installers, cab files, .msi's, etc).

I'm setting SetCompress to off, but during compile NSIS still says compressing for each file (using version 1.2c). When it reaches certain files like .cab or other highly compressed files, it chews up about 200mb of RAM and actually ends up increasing the file sizes. The source files were added doing a File /r.

(Insert feature request here: I can't seem to add a root directory, like File /r F: or F:\.. It wants me to have a subdir like File /r F:\files, not even File /r F:\files\.

Insert second feature request: When the source files aren't found, can you emit an error, rather than just saying compile completed?)


Thanks in advance for any help you guys can provide!!
Richard

justin
7th April 2001 23:09 UTC

There is a current limitation in the installer compiler, in that it needs enough memory for about 1.5x what you are compiling. For example, if you need to make a 300mb installer, you really need at least 300-400mb of ram.

This limitation could be fixed pretty easily, by making add_data_misc add to a temp file, as opposed to memory buffer (and so on). If there is enough demand for it, I
will add it.

As far as the files growing, the most that they will grow is by 4 bytes/file.

-Justin

rtg
8th April 2001 00:11 UTC

Bear with me, I'm still a bit confused! It was my impression that it was still trying to compress, despite the fact that I had turned compression off. I thought the compression algorithm, when run against already compressed files, was what was chewing up the memory...
The only time the memory raises is in "spikes" when it hits those files, rather than just slowly growing over time. It results in the compile time for the 300mb running at about 2hrs so far. The server has 1GB physical ram.

Is it a bug, something I'm doing, or am I in left field? And is it supposed to say compressing then adding, even when SetCompress is off?

Several people at work have already switched from Installshield since I showed them NSIS, btw!

Thanks!

justin
9th April 2001 09:11 UTC

Originally posted by rtg
I'm trying to basically create a very simple installer, similar to a self extracting zip. The contents are the contents of a cdrom (subdirectories, installshield installers, cab files, .msi's, etc).

I'm setting SetCompress to off, but during compile NSIS still says compressing for each file (using version 1.2c). When it reaches certain files like .cab or other highly compressed files, it chews up about 200mb of RAM and actually ends up increasing the file sizes. The source files were added doing a File /r.

(Insert feature request here: I can't seem to add a root directory, like File /r F: or F:\.. It wants me to have a subdir like File /r F:\files, not even File /r F:\files\.

Insert second feature request: When the source files aren't found, can you emit an error, rather than just saying compile completed?)


Thanks in advance for any help you guys can provide!!
Richard

rtg
11th April 2001 21:19 UTC

Okay Justin, I swore I was gonna drop this, but I think there really is a bug..

It still took a loooong time(overnight) to compile. The data size is approx 250mb or so.. here's my .nsi file:
--------------------------------------------
; example1.nsi
;
; This script is perhaps one of the simplest NSIs you can make. All of the
; optional settings are left to their default settings. The instalelr simply
; prompts the user asking them where to install, and drops of notepad.exe
; there. If your Windows directory is not C:\windows, change it below.
;

; The name of the installer
Name "Package"

LicenseText ""

; The file to write
OutFile "e:\idrive\output.exe"

; The default installation directory
InstallDir $EXEDIR

; The text to prompt the user to enter a directory
DirText ""

; The stuff to install
Section -Default
; Set output path to the installation directory.
SetOutPath $INSTDIR
; Put file there
SetCompress off
File /r e:\*.*
; eof
-------------------------------------------------------------------------------------





Here's the output of the compile:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Install: 1 section (1 required).
Install: 1020 instructions, 6770 byte string table.
Datablock optimizer saved a whopping 109952062 bytes.

EXE header size: 32768 / 32768 bytes
Install code+strings: 35470 / 35470 bytes
Install data: 197490595 / 307442657 bytes
CRC (0x56E7A3C6): 4 / 4 bytes
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Total size: 36868 / 307510899 bytes

succeeded
MakeNSIS done - hit enter to close...


-----------------------------------------------------------

Note the fact that it created a 37k file! Not a couple hundred megs....Also, the datablock optimizer saved over 100mbytes?! I checked the file, it is definitely 37k.. When run, it reports "installer too small". The only possibility is that i had too little disk space, but I don't believe this is true..In any case it reported complete successfully.
Your new version did seem to run a bit faster, but still slowed down on many of the same files as before, and ate up the same amount of memory..

Should I give up?
Thanks in advance,
Richard

rainwater
12th April 2001 06:37 UTC

I noticed you had e:\idrive. You aren't trying to create the installer using a network drive are you? If so, that could be your problem because it shouldnt take that long for 250 megs. I created an install of about 200 megs and it took about 15 minutes. And I have 64 megs of physical RAM.

rtg
12th April 2001 14:19 UTC

Idrive is a shortened form of installer drive, it's basically a blank drive that I was using for scratch work :-) But good call on the network, I tried it that way when I first tried NSIS and, sadly, it still takes just as long when doing it on a local drive.. Longer even...

I'm still not sure about that 37k bug.. seems very weird... I'm going to run it yet again today and see what happens... I hate to bug people when they give you software for free!

Richard

Bor
12th April 2001 14:27 UTC

Change the output Dir
Try this, it may work.
change the outpath of the file from e:\idrive top say c:\idrive
as long as it is on another disk or partition.
I'm not sure if it will work and why I think it can solve the problem, but after your problem is solved then I will know the answer why it previously didn't work. (yes, it is wierd)

Bor

rtg
12th April 2001 14:37 UTC

Okay, I forgot I had a compile running overnight.. Here are the new statistics:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Install: 1 section (1 required).
Install: 1020 instructions, 6770 byte string table.
Datablock optimizer saved a whopping 109952062 bytes.

EXE header size: 32768 / 32768 bytes
Install code+strings: 35470 / 35470 bytes
Install data: 197490595 / 307442657 bytes
CRC (0x56E7A3C6): 4 / 4 bytes
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Total size: 197558837 / 307510899 bytes

succeeded
MakeNSIS done - hit enter to close...


-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--=-=-=-=-=--=-=--=-

After I freed up some disk space, it seemed to work better ;-) (feature req: report error when out of diskspace, rather than "successful")

It STILL took overnight, and as you can see it still compressed heavily, over 100mbytes, where I know the data on the CD was already fairly compressed as is.

So, I guess I'm back to my original quandary, which is: why does it insist on compressing? And/or, is that datablock optimizer soooo good that it can save that much space? And if so, why does it take me 5 hours and rainwater far less?

Thanks,
Richard

rtg
12th April 2001 14:38 UTC

Bor, thanks for the suggestion.. Just tried it, same result so far..

rtg
13th April 2001 02:21 UTC

Okay, thanks to an email Bor sent, I tried to package it with compression set to auto.. It went FAAAST! Like 5 minutes! The only thing I noticed was that some files would read like 0/353434 after finished...I hope that means it just didn't try to compress them.

So, tried it again, just changing compression to off, old problem returns! As Bor suggests to me, the memory management when compression is off is screwed up somehow..

Hopefully this thread has been of some use to the Nullsoft guys!

Richard

Bor
13th April 2001 22:44 UTC

Yes, It is true.

I tested it with the FreeDB database (http://www.freedb.org)
and with the install cd of Windows 2000 and with the install cd of Office 2000.
All three where majorly slow with compression turned off, and the memory usage went sky high.
But with compression set to auto it compressed fast, no problems at all.

Bor

rtg
14th April 2001 00:07 UTC

I'm still curious about those 0/whatever numbers. If Bor didn't see them, I wonder if I'm seeing another bug..

Guilty admission: I added a BGBG to my installer but quickly removed it ;) Thanks 1.2f!