makeNSIS fails on big file

Archive: makeNSIS fails on big file

Wizou
18th April 2008 14:18 UTC

makeNSIS fails on big file
Hello.. I have a problem on big files with NSIS 2.36 :

SetCompressor lzma

Section "Main"
File "bigfile.dat"
SectionEnd

bigfile is 1 740 336 643 bytes long

makeNSIS either crashes or display the following error:

File: "bigfile.dat" [compress]Error: add_db_data() - compress() failed(input/output error [-4])

Wizou
18th April 2008 15:42 UTC

when using bzip2 ou zlib, it always crashes

Wizou
18th April 2008 18:14 UTC

From my investigation using Process Explorer, for each file, makensis.exe tries to map in memory :
- the file to compress
- a temporary file which size is 1.25 times the size of the file to compress (probably a buffer for compression)

Now I can see the memory address space of makensis has a free space between ctype.nls (0x890000+0x3000) and lpk.dll (0x62DC0000) under Windows XP SP2

This free memory space is 1 649 594 368 bytes long, so the biggest "compressable" file should be about 733 151 232 bytes long

Do you confirm my analysis ?

kichik
18th April 2008 18:21 UTC

NSIS doesn't support files as large as that. There are solutions on the Wiki like CABSetup.

Wizou
21st April 2008 10:50 UTC

:(
I don't want to extract a CAB file on the user harddisk and then uncompress it to the installation target folder, (that would require a lot of free space on disk), when it seems possible to directly uncompress from the installer to the target folder. Plus I need a one-file installer (can't span on multiple installer files).

Besides, NSIS worked perfectly with big file up to 1.52 GB but crashes with a file 1.62 GB long

(forget the 700 MB limit from my previous analysis)

Wizou
21st April 2008 17:10 UTC

ok I've analyzed NSIS sources and I've come to the conclusion that the biggest file (if alone) that NSIS can currently support should be under (about) 1 704 564 322 bytes = 1.58 GB

formula= (0x7FFFFFFF-4-1024-(16<<20))/5*4

Wizou
22nd April 2008 10:03 UTC

kichik, you could add support for files up to 2GB simply by adding the following lines in build.cpp in function CEXEBuild::add_db_data

    int bufferlen = length + 1024 + length / 4; // give a nice 25% extra space
    if (bufferlen < 0)          // we've hit a signed integer overflow (file is over 1.6 GB)
        bufferlen = INT_MAX-st-sizeof(int)-(16 << 20); //  so maximize compressor room and hope the file compresses well

as commented, it should work correctly as long as the big file to compress do compress well, under the room allocated for compression

kichik
22nd April 2008 10:42 UTC

Please submit it as a patch.

evin
22nd April 2008 11:24 UTC

Use the Vpatch. (NSIS\Bin\GenPat.exe)

Wizou
22nd April 2008 13:16 UTC

done.

Vpatch doesn't apply for a first installation

However I did develop a WPatch, more powerful for patching files *in-place* without requiring more free space on disk than the size difference
I plan to release it soon