Mr Inches
28th February 2006 09:17 UTC

Multi-Volume NSIS Installer Suite - CABSetup
I read a number of threads about creating NSIS Installers that supported multiple source media e.g. an installer that spans multiple floppy disks or multiple CDs/DVDs, but there didn't seem to be an bundled solution to it and I know it is on the wish list for NSIS. I felt that this is a capability that NSIS should have.

I spent the last month developing CABSetup, a plugin that creates multi-volume data sets and can extract the contents of multiple source media. The plugin leverages part of the operating system that Microsoft uses to support the Windows Installer to do the bulk of the work.

Included in the suite is InstStub, a program that can launch another executable and delete it and itself when both processes complete. Using InstStub, the core NSIS installer can now be copied to the temporary directory on the local hard drive and be deleted afterwards, thereby leaving no files on the user's computer after the install completes.

Instructions on how to create install sets using the suite are included in the CABSetup.txt file included in the package.

I have tested the suite on Windows NT 4.0, Windows 2000 and Windows XP SP2, with both floppy disks and CDs and believe it is quite useable. Of course, I have my wish list too - I intend to improve the functionality in subsequent releases, but what is there is enough for a first release.

Please read the CABSetup.txt file in the attached file for further information.

Enjoy

Duncan (Mr Inches)

JasonFriday13
5th March 2006 21:39 UTC

I suggest putting this on the wiki so that we don't have to search the forums if we want to use it (like plugins).

Mr Inches
13th March 2006 10:16 UTC

Took me a little while to figure out how to drive the NSIS Wiki pages :)

Posted to Plugins section here:

http://nsis.sourceforge.net/CABSetup_plug-in

Duncan (Mr Inches)

JasonFriday13
10th May 2006 00:30 UTC

Just a simple idea: You could use nsis to make a wrapper for the real installer, instead of Inststub. I do this all the time.

Name "Wrapper"
OutFile "Setup.exe" ;or whatever name you want.

;we want the install to be silent.
SilentInstall silent

Section
;initialize the pluginsdir
InitPluginsDir
;set the path to the pluginsdir
SetOutPath "$PLUGINSDIR"
;this is where you include the real world installer,
;and extract it to the pluginsdir.
File "_setup.exe"
;run the setup with the path on the command line,
;and wait for it to finish.
ExeWait '"$PLUGINSDIR\_setup.exe" "$EXEDIR"'
;Pluginsdir will be deleted when this installer finishes.
SectionEnd

Mr Inches
18th May 2006 13:04 UTC

The wrapper doesn't solve the same problem that InstStub does.

Try putting one of the installers created by your wrapper onto removable media like a floppy, flash disk or CD and run it. Then remove the media and set what happens.

Chances are you either can't remove it politely (for flash disk) or the installer will complain when it next needs to access the executable because the original media is not around.

Using InstStub stops Windows holding files open on the source media that the installer was launched from and allows that media to be exchanged for another one.

Duncan (Mr Inches)

Anders
18th May 2006 15:30 UTC

Some nice code here, but does msiexec.exe really exist on all systems (I'm pretty sure win95/98 doesnt have it if you have never used a msi installer on it) why not use a more common .exe instead, like rundll32.exe(The best choice IMHO), explorer.exe, winhlp32.exe, winver.exe or notepad.exe

Mr Inches
18th May 2006 22:46 UTC

Thanks Anders, I figured the code looked pretty cool too when I saw Arkon's example.

I did not test the code all the way back to Win95/Win98. As noted in the code, the GetModuleFileNameEx call requires Windows NT or later and the PSAPI.DLL/LIB, so operation on those earlier platforms isn't available to me. Getting the filename for the child process is a pain without this call. (Unless someone knows a better way....?)

I did test InstStub on a fresh install of NT 4.0, it includes Windows Installer (msiexec.exe) out of the box.

I originally used notepad.exe as the host, but I guess I figured it looks more like an installer when I used msiexec.exe since the host process is what shows up in Task Manager (if you happen to look in it while the installer is running.) and having an extra copy of notepad.exe sitting there didn't look quite right. But yes, any executable that is known to reside on the system could serve as the host process.

JasonFriday13
19th May 2006 03:11 UTC

Ah yes, of course. That is why it is called CABSetup (multi volume), so that you could change install media. If I run a program off my flash disk, and try removing it, windows says it cannot be removed at this time. I forgot about that.

[edit]fixed typo[/edit]

Anders
19th May 2006 04:18 UTC

Yeah, I guess you need to use the ToolHelp helper lib on 9x

Mr Inches
19th May 2006 11:38 UTC

I guess so. I looked at the ToolHelp stuff when I was doing the PSAPI work - it might be doable.

I am looking at doing an update to CABSetup in the next week or so; I will see if I can update InstStub/ExitStub as well.

Duncan (Mr Inches)

JasonFriday13
1st June 2006 01:50 UTC

Suggestions:
1. Option to show the current file being extracted on the detailprint label and/or in the list (if not already done).

2. Add progressbar support (you're probably already doing this as we speak).

3. Option to display a custom dialog that looks like an MSI style loading bar dialog during the extraction process (with the features above).

4. Split the code in half and have a specialized makecabs utility to make the cabinets required. You could make the utility with NSIS and installoptions pages, and have the buildfiles half as a plugin to the utility. The setup would contain the other half of the plugin (ExtractFiles), which makes the plugin smaller for the setup program (less overhead). (I am actually thinking of doing this bit myself).

Your thoughts and comments are welcome.

[edit]Added: (if not already done)[/edit]

Mr Inches
1st June 2006 23:44 UTC

Thanks for the suggestions.

I will see what I can do, the DetailPrint thing should be simple, the others will require some work, but not impossible, especially since I should be able to leverage existing code.

I spent the last week making ExitStub work with Windows 98 (and hopefully Win95). Turns out it was no small undertaking. It takes more than 'just using the ToolHelper library' since one of the functions I used in BuildExitStub is only supported in NT and later, requiring that function to be rewritten.

Next delivery should be in next couple of days, I would suggest you don't try to split the code just yet, I have fixed some bugs that impact both modules.

Duncan

JasonFriday13
2nd June 2006 00:35 UTC

Oh, right.

For point 3 above, you could use the dialog and code from the nxs plugin (link).

I didn't say this in my last post, but I have a long term thought of integrating the source code of this plugin with the source of NSIS. It will probably never happen anytime soon, because I still need to learn the C/C++ language inside-out.

Call me crazy :igor:, but I might have a go at making a Nero Burning Rom plugin for NSIS (for burning distribution test cds).

Mr Inches
7th June 2006 12:32 UTC

I have released a new version of CABSetup, 1.1.0.0.

This release is mainly to address some bugs I found in the initial release, in particular:

- remove MSVC runtime dependencies i.e. need for MSVCRT7x.DLL.
- fixed bug/design issue that resulted in the NSIS stack being emptied when plugin functions were called.
- allow installer to be run on Windows 98. (Haven't tested Win95.)
- Report function now prints details to both the DetailLog and the DetailText item.

Update is here:
http://nsis.sourceforge.net/CABSetup_plug-in

To-do list includes Jason's suggestions such as:
- Progress bar/MSI like dialog.
- Split plugin into a 'make' plugin and a 'distributable' plugin.

Enjoy!

Duncan

JasonFriday13
8th June 2006 01:08 UTC

I did a little testing, and found out that adding files with a space in the path does not work.

This works: "$PROGRAMFILES\InstallShield\*.*"

This does not: "$PROGRAMFILES\Cisco Systems\*.*"

If you use a space in the path, you get a .DDF that lists all the folders in the $PROGRAMFILES directory.

And I just found a workaround:
StrCpy $R0 "$PROGRAMFILES\Cisco Systems\*.*"
CABSetup::BuildSetupFiles /NOUNLOAD /add=$R0 /s |

This does work :weird: . I have had a little trouble with this problem on my background text on my (now non-existent) Shield.dll plugin. Having quotes or apostrophes in the text caused it to be truncated. I got round it by StrCpy'ing the text to a variable, then using the variable instead of the quoted path as an argument.

Hope this information helps.

Mr Inches
8th June 2006 10:54 UTC

Thanks Jason - well spotted and thanks for testing!

I have had a look at this and I don't think it is just a CABSetup bug, but rather an effect of how NSIS/ExDLL passes the parameters to the plugin.

Passing a parameter to a plugin with any comment start ('#' or ';') or whitespace characters (and probably other NSIS special characters such as quotes and apostrophes), that are not in a variable, causes the parameter to terminate at that character, thereby splitting it e.g. /add="$PROGRAMFILES\Cisco Systems" ends up as two parameters /add="$PROGRAMFILES\Cisco" and "Systems" when returned by popstring. This appears to confuse my ScanDir code, as the path is not valid = bug for me to fix.

An alternative fix to the one Jason proposed is to enclose any parameter that could have this problem in quotes, including the switch, like this:

CABSetup:BuildSetupFiles "/add=$PROGRAMFILES\Cisco Systems" /s |

This ensures that the entire parameter makes it to CABSetup in one piece. This should fix the NSIS/ExDLL side of things.

I will update my example scripts to use this form for the next release and have a look at the ScanDir code to see why it would scan every subdirectory of the parent if the initial path was invalid/non-existent.

Duncan

Mr Inches
8th June 2006 14:11 UTC

I checked my code and there isn't actually a bug in the ScanDir code, but it's behaviour may not be entirely clear.

The ScanDir function expects to be passed a single string with a complete path and file specification within it such as "C:\DATA\File*.*" or "C:\MyApp\Source Files\MyApp.exe".

The file specification is always expected to be present and the "search subdirectories" option can be used with either form, allowing you for example to include all of the files called 'Readme.txt' in all subdirectories of the source directory.

This approach also allows the cabinet file to hold relative paths for the subsequent extraction without including any the developer's containing source folder structure.

As a result of the parameters being broken up incorrectly in the above example, ScanDir thinks it is to add all of the files matching the filespec "Cisco" in all of the subdirectories of "C:\Program Files\". "Systems" becomes a parameter that CABSetup ignores since it doesn't recognise it.

Applying either of the suggested fixes avoids the problem, without a need to change the CABSetup plugin code.

Duncan

Mr Inches
3rd October 2006 23:15 UTC

I have released a new version of the CABSetup plug-in, 1.2.3.0.

This version includes:
- a progress bar (finally!),
- a supporting function to allow the update of section sizes to include the uncompressed size of files within a dataset (DataSetSize)
- ability to disable compression for a dataset.

The BuildSetupFiles function no longer supports the DIRECTIVEFILE parameter. Please use the new DATASET and DATASETPATH parameters instead of DIRECTIVEFILE.

The example scripts and CABSetup.txt in the package show and explain how the new functionality is used.

Duncan

Mr Inches
12th December 2006 10:13 UTC

I have released a new version of the CABSetup plug-in, 1.3.0.0.

This version includes:
- ability to cancel the extraction of a dataset
- support for showing the extraction rate and estimated completion time for a dataset (developed from code in the Inetc and InetLoad plug-ins - thanks Takhir)
- support for resuming the extraction of a partially extracted dataset without extracting all of the files already present.

As usual, the example scripts and CABSetup.txt file in the package have been updated to use/describe these new functions.

Duncan

Altair68
17th April 2008 00:09 UTC

Bitten by 32bits
I'm very glad you created this plug-in but while it helped get past the 2GB limit I've hit a snag.

It looks like there is a problem with a dataset that is larger than 4GB.

DataSetSize uses the report file to determine the uncompressed size of the data. This would be fine, but the report file's data appears to be 32bit.

This is the report from my dataset which is about 4.89GB (5256994676 bytes according to windows):

MakeCAB Report: Tue Apr 15 21:25:04 2008

Total files:            860
Bytes before:   962,036,773
Bytes after:  -1,606,658,832
After/Before:           279.44% compression
Time:                  2255.66 seconds ( 0 hr 37 min 35.66 sec)
Throughput:             416.50 Kb/second

Mr Inches
20th April 2008 04:01 UTC

I thought the 32-bit limit on the dataset size was imposed by MAKECAB and the cabinet format itself.

As I noted in the Compatibility Issues in cabsetup.txt (point 4), MAKECAB crashes if you give it more than 4GB of (uncompressed) data (Vista and 2008's MAKECABs don't crash, but they still bail out...). I am surprised your compression run with 4.89GB didn't do this - I am interested in how you managed it! When I did my testing with more than 4GB of data, MAKECAB crashed out each time right at the end of the run; maybe it has something to do with how well the data compresses...

The only answer I found was to create a second dataset (currently you have to work out where to divide your data, although I have looked into what might be done to assist that too, but that looks like it would get complex fast...)

The GetDataSetSize function was written to deal specifically with how MAKECAB reports the compressed data, including the storage of the data sizes as signed 32-bit integers - thus the special handling of negative numbers to reflect the actual data compressed (note the 'massive' compression ratio of 279% in your RPT file as MAKECAB itself tried to make sense of the negative 'Bytes after' figure). The .RPT file and the .INF should agree on these quantities too.

I am not sure I understand your derived arithmetic below, for datasets <= 4GB I have always had the correct 'Bytes before' figures; it was only the 'Bytes after' that seemed to be a signed 32-bit integer.

It would pretty simple to allow GetDataSetSize to perform a conversion to KB, MB, or GB for its return value; I will see what I can do.

Re the 64-bit compatibility, I believe the code is relatively 32-bit clean, I only used 64-bit code where I needed to deal with quantities that large (such as the progress calculations), the rest was written as a Win32 application.

The progress bar code has changed substantially since 1.3.0; in my development code I have added support for advancing the progress bar while large files are being extracted - so this system has been reworked - although it is probably still Win32 code.

Duncan

Altair68
20th April 2008 05:02 UTC

MAKECAB did crash the first time I tried it. But I reduced the media size to 512MB and it worked fine after that. I verified that it worked properly by doing a file/folder comparison with Araxis.

The numbers are "fuzzy" because I was using windows' properties dialog to get the totals for the files.

Since the number wrapped around again your code sees it as not needing special handling. This is why I thought it would be a good idea to use the INF file since the code would just have to add it all up instead of infer based on MAKECAB's bogus reporting. My original idea was to write a PERL script to *fix* the RPT file using the INF, but then I saw that it would still cause issues with the rest of the code.

I think the 64bit issue would be easily dealt with by rounding to KB. The code that uses the GetDataSetSize function would just need to be aware that it was in KB. Then you could support 4TB numbers which should take a decade it became an issue again. Do you think that not being byte accurate will cause problems elsewhere? I was thinking it would be OK to overstate it, but I haven't read through all of the code.

Thanks again for writing this very useful plugin. =)

- Altair

jozechu
23rd May 2008 15:55 UTC

Hi All,

Does anyone experienced problems to use CABSetup::Extract on Vista??

It always file when try to extract from the CAB.
I'm using a CABSize lower than 4Gb ...

thanks,
Joze

Mr Inches
24th May 2008 16:21 UTC

Hi Joze

I haven't done as much testing on Vista as Windows XP/2003 and 2000, but I have successfully used the plug-in under Vista.

Please provide some more information about what fails; does the plugin return a status code (it leaves it on the stack so you have to pop it), does it crash or fail silently?

Also, what settings did you use when you created the dataset (compression method, disk/cabinet sizes)?

Any information you can provide may help isolate what is not working.

Duncan

jozechu
24th May 2008 17:39 UTC

Hi Duncan,

Well, I'm experiencing the problems always on Vista.

I've tried on both 64 and 32 versions. Fails in both of them.

Playing with the compression format(LZX, MSZIP and None) makes the scenario get even stranger!

Get an AMD Vista64 working perfectly with them all, another Intel Dual Vista64 which only works the None compressed version and the Quad Xeon Vista32 which get intermittent fails(cant figure out why!)

I've get always a Code 13 onto the Stack, that makes me think about a corrupted or lack of permissions ... but I can assure you that it doesn't!

The program always return me a "-1" so the code never crash, and NSIS installation exit as a "Fail extracting cabinets".

They same packages (all the different versions of compression) works perfectly onto any XP configuration.

Seems that the Extract together with the PSAPI.dll makes something strange on vista, but it's only an Idea...

Hope it helps!!
Tell me if you want me to elaborate more...

Cheers,
Joze

Mr Inches
1st June 2008 02:37 UTC

Hi Joze

Sorry it took me a while to get back to this, I had to build a Vista development environment to have a look at this (I do most of my development work under XP Professional).

It looks like the problem occurs with cabinets produced by Vista's MAKECAB; at least that is what I get on my Q6600s. I find this very odd given that the extraction is being done on the same platform that they are generated on... Maybe they are doing something funky with Unicode (CABSetup is not Unicode aware yet, as VS2005 seems keen to remind me).

I am not sure how easy this will be to fix and my current development build (with the GetDataSetSize from the INF file functionality requested by Altair68) is crashing on Vista so there is not much point releasing it yet.

The only solution I can offer at the moment is to build your dataset using a previous version of MAKECAB (either build it on an XP/2003 box or copy MAKECAB in from one of these environments to the same directory you run makesetup from (so it overrides the OS one); they seem to be OK.

Duncan

Mr Inches
2nd June 2008 13:03 UTC

I have released a new version of the CABSetup plug-in, 1.3.1.0.

This version is specifically to address the issue of installers crashing when using Extract on Vista; this should no longer occur.

No other functions have been affected or changed.

Duncan

jozechu
3rd June 2008 16:47 UTC

Hi Duncan,

I've just come for a long Weekend.
First want to thank you for your excellent work and hope my explanation helped you to find out what's going on inthere ...

I did a fast check and now works perfectly on my worst scenario.

Now I'm readjusting the system to come back to the CAB setup instead of ZIP files.

Congrat mate!

thanatos83
27th December 2009 01:13 UTC

Hi, need a little help...

I need to set the size of any cab to (2GB) but if i set 2GB in bytes in "disksize" then makecab crash...

So i need a multiple of 512 but, what number is the proper to number to set in disksize to get 2GB of any cab???.

:S

thanatos83
31st December 2009 12:50 UTC

Hi, one question...

i'm reading the CABSetup.txt and i don't understand this part:

Example Build Process (assuming that installer is complete) ----------------------------------------------------------- 1) Build and test the NSIS installer package. Ensure that it produces "_setup.exe" as its output file.

2) Place all of the files to be packaged into a single location. Include subdirectories if required, CabSetup will recreate the same directory structure (under the source path) during the Extract process.

3) Tailor makesetup.nsi to include the required files from the source location in the cabinet set. Remember to allow enough room for the NSIS installer on disk 1 - you may need to repeat this step after creating the "setup.exe" file in step 6. Do not include the setup.exe in the package source files.

4) Compile makesetup.nsi and run the resulting makesetup.exe. This will create the cabinet set containing the data files.

5) Place the "_setup.exe" file created in step 1 in the same directory as stubsetup.nsi.

6) Compile the stubsetup.nsi file. This should create a "setup.exe" file containing the InstStub.exe and your installer from step 1.

7) Copy the contents of the first installation disk to the target installation media and copy the "setup.exe" file to the same disk.

8) Copy the remaining cabinets to the appropriate target installation media.

9) Ensure that the installation media have volume labels that match the labels generated by CABSetup e.g. SETUP#1. Check the [disk list] section of the setup.inf file in the directory with the cabinet files in it for the disk-number-to-label mapping.

10) Create the setup media i.e copy the cabinet files to floppy disks or burn the source media CDs/DVDs.

daofeihu110
29th March 2013 08:56 UTC

I got this problem
when i run the _setup_multisets,i got this problem

get data Set Size for set1 failed with code -1

and i am not sure what cause it

i found the temp file $PLUGINSDIR have got the right file

T.Slappy
6th April 2013 17:51 UTC

Are any files processed?

Mr Inches
3rd May 2013 11:28 UTC

A return value of -1 from GetDataSetSize means that the function couldn't find the line specifying the dataset size in the report file.

Make sure that you are passing it the path to the .RPT file generated by MAKECAB and not the .INF file.

If you are sure you are passing the correct file, please post or attach a copy of the file for analysis.

Duncan